Cluster management method, device and equipment, readable storage medium and program product
By obtaining the status metric data and load requirements of the Kafka cluster and adjusting storage shards, the problem of unbalanced cluster load is solved, and the user experience and system stability are improved.
Patent Information
- Application Number
- CN202411226587.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-17
AI Technical Summary
The Kafka cluster has unbalanced load due to uneven distribution of storage shards, which affects the user experience.
By obtaining cluster status indicator data, generating cluster status description information, adjusting storage shards according to load requirements, and realizing load balancing of storage nodes.
The load balancing of Kafka cluster is realized, which improves user experience, reduces the response time of storage nodes, and improves system stability.
Smart Images

Figure CN119155301A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a cluster management method, apparatus, device, readable storage medium and program product. Background Art
[0002] Kafka is a high-throughput distributed publish-subscribe messaging system, mainly used in large data stream storage scenarios. A Kafka cluster includes one or more storage nodes (Broker), and a storage node can include one or more storage objects (Topic). Message consumers can get messages from Topics, and message producers can write messages to Topics. A Topic is usually composed of multiple storage shards (Partitions).
[0003] In actual applications, since the storage shards contained in multiple storage nodes are not exactly the same, the resources occupied by different storage nodes (for example, network resources, storage resources, etc.) are usually different, which can easily lead to load imbalance in the Kafka cluster and affect the user experience of the cluster. Summary of the invention
[0004] The embodiments of the present application provide a cluster management method, apparatus, device, readable storage medium and program product, which can determine storage shard adjustment information based on cluster status indicator data, which is conducive to load adjustment of storage nodes, thereby achieving load balancing of the cluster and improving the user experience.
[0005] On the one hand, an embodiment of the present application provides a cluster management method, the cluster management method comprising:
[0006] Acquire cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes load information of each storage shard in the message storage cluster and load information of each storage node;
[0007] Generate cluster state description information of the message storage cluster according to the cluster state indicator data, wherein the cluster state description information is used to indicate load information of upper nodes of each storage shard in the message storage cluster;
[0008] Storage shard adjustment information is generated based on the cluster status description information and the cluster load requirement of the message storage cluster, and the storage shard adjustment information is sent to the target storage node in the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs, so as to adjust the load information of the target storage node.
[0009] On the one hand, an embodiment of the present application provides another cluster management method, the cluster management method comprising:
[0010] Receiving storage shard adjustment information sent by a management node to a target storage node; the management node and the target storage node are included in a message storage cluster, the message storage network includes multiple storage nodes, and the storage node includes one or more storage shards;
[0011] According to the storage shard adjustment information, the partition to which the target storage shard belongs is adjusted;
[0012] Among them, the storage shard adjustment information is generated by the management node based on the cluster status description information and the cluster load requirement of the message storage cluster, the cluster status description information is generated based on the cluster status indicator data of the message storage cluster, and the cluster status description information is used to indicate the load information of the upper-level nodes of each storage shard in the message storage cluster; the cluster status indicator data of the message storage cluster is obtained by the management node, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node.
[0013] Accordingly, an embodiment of the present application provides a cluster management device, the cluster management device comprising:
[0014] An acquisition unit, used to acquire cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes load information of each storage shard in the message storage cluster and load information of each storage node;
[0015] A generating unit, configured to generate cluster state description information of the message storage cluster according to the cluster state indicator data, wherein the cluster state description information is used to indicate load information of upper nodes of each storage shard in the message storage cluster;
[0016] A sending unit is used to generate storage shard adjustment information according to the cluster status description information and the cluster load requirement of the message storage cluster, and send the storage shard adjustment information to the target storage node in the message storage cluster, wherein the storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs, so as to adjust the load information of the target storage node.
[0017] Accordingly, an embodiment of the present application provides another cluster management device, which includes:
[0018] A receiving unit, configured to receive storage shard adjustment information sent by a management node to a target storage node; the management node and the target storage node are included in a message storage cluster, the message storage network includes multiple storage nodes, and the storage node includes one or more storage shards;
[0019] An adjustment unit, configured to adjust the partition to which the target storage slice belongs according to the storage slice adjustment information;
[0020] Among them, the storage shard adjustment information is generated by the management node based on the cluster status description information and the cluster load requirement of the message storage cluster, the cluster status description information is generated based on the cluster status indicator data of the message storage cluster, and the cluster status description information is used to indicate the load information of the upper-level nodes of each storage shard in the message storage cluster; the cluster status indicator data of the message storage cluster is obtained by the management node, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node.
[0021] Accordingly, an embodiment of the present application provides a computer device, the computer device comprising:
[0022] a processor suitable for implementing a computer program;
[0023] A computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned cluster management method.
[0024] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned cluster management method.
[0025] Accordingly, an embodiment of the present application further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned cluster management method.
[0026] In the present application, the management node in the message storage cluster can obtain the cluster status indicator data of the message storage cluster, and generate cluster status description information based on the cluster status indicator data, and the cluster status description information can indicate the load information of objects at all levels in the message storage cluster (for example: storage nodes, storage shards, etc.); storage shard adjustment information can be generated based on the cluster status description information and cluster load requirements, thereby realizing the adjustment of the partition to which the storage shard belongs. Through the cluster management method provided by the embodiment of the present application, cluster status description information can be generated based on cluster status indicator data, and the cluster status description information can more comprehensively reflect the load status of objects at all levels in the message storage cluster, which is conducive to the subsequent adjustment of the load status of the message storage cluster; storage shard adjustment information can be generated based on the cluster status description information and cluster load requirements, thereby realizing the adjustment of the partition to which the storage shard belongs, so as to realize the load information adjustment of the storage node, and then the load balancing of the message storage cluster can be realized. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0028] Figure 1 It is a schematic diagram of the system architecture of a cluster management system provided by an embodiment of the present application;
[0029] Figure 2 It is a flow chart of a cluster management method provided in an embodiment of the present application;
[0030] Figure 3 It is a schematic diagram of a cluster status indicator data acquisition method provided in an embodiment of the present application;
[0031] Figure 4 is a schematic diagram of cluster status description information provided in an embodiment of the present application;
[0032] Figure 5 is a schematic diagram of a cluster management method provided in an embodiment of the present application;
[0033] Figure 6 It is a flowchart of another cluster management method provided in an embodiment of the present application;
[0034] Figure 7 It is a flowchart of another cluster management method provided in an embodiment of the present application;
[0035] Figure 8It is a structural block diagram of a cluster management device provided in an embodiment of the present application;
[0036] Fig. 9 is a structural block diagram of another cluster management device provided in an embodiment of the present application;
[0037] Fig.10 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0039] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.
[0040] The Kafka cluster includes one or more storage nodes, a storage node may include one or more storage objects, and a storage object may include one or more storage shards. In a Kafka cluster, traffic fluctuations, the creation and deletion of storage objects, and the startup and demise of storage nodes all occur dynamically. These changes may cause various resources (for example, network resources) to be unevenly distributed among the storage nodes in the cluster, thereby affecting the business stability of the cluster. For example: a storage node includes a first storage shard and a second storage shard, and there are many access requests for the first storage shard and the second storage shard. At this time, the network load of the storage node is greater than that of other storage nodes, and the cluster is unevenly loaded. In addition, due to the large network load of the storage node, the response time of the storage node may be longer, which is likely to affect the user experience.
[0041] Based on this, the embodiment of the present application provides a cluster management method, which can obtain cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node; the cluster status description information of the message storage cluster is generated according to the cluster status indicator data, and the cluster status description information is used to indicate the load information of the upper nodes of each storage shard in the message storage cluster; according to the cluster status description information and the cluster load demand of the message storage cluster, the storage shard adjustment information is generated, and the storage shard adjustment information is sent to the target storage node in the message storage cluster, and the storage shard adjustment information is used to indicate the target storage node to adjust the partition to which the target storage shard belongs, so as to adjust the load information of the target storage node. Through the method provided by the embodiment of the present application, the storage shard adjustment information can be determined according to the cluster status indicator data, which is conducive to realizing the load adjustment of the storage node, thereby realizing the load balancing of the cluster and improving the user experience.
[0042] The architecture of the cluster management system provided in the embodiment of the present application will be introduced below with reference to the accompanying drawings.
[0043] See also Figure 1 , which is a schematic diagram of the system architecture of a cluster management system provided by an embodiment of the present application, wherein the cluster management system includes a message acquisition device 101, a message production device 102 and a message storage cluster 103, wherein the message storage cluster 103 includes a management node 1031 and multiple storage nodes (i.e. Figure 1 , storage node 1032, storage node 1033, ..., storage node 103N, where N is an integer greater than 3). Wherein:
[0044] The message acquisition device 101 is a device for acquiring messages or data from a message storage cluster, and the message acquisition device 101 may include multiple specific message acquisition devices. The message acquisition device 101 may be a handheld device with communication function (such as a smart phone, a tablet computer), a computing device (such as a personal computer (PC)), a vehicle terminal, an intelligent voice interaction device, a wearable device or other intelligent device, but is not limited thereto.
[0045] The message production device 102 is a device that stores messages or data in a message storage cluster, and the message production device 102 may include multiple specific message production devices. The message production device 102 may be a terminal device or a server. When the message production device 102 is a terminal device, the message production device 102 may be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc., but is not limited thereto. When the message production device 102 is a server, the message production device 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0046] The message storage cluster 103 may include a management node 1031 and multiple storage nodes, and each storage node may include one or more storage objects. Each storage object may include one or more storage shards, and the storage shards store specific messages or data. The management node may determine the cluster state description information of the message storage cluster, and adjust the load state of the message storage cluster according to the cluster state description information. The message storage cluster 103 may be a Kafka cluster or other message storage clusters.
[0047] The following will explain in detail Figure 1 The working principle of the cluster management system shown:
[0048] The message storage cluster 103 includes storage nodes 1032 and 1033. Storage node 1032 includes storage shard 1 and storage shard 2. The data in storage shard 1 and storage shard 2 are generated by the message production device 102 and sent to the message storage cluster. The message acquisition device 101 can acquire data from storage shard 1 and storage shard 2.
[0049] The management node 1031 may obtain cluster status indicator data of the message storage cluster 103, which may include load information of storage shard 1, storage shard 2, storage node 1032, and storage node 1033, and the load information may include network load data (e.g., network bandwidth). The management node 1031 may generate cluster status description information of the message storage cluster based on the cluster status indicator data, and the cluster status description information may indicate the load information of the upper nodes (i.e., storage nodes, message storage clusters) of each storage shard in the message storage cluster 103.
[0050] Management node 1031 can generate storage shard adjustment information based on the cluster status description information and the cluster load requirement of the message storage cluster. Assuming that the cluster status description information indicates that the network load of storage node 1032 is high, and the network loads of storage shard 1 and storage shard 2 are also high, and the cluster load requirement of the message storage cluster is "network load balance between storage nodes", management node 1031 can generate storage shard adjustment information, and the storage shard adjustment information can indicate "transfer storage shard 2 to storage node 1033 for storage". Management node 1031 can send the storage shard adjustment information to the target storage nodes in the message storage cluster (i.e., storage nodes 1032 and 1033, Figure 1 Only the process of sending the storage shard adjustment information to the storage node 1032 is shown). The storage node 1032 and the storage node 1033 can adjust the partition to which the storage shard 2 belongs (that is, the storage shard 2 is stored in the storage node 1033), thereby reducing the network load of the storage node 1032 and achieving load balancing of the message storage cluster. Through the cluster management method provided in the embodiment of the present application, the storage shard adjustment information can be determined according to the cluster status indicator data, which is conducive to realizing the load adjustment of the storage node, and then realizing the load balancing of the Kafka cluster, and improving the user experience.
[0051] It is understandable that the architecture diagram of the cluster management system described in the embodiment of the present application is to more clearly illustrate the cluster management method of the embodiment of the present application, and does not constitute a limitation on the cluster management method provided in the embodiment of the present application. For example, the cluster management method provided in the embodiment of the present application can be executed by the management node 1031, and can also be executed by other devices that are different from the management node 1031 and can communicate with the storage nodes in the message storage cluster 103. It is known to those skilled in the art that Figure 1 The number of message acquisition devices 101, storage nodes, and message production devices 102 is only illustrative. Any number of devices and nodes can be configured according to business implementation needs. In addition, with the evolution of system architecture and the emergence of new business scenarios, the cluster management method provided in the embodiment of the present application is also applicable to similar technical problems.
[0052] It should be noted that the collection and processing of relevant data in this application (for example, cluster status indicator data of the message storage cluster) should be strictly in accordance with the requirements of relevant laws and regulations when applied in instances, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.
[0053] See also Figure 2 , Figure 2A flow chart of a cluster management method provided in an embodiment of the present application. The cluster management method can be implemented by the above-mentioned management node 1031, or by other devices that can implement the cluster management method. The following is an example of the cluster management method being implemented by the above-mentioned management node 1031. The flow of the cluster management method provided in the embodiment of the present application includes but is not limited to:
[0054] S201. Obtain cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes load information of each storage shard in the message storage cluster and load information of each storage node.
[0055] In an embodiment of the present application, the management node in the message storage cluster can obtain the cluster state indicator data of the message storage cluster, which may include multiple storage nodes, each storage node may include one or more storage shards, and the cluster state indicator data may include the load information of each storage shard in the message storage cluster and the load information of each storage node. The load information of the storage shard may include the load indicator data of each dimension of the corresponding storage shard (for example: the network bandwidth of the storage shard, the occupied computing resources, etc.), and the load information of the storage node may include the load indicator data of each dimension of the corresponding storage node (for example: node status, node capacity, the traffic of each storage object or storage shard in the node, etc.). In some cases, the cluster state indicator data may be expressed as Metrics. The message storage cluster may be a Kafka cluster or other message storage clusters. Through the method provided in an embodiment of the present application, the load information of each storage shard in the message storage cluster and the load information of the storage node may be obtained, which is conducive to the subsequent determination of the cluster state description information according to the cluster state indicator data.
[0056] In one embodiment, the specific implementation method of obtaining the cluster status indicator data of the message storage cluster can be: obtaining the node status indicator data of each storage node in the message storage cluster, and the shard status indicator data of each storage shard; the node status indicator data includes one or more of the network traffic data, computing resource occupancy data, node capacity data, and storage resource description data of the corresponding storage node, and the shard status indicator data includes one or more of the network traffic data, computing resource occupancy data, and storage resource occupancy data of the corresponding storage shard; generating the cluster status indicator data of the message storage cluster, the cluster status indicator data includes the node status indicator data of each storage node and the shard status indicator data of each storage shard. The management node can obtain the node status indicator data of each storage node in the message storage cluster, and the shard status indicator data of each storage shard.
[0057] The node status indicator data of the storage node may include one or more of the network traffic data, computing resource occupancy data, node capacity data, and storage resource description data of the storage node, wherein the network traffic data of the storage node refers to the traffic indicator data associated with the storage node, such as: the network import and export traffic data of the storage node, the network bandwidth corresponding to the storage node, etc. The computing resource occupancy data refers to the number of computing resources occupied by the storage node, such as: the CPU utilization rate of the storage node, etc. The node capacity data refers to the indicator data of the node storage capacity of the storage node, such as: the remaining capacity of the storage node is 20%. The storage resource description data refers to the storage indicator data associated with the storage node, such as: the amount of disk resources occupied by the storage node, the amount of storage resources occupied by the replicated storage objects in the storage node, etc. It should be noted that, in some cases, the node status indicator data of the storage node may also include node status information, which is used to indicate the current state of the storage node (for example: whether it is in a "survival" state).
[0058] The shard status indicator data of the storage shard may include one or more of the network traffic data, computing resource occupancy data, and storage resource occupancy data of the corresponding storage shard, wherein the network traffic data of the storage shard refers to the traffic indicator data associated with the storage shard, such as: network import and export traffic data for the storage shard, etc. The computing resource occupancy data refers to the number of computing resources occupied by the storage shard, such as: CPU utilization, etc. The storage resource occupancy data refers to the number of storage resources occupied by the storage shard.
[0059] After the management node obtains the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard, the cluster status indicator data of the message storage cluster can be generated. Through the method provided in the embodiment of the present application, cluster status indicator data containing indicator data of multiple different dimensions can be determined, which is conducive to the subsequent determination of cluster status description information that can accurately reflect the load status of the message storage cluster based on the cluster status indicator data.
[0060] In one embodiment, the message storage cluster further includes a target storage object, in which the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard included in the corresponding storage node are stored; then the specific implementation method of obtaining the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard may be: each time the indicator data acquisition cycle is reached, the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard are obtained from the target storage object.
[0061] The message storage cluster may contain a target storage object (when the message storage cluster is a Kafka cluster, the target storage object may be a specific Topic), which stores the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard contained in the corresponding storage node. When the indicator data acquisition cycle is reached, the management node may obtain the node status indicator data of each storage node and the shard status indicator data of each storage shard from the target storage object. The indicator data acquisition cycle may be adaptively adjusted according to the indicator data collection cycle and application requirements. For example, if the indicator data collection cycle is 10 seconds, that is, the node status indicator data of each storage node and the shard status indicator data of each storage shard are collected every ten seconds and stored in the target storage object, the management node may obtain the required status indicator data from the target storage object every 10 seconds.
[0062] It should be noted that the message storage cluster may also include an indicator collection component, which is used to collect node status indicator data of each storage node in the message storage cluster and shard status indicator data of each storage shard, and store the collected data in the target storage object.
[0063] See also Figure 3 , which is a schematic diagram of a cluster status indicator data acquisition method provided by an embodiment of the present application. The message storage cluster includes a management node, multiple storage nodes (such as Figure 3 Storage node 1, storage node 2, ..., storage node M in the storage node, M is an integer greater than 2) and an indicator collection component (also called an indicator reporting component (Client MetricsReporter)), the management node can correspond to the above Figure 1 The management node 1031 in the management node. The management node may include a metrics acquisition component (also called a metrics monitoring component (Metrics Monitor)), and the metrics acquisition component includes a metrics acquirer (also called a metrics sampler (Metrics Sampler)). The metrics acquisition component may collect node status metrics data of each storage node in multiple storage nodes according to the metrics data collection cycle, as well as shard status metrics data of the storage shards contained in each storage node. The metrics acquisition component may store the collected node status metrics data and shard status metrics data in a target storage object (also called a metrics storage object (Metrics Topic)) ( Figure 3 In the example, the target storage object is contained in the storage node 1).
[0064] The indicator acquirer in the management node can obtain node status indicator data and shard status indicator data from the target storage object according to the indicator data acquisition cycle. The management node also includes a data analysis component (also called Analyzer) and an anomaly determination component (also called Detector). The data analysis component is used to analyze and process the node status indicator data and the shard status indicator data, generate cluster status indicator data of the message storage cluster, and perform subsequent processing to achieve load balancing of the message storage cluster. The anomaly determination component is used to perform anomaly determination based on the node status indicator data and the shard status indicator data to determine whether an anomaly occurs in the message storage cluster. Through the method provided in the embodiment of the present application, the node status indicator data and the shard status indicator data of the message storage cluster can be obtained in a timely manner, and the cluster status indicator data can be generated, which ensures the validity and accuracy of the cluster status indicator data, and is conducive to the subsequent cluster status indicator data to accurately achieve load balancing of the message storage cluster.
[0065] S202: Generate cluster state description information of the message storage cluster according to the cluster state indicator data, where the cluster state description information is used to indicate load information of upper nodes of each storage shard in the message storage cluster.
[0066] In an embodiment of the present application, the management node can perform index data aggregation processing according to the cluster state index data to generate cluster state description information of the message storage cluster, and the cluster state description information can be used to indicate the load information of the upper nodes of each storage shard in the message storage cluster. The load information of the upper node can include the load index data of each dimension (for example, dimensions such as computing resources, storage resources, and network resources) of the corresponding upper node. In some cases, the cluster state description information can also be referred to as a cluster snapshot or a cluster load model (Cluster Model). The cluster state description information can intuitively reflect the load state of each layer of nodes in the message storage cluster at a certain moment. The upper node of the storage shard in the message storage cluster may include storage nodes, storage nodes, clusters, etc. The cluster state description information can also intuitively reflect the load state of each layer of nodes in the message storage cluster and the association relationship between each layer of nodes. Through the method provided in the embodiment of the present application, the cluster state description information that directly reflects the load state of each layer of nodes in the cluster can be determined, which is conducive to determining accurate storage shard adjustment information according to the cluster state description information in the future, thereby realizing the load state adjustment of the cluster.
[0067] In one embodiment, a specific implementation method for generating cluster status description information of a message storage cluster based on cluster status indicator data may be: determining the upper node of the storage node to which each storage shard belongs based on the cluster topology information of the message storage cluster, the cluster topology information is used to indicate the hierarchical relationship among storage racks, storage nodes, and storage shards in the message storage cluster, and the upper node of the storage node includes the storage rack; the management node may obtain the cluster topology information of the message storage cluster, and the cluster topology information is used to indicate the association relationship between nodes at each layer in the message storage cluster. The message storage cluster includes a storage rack, a storage rack (Rack) may include one or more storage nodes, and a storage node may include one or more storage shards. Based on the cluster topology information, the upper node of the storage node to which each storage shard belongs may be determined, and the upper node of the storage node includes the storage rack.
[0068] The load information of each upper-level node is determined based on the load information of the storage nodes and storage shards contained in each upper-level node; the management node can determine the load information of each upper-level node based on the load information of the storage nodes and storage shards contained in each upper-level node (the load information may include network traffic data, computing resource occupancy data, storage resource occupancy data, etc.). For example: the upper-level node is storage rack 1, storage rack 1 contains storage node 1 and storage node 2, storage node 1 contains storage shard 1, and the network traffic data of storage shard 1 is 20M, the network traffic data of storage node 1 is 35M, and the network traffic data of storage node 2 is 5M, then the network traffic data of storage rack 1 can be determined to be 40M. The same method can be used to determine the load data of other dimensions.
[0069] The cluster state description information is generated according to the load information of each upper node, the load information of each storage node, and the load information of each storage shard; the management node can generate the cluster state description information according to the load information of each upper node, the load information of each storage node, and the load information of each storage shard. The cluster state description information can also include the overall load information of the cluster. It should be noted that in the actual application process, the management node can directly generate the cluster state description information based on the cluster state indicator data and the cluster topology information, and can also update the already generated cluster state description information based on the cluster state indicator data. Through the method provided in the embodiment of the present application, the load information of the upper node can be determined according to the load information of the lower node and the cluster topology information, so that the cluster state description information that can reflect the load state of the entire cluster can be obtained, which is conducive to improving the accuracy of determining the storage shard adjustment information according to the cluster state description information.
[0070] See also Figure 4, which is a schematic diagram of cluster status description information provided by an embodiment of the present application. Figure 4 As shown, the cluster status description information includes the topology information of each layer of nodes in the message storage cluster, and the load information of each layer of nodes. Specifically, the "cluster" node includes "storage rack 1" and "storage rack 2"; "storage rack 1" includes "storage node 1" and "storage node 2", and "storage rack 2" includes "storage node 3" and "storage node 4"; "storage node 1" includes "storage shard 1", "storage shard 2", and "storage shard 3", "storage node 2" includes "storage shard 4", "storage shard 5", and "storage shard 6", "storage node 3" includes "storage shard 7", "storage shard 8", and "storage shard 9", and "storage node 4" includes "storage shard 10", "storage shard 11", and "storage shard 12".
[0071] The management node can obtain cluster status indicator data, which may include node status indicator data of "storage node 1", "storage node 2", "storage node 3", "storage node 4", and shard status indicator data of "storage shard 1" to "storage shard 12". The management node can determine the upper-level nodes corresponding to each storage node based on the cluster topology information, and determine the load information of each upper-level node based on the load information of the storage nodes and storage shards contained in each upper-level node. For example: "Storage rack 1" contains "storage node 1" and "storage node 2", the computing resource occupancy data in the load information of "storage node 1" (that is, the CPU occupancy rate is 5%), and the computing resource occupancy data in the load information of "storage node 2" (that is, the CPU occupancy rate is 60%), then it can be determined that the computing resource occupancy data in the load information of "storage rack 1" is 65%. Through the method provided in the embodiment of the present application, the load status of each layer of nodes in the cluster can be intuitively reflected.
[0072] S203. Generate storage shard adjustment information based on the cluster status description information and the cluster load requirement of the message storage cluster, and send the storage shard adjustment information to the target storage node in the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs, so as to adjust the load information of the target storage node.
[0073] In the embodiment of the present application, the management node can generate storage shard adjustment information according to the state description information and the cluster load demand of the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs. The partition to which the target storage shard belongs can refer to the storage node to which the target storage shard belongs, or can refer to the disk partition to which the target storage shard belongs. The management node can send the storage shard adjustment information to the target storage node so that the target storage node adjusts the storage location of the target storage shard, thereby adjusting the load information of the target storage node. For example: the partition to which the target storage shard belongs is the target storage node, and the network traffic data of the target storage node is too high, then the management node can generate storage shard adjustment information and send it to the target storage node so that the target storage shard stored in the target storage node is transferred to other storage nodes, thereby reducing the network traffic data of the target storage node, thereby reducing the delay of the target storage node in responding to the access request, and improving the user experience. Through the method provided in the embodiment of the present application, accurate storage shard adjustment information can be determined, which is conducive to realizing the load balancing of the storage nodes in the message storage cluster and improving the user experience for the message storage cluster.
[0074] It should be noted that the time interval between the management node generating cluster status description information and generating storage shard adjustment information based on the cluster status description information can be adjusted according to different application requirements. For example, the management node can obtain cluster status indicator data every 10 seconds and generate cluster status description information, and the management node can generate storage shard adjustment information based on the cluster status description information every 5 minutes, that is, the management node can adjust the partition of the storage shard in the message storage cluster more frequently to achieve dynamic load balancing of the message storage cluster, and can also respond to various situations in the message storage cluster (for example, business surge, business contraction, etc.) in a more timely manner.
[0075] In one embodiment, the cluster load demand includes at least one of the following: shard partition unification demand, single node resource balancing demand, multi-node resource balancing demand, storage resource balancing demand; then the specific implementation method of generating storage shard adjustment information according to the cluster state description information and the cluster load demand of the message storage cluster can be: taking the cluster load demand as the target, generating storage shard adjustment information about the target storage node according to the cluster state description information. The management node can determine the cluster load demand, and the cluster load demand can include one or more of the shard partition unification demand, single node resource balancing demand, multi-node resource balancing demand, and storage resource balancing demand. Among them, the shard partition unification demand refers to the unification (or same) of the rack partitions to which a storage shard and the replica shard of the storage shard belong, for example: the partition to which storage shard 1 belongs is "storage node 1", and the partition to which its replica shard belongs is "storage node 2", and "storage node 1" and "storage node 2" are included in "storage rack 1".
[0076] The single-node resource balancing requirement refers to a resource (e.g., capacity, computing resources, network resources, etc.) of a single storage node not exceeding the preset upper limit. For example, if the upper limit of the storage node's capacity usage is "storage capacity*0.8", if the capacity usage of a storage node exceeds the preset upper limit, the single-node resource balancing requirement is not met, and the storage shards in the storage node need to be partitioned and adjusted.
[0077] The multi-node resource balancing requirement refers to the balanced distribution of a resource in the cluster (e.g., storage resources, computing resources, network resources, etc.) among various storage nodes. For example, if a cluster contains two storage nodes, one of which occupies 80% of the computing resources and the other only occupies 20% of the computing resources, the storage shards in the storage nodes need to be partitioned and adjusted to achieve a balanced distribution of computing resources among the storage nodes.
[0078] The storage resource balancing requirement refers to the load balancing of a single storage node between different disks to avoid the storage node consuming too much time to read and write data. The management node can use the cluster load demand as a target and generate storage shard adjustment information about the target storage node based on the cluster state description information. Through the method provided in the embodiment of the present application, corresponding storage shard adjustment information can be generated based on the cluster load demand, which is conducive to realizing automatic load balancing of the message storage cluster. It does not require manual intervention, the process is simple and efficient, and the stability of the services provided by the message storage cluster can be effectively guaranteed.
[0079] It should be noted that the management node can determine the corresponding cluster load algorithm based on different cluster load requirements, and determine the storage shard adjustment information based on the cluster load algorithm and cluster status description information. Specifically, the cluster load algorithm corresponding to the shard partition unification requirement may include the "RackAwareGoal" algorithm and the "BrokerSetAwareGoal" algorithm; the cluster load algorithm corresponding to the single-node resource balancing requirement may include the "ReplicaCapacityGoal" algorithm, the "DiskCapacit yGoal" algorithm, the "NetworkInboundCapacityGoal" algorithm, the "NetworkOutboundCapacityGoal" algorithm, and the "CpuCapacityGoal" algorithm; the cluster load algorithm corresponding to the multi-node resource balancing requirement may include the "Disk UsageDistributionGoal" algorithm, the "NetworkInboundUsageDistributionGoal" algorithm, the "NetworkOutbo undUsageDistributionGoal" algorithm, the "CpuUsageDistributionGoal" algorithm, and the "LeaderBytesInDistribution" algorithm. "ionGoal" algorithm, "ReplicaDistributionGoal" algorithm, "LeaderReplicaDistributionGoal" algorithm, and "TopicReplicaDistributionGoal" algorithm; the cluster load algorithm corresponding to the storage resource balancing requirement may include the "Intra BrokerDiskCapacityGoal" algorithm and the "IntraBrokerDiskUsageDistributionGoal" algorithm. When the cluster load requirements include multiple types, the cluster load algorithm corresponding to the cluster load requirements can be used to process the cluster state description information to obtain storage shard adjustment information, so that after the partition adjustment of the storage shard is completed, the state of the message storage cluster reaches the state indicated by the multiple cluster load requirements. Through the method provided in the embodiment of the present application, the storage shard adjustment information can be determined so that the message storage cluster meets the cluster load requirement, thereby realizing automatic load balancing of the message storage cluster.
[0080] In one embodiment, the method provided by the embodiment of the present application can also realize the automatic elastic expansion of the message storage cluster. When the business demand increases, it is usually necessary to expand the message storage cluster. In general, a new storage node can be configured, and professional and technical personnel can transfer the storage shard to the new storage node to realize the expansion of the message storage cluster. However, this process usually takes a long time. The cluster management method provided by the embodiment of the present application can realize the automatic elastic expansion of the message storage cluster. Specifically, the message storage cluster configures the new storage node, and the management node can obtain the cluster status indicator data of the message storage cluster and generate cluster status description information. Storage shard adjustment information can be generated according to the cluster status description information and the cluster load demand (at this time, the cluster load demand can indicate the balanced distribution of resources of each node in the cluster). The storage shard adjustment information can indicate that the target storage shard is transferred to the new storage node, so as to realize the automatic expansion of the cluster; when the business shrinks, the method provided by the embodiment of the present application can also be used to automatically compress the message storage cluster. In addition, since the storage shard adjustment information in this application is adjusted for the partition of the target storage shard, and the amount of data in the storage shard is usually small, the method provided by this application can achieve "micro-batch" expansion, which can ensure that the expansion cost is almost matched with the growth of business traffic. Through the method provided in the embodiment of this application, the rapid elastic expansion of the message storage cluster can be achieved during business peaks; the surge in business traffic can be determined through cluster status indicator data, and automated resource management and dynamic scaling strategies can be performed in response to this situation, thereby ensuring the stability of the business of the message storage cluster during high traffic loads, and effectively ensuring that the user can still obtain a better service experience during business peaks.
[0081] See also Figure 5 , which is a schematic diagram of a cluster management method provided in an embodiment of the present application. Figure 5 It includes management nodes, multiple storage nodes (i.e. Figure 5 Storage node 1, storage node 2, ..., storage node M, where M is an integer greater than 2) and indicator collection components, the management node can correspond to the above Figure 1 The management node 1031 in the storage node can correspond to the above Figure 1 The index collection component can collect the node status index data of each storage node in the message storage cluster, as well as the shard status index data of the storage shards contained in each storage node. The index collection component can store the collected data in the target storage object ( Figure 5In the example, the target storage object is included in the storage node 1). The management node includes an indicator acquisition component, which includes an indicator acquirer. The indicator acquirer can acquire the node status indicator data of each storage node and the shard status indicator data of the storage shards included in each storage node from the target storage object, and generate cluster status indicator data. The management node can generate cluster status description information based on the cluster status indicator data and the cluster topology information of the message storage cluster. Figure 5 As shown, the cluster status description information includes that the storage rack includes storage node 0 and storage node 1, storage node 1 includes storage slice a and storage slice b, and the cluster status description information also includes the load information of each layer node. Figure 5 Not specifically shown.
[0082] The management node also contains a data analysis component (also called a scheduling decision maker), which can determine the cluster load requirements of the message storage cluster. Figure 5 The network load requirement (also called the network inbound goal), disk distribution requirement (also called the disk distribution goal), capacity requirement (also called the capacity coal), and node distribution requirement (also called the leader distribution goal) in the network load requirement. Among them, the network load requirement can correspond to the above-mentioned multi-node resource balancing requirement, the disk distribution requirement can correspond to the above-mentioned storage resource load requirement, the capacity requirement can correspond to the above-mentioned single-node resource balancing requirement, and the leader distribution requirement can correspond to the above-mentioned sharding and partitioning unification requirement. After determining the cluster load requirement, the cluster status description information can be periodically obtained; a cluster adjustment plan is generated according to the cluster load requirement and the cluster status description information, that is, the adjustment information generation component (also called the proposal generation component (Proposal Executor)) can generate storage sharding adjustment information according to the cluster adjustment plan.
[0083] The adjustment information generation component in the management node can send the storage shard adjustment information to the target storage node ( Figure 5 In the example, the adjustment information generation component sends the storage shard adjustment information to the storage node M) to adjust the load information of the target storage node, thereby achieving load balancing between the storage nodes in the message storage cluster. The method provided by the embodiment of the present application can realize automatic load balancing of the message storage cluster, improve the efficiency of cluster load balancing, and also help to improve the quality of services provided by the message storage cluster.
[0084] Through the cluster management method provided in the embodiment of the present application, cluster status indicator data of the message storage cluster can be obtained, and cluster status description information can be generated, so as to intuitively reflect the load status of each layer of nodes in the message storage cluster; storage shard adjustment information can be generated according to the cluster status description information, so as to achieve automatic load balancing of the message storage cluster without manual intervention, effectively reducing the time consumed by the load balancing process; storage shard adjustment information can be generated more frequently, so as to achieve minute-level in-situ smooth and rapid elastic expansion of the message storage cluster during business peaks, which improves the operation and maintenance efficiency by a hundred times compared with manual operations; the surge in business traffic can be determined through cluster status indicator data, and automated resource management and dynamic scaling strategies can be performed in response to the situation, so as to ensure the stability of the services provided by the message storage cluster during high traffic loads, effectively ensuring that users can still obtain a good service experience during business peaks.
[0085] See also Figure 6 , Figure 6 A flow chart of another cluster management method provided in an embodiment of the present application. The cluster management method can be Figure 1 The cluster management method may be implemented by the storage node in the embodiment of the present application, or may be implemented by other devices that can implement the cluster management method. The cluster management method is implemented by the above storage node as an example. The process of the cluster management method provided in the embodiment of the present application includes but is not limited to:
[0086] S601. Receive storage shard adjustment information sent by a management node to a target storage node; the management node and the target storage node are included in a message storage cluster, the message storage cluster includes multiple storage nodes, and the storage node includes one or more storage shards.
[0087] In an embodiment of the present application, a message storage cluster includes a management node and multiple storage nodes, each storage node may include one or more storage shards. The management node may perform the process of steps S201-S203 above, generate storage shard adjustment information, and send the storage shard adjustment information to a target storage node. The target storage node may receive the storage shard adjustment information sent by the management node. The target storage node may be one or more storage nodes.
[0088] Among them, the storage shard adjustment information is generated by the management node based on the cluster status description information and the cluster load demand of the message storage cluster. The cluster status description information is generated based on the cluster status indicator data of the message storage cluster. The cluster status description information is used to indicate the load information of the upper-level nodes of each storage shard in the message storage cluster. The cluster status indicator data of the message storage cluster is obtained by the management node, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node.
[0089] It should be noted that the storage nodes in the message storage cluster can generate node indicator information and shard indicator information of each storage shard contained in the storage node. The message storage cluster can include an indicator collection component, which can regularly obtain the required node status indicator data and shard status indicator data from the node indicator information and shard indicator information generated by each storage node.
[0090] S602: According to the storage shard adjustment information, adjust the partition to which the target storage shard belongs.
[0091] In an embodiment of the present application, the target storage node can adjust the partition to which the target storage shard belongs according to the storage shard adjustment information. The partition to which the target storage shard belongs may refer to the storage node to which the target storage shard belongs, or may refer to the disk partition to which the target storage shard belongs. For example: the target storage node contains the target storage shard, the network load of the target storage node is large, and the network load of the target storage node is mainly for the target storage shard. Assuming that the storage shard adjustment information instructs the target storage node to delete the target storage shard, the target storage node can delete the target storage shard it stores, thereby reducing the network load of the target storage node. Through the method provided in the embodiment of the present application, the partition adjustment of the target storage shard can be achieved, thereby achieving the load state adjustment of the target storage node, and then achieving load balancing between the storage nodes in the message storage cluster.
[0092] In one embodiment, the storage shard adjustment information indicates that the target storage shard is stored in the target storage node, the target storage shard is included in the first storage node, and the first storage node is included in the message storage cluster; then, according to the storage shard adjustment information, the specific implementation method of adjusting the partition to which the target storage shard belongs can be: receiving the storage shard transfer information sent by the first storage node, the storage shard transfer information includes the target storage shard; storing the target storage shard in the target storage node, and generating transfer operation response information; sending the transfer operation response information to the first storage node, the transfer operation response information is used to instruct the first storage node to delete the target storage shard included in the first storage node. The target storage node can receive the storage shard transfer information sent by the first storage node, and store the target storage shard in the target storage node. After the storage operation is completed, the target storage node can generate a transfer operation response information and send it to the first storage node, so that the first storage node deletes the target storage shard stored therein, thereby realizing the transfer of the target storage shard from the first storage node to the target storage node. Through the method provided in the embodiment of the present application, partition adjustment of storage shards can be achieved, which is conducive to achieving load balancing of the message storage cluster.
[0093] In one embodiment, the storage shard adjustment information indicates that the target storage shard is stored in the second storage node, the second storage node is included in the message storage cluster, and the target storage shard is included in the target storage node; then, according to the storage shard adjustment information, the specific implementation method of adjusting the partition to which the target storage shard belongs can be: generating storage shard transfer-out information, and sending the storage shard transfer-out information to the second storage node, the storage shard transfer-out information is used to instruct the second storage node to perform storage processing on the target storage shard; receiving the transfer-out operation response information sent by the second storage node, and deleting the target storage shard stored in the target storage node according to the transfer-out operation response information. When performing the transfer-out operation of the storage shard, the target storage node can generate the storage shard transfer-out information and send it to the second storage node. The second storage node can store the target storage shard according to the storage shard transfer-out information, and generate the transfer-out operation response information. After receiving the transfer-out operation response information, the target storage node can delete the target storage shard stored in the target storage node according to the transfer-out operation response information, thereby realizing the transfer of the target storage shard from the target storage node to the second storage node. Through the method provided in the embodiment of the present application, partition adjustment of storage shards can be achieved, which is conducive to achieving load balancing of the message storage cluster.
[0094] Through the cluster management method provided in the embodiment of the present application, the partition adjustment of the storage shards can be implemented according to the storage shard adjustment information, and then the load status of the storage node can be adjusted, which is conducive to achieving load balancing between the storage nodes in the message storage cluster. There is no need for manual intervention by the administrator, which saves resources and improves the efficiency of the message storage cluster in achieving load balancing, which is conducive to ensuring the stability of the message storage cluster, improving the quality of the services provided by the message storage cluster, and improving the user experience.
[0095] See also Figure 7 , Figure 7 A flow chart of another cluster management method provided in an embodiment of the present application. The cluster management method can be performed by the above Figure 1 The cluster management system in the embodiment can also be implemented by other systems that can implement the cluster management method. The cluster management system can include management nodes and target storage nodes. The cluster management method implemented by the above cluster management system is used as an example for explanation. The process of the cluster management method provided in the embodiment of the present application includes but is not limited to:
[0096] S701. The management node obtains cluster status indicator data of the message storage cluster.
[0097] In an embodiment of the present application, a message storage cluster may include a management node and multiple storage nodes, each storage node includes one or more storage objects, each storage object includes one or more storage shards, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node. The management node may obtain the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard from the target storage object, and generate the cluster status indicator data according to the node status indicator data and the shard status indicator data.
[0098] S702: The management node generates cluster state description information of the message storage cluster according to the cluster state indicator data.
[0099] In an embodiment of the present application, the management node can generate cluster state description information of the message storage cluster based on the cluster state indicator data, and the cluster state description information can be used to indicate the load information of the upper nodes of each storage shard in the message storage cluster. The upper nodes of the storage shards in the message storage cluster may include storage nodes, storage nodes, clusters, etc. The cluster state description information can intuitively reflect the load status of each layer of nodes in the message storage cluster and the association relationship between each layer of nodes. In some cases, the management node can generate the cluster state description information of the message storage cluster based on the cluster state indicator data and the cluster topology information of the message storage cluster, and can also use the cluster state indicator data to update the generated cluster state description information. Through the method provided in the embodiment of the present application, the cluster state description information that directly reflects the load status of each layer of nodes in the cluster can be determined, which is conducive to the subsequent determination of accurate storage shard adjustment information based on the cluster state description information, thereby realizing the load state adjustment of the cluster.
[0100] S703: The management node generates storage shard adjustment information according to the cluster state description information and the cluster load requirement of the message storage cluster.
[0101] In an embodiment of the present application, the management node can generate storage shard adjustment information based on the status description information and the cluster load demand of the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs. The partition to which the target storage shard belongs is the storage node to which the target storage shard belongs. In some cases, the management node can generate storage shard adjustment information based on the cluster status description information with the cluster load demand as the target. The cluster load demand may include one or more of the shard partition unification demand, the single-node resource balancing demand, the multi-node resource balancing demand, and the storage resource balancing demand. Through the method provided in the embodiment of the present application, the corresponding storage shard adjustment information can be generated based on the cluster load demand, which is conducive to realizing the automatic load balancing of the message storage cluster, does not require manual intervention, and the process is simple and efficient. It can also effectively ensure the stability of the services provided by the message storage cluster.
[0102] S704: The management node sends the storage shard adjustment information to the target storage node.
[0103] In an embodiment of the present application, the management node may send storage sharding adjustment information to a target storage node, which may be one or more storage nodes. For example: the storage sharding adjustment information is used to indicate that the target storage node is load balanced among different disks, so the management node may send the storage sharding adjustment information to the target storage node so that the target storage node adjusts the partition of the target storage shard. For another example: the storage sharding adjustment information is used to indicate that network resources are evenly distributed among multiple target storage nodes, so the management device may send the storage sharding adjustment information to multiple target storage nodes.
[0104] S705: The target storage node adjusts the partition to which the target storage shard belongs according to the storage shard adjustment information.
[0105] In an embodiment of the present application, the target storage node can adjust the partition to which the target storage shard belongs according to the storage shard adjustment information, thereby adjusting the load state of the target storage node, and further achieving load balancing among the storage nodes in the message storage node.
[0106] Through the cluster management method provided in the embodiment of the present application, cluster status indicator data of the message storage cluster can be obtained, and cluster status description information can be generated, so as to intuitively reflect the load status of each layer of nodes in the message storage cluster; storage shard adjustment information can be generated according to the cluster status description information, so as to achieve automatic load balancing of the message storage cluster without manual intervention, effectively reducing the time consumed by the load balancing process; storage shard adjustment information can be generated more frequently, so as to achieve rapid elastic expansion of the message storage cluster during business peaks, so that the message storage cluster has cloud-native software and hardware elasticity capabilities; the situation of a surge in business traffic can be determined through cluster status indicator data, and automated resource management and dynamic scaling strategies can be performed in response to the situation, so as to ensure the stability of the services provided by the message storage cluster during high traffic loads, effectively ensuring that users can still obtain a better service experience during business peaks.
[0107] See also Figure 8 , Figure 8 This is a structural block diagram of a cluster management device provided in an embodiment of the present application. The cluster management device can be set in the computer device provided in an embodiment of the present application, and the computer device can be the above-mentioned Figure 1 The management node 1031 in the cluster management system is shown. Figure 8 The cluster management device shown may be a computer program running in a computer device, and the cluster management device may be used to execute Figure 2 or Figure 7 Part or all of the steps in the cluster management method embodiment shown. Figure 8, the cluster management device may include the following units:
[0108] An acquisition unit 801 is used to acquire cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes load information of each storage shard in the message storage cluster and load information of each storage node;
[0109] A generating unit 802 is used to generate cluster state description information of the message storage cluster according to the cluster state indicator data, wherein the cluster state description information is used to indicate load information of upper nodes of each storage shard in the message storage cluster;
[0110] Processing unit 803 is used to generate storage shard adjustment information according to the cluster status description information and the cluster load requirement of the message storage cluster, and send the storage shard adjustment information to the target storage node in the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs so as to adjust the load information of the target storage node.
[0111] In one embodiment, when the acquisition unit 801 is used to acquire the cluster status indicator data of the message storage cluster, it is specifically used to perform the following steps:
[0112] Obtain node status indicator data of each storage node in the message storage cluster, and shard status indicator data of each storage shard; the node status indicator data includes one or more of network traffic data, computing resource occupancy data, node capacity data, and storage resource description data of the corresponding storage node, and the shard status indicator data includes one or more of network traffic data, computing resource occupancy data, and storage resource occupancy data of the corresponding storage shard;
[0113] Generate cluster status indicator data of the message storage cluster, wherein the cluster status indicator data includes node status indicator data of each storage node and shard status indicator data of each storage shard.
[0114] In one embodiment, the message storage cluster further includes a target storage object, in which the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard included in the corresponding storage node are stored; the acquisition unit 801 is used to acquire the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard, and is specifically used to perform the following steps:
[0115] Every time an indicator data acquisition cycle is reached, node status indicator data of each storage node in the message storage cluster and shard status indicator data of each storage shard are acquired from the target storage object.
[0116] In one embodiment, the generating unit 802 is used to generate the cluster state description information of the message storage cluster according to the cluster state indicator data, and is specifically used to perform the following steps:
[0117] Determine the upper node of the storage node to which each storage shard belongs according to the cluster topology information of the message storage cluster, wherein the cluster topology information is used to indicate the hierarchical relationship among the storage racks, storage nodes, and storage shards in the message storage cluster, and the upper node of the storage node includes the storage rack;
[0118] Determine the load information of each upper-layer node according to the load information of the storage nodes and storage slices contained in each upper-layer node;
[0119] The cluster status description information is generated according to the load information of each upper node, the load information of each storage node, and the load information of each storage shard.
[0120] In one embodiment, the cluster load requirement includes at least one of the following: shard partition unification requirement, single-node resource balancing requirement, multi-node resource balancing requirement, and storage resource balancing requirement; the processing unit 803 is used to generate storage shard adjustment information according to the cluster state description information and the cluster load requirement of the message storage cluster, specifically for performing the following steps:
[0121] Taking the cluster load requirement as a target, storage shard adjustment information about the target storage node is generated according to the cluster state description information.
[0122] Figure 8 The various units in the cluster management device shown can be individually or completely combined into one or several other units to constitute, or one (some) of the units can be further divided into multiple functionally smaller units to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the cluster management device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
[0123] In one embodiment, the program can be executed by running on a general-purpose computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 2 or Figure 7 A computer program for each step involved in part or all of the method shown in the figure is used to construct a computer program for each step involved in part or all of the method shown in the figure. Figure 8 The cluster management device shown in and the cluster management method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.
[0124] Through the cluster management device provided in the embodiment of the present application, the cluster status indicator data of the message storage cluster can be obtained, and the cluster status description information can be generated, so as to intuitively reflect the load status of each layer of nodes in the message storage cluster; the storage shard adjustment information can be generated according to the cluster status description information, so as to realize the automatic load balancing of the message storage cluster without manual intervention, and effectively reduce the time consumed by the load balancing process; the storage shard adjustment information can be generated more frequently, so as to realize the rapid elastic expansion of the message storage cluster during business peaks; the situation of a surge in business traffic can be determined through the cluster status indicator data, and automatic resource management and dynamic scaling strategies can be performed in response to the situation, so as to ensure the stability of the services provided by the message storage cluster during high traffic loads, and effectively ensure that the users can still obtain a better service experience during business peaks.
[0125] See also Fig. 9 , Fig. 9 This is a structural block diagram of another cluster management device provided in an embodiment of the present application. The cluster management device can be set in the computer device provided in an embodiment of the present application, and the computer device can be the above-mentioned Figure 1 The storage nodes in the cluster management system are shown. Fig. 9 The cluster management device shown may be a computer program running in a computer device, and the cluster management device may be used to execute Figure 6 or Figure 7 Part or all of the steps in the cluster management method embodiment shown. Fig. 9 , the cluster management device may include the following units:
[0126] A receiving unit 901 is configured to receive storage shard adjustment information sent by a management node to a target storage node; the management node and the target storage node are included in a message storage cluster, the message storage cluster includes multiple storage nodes, and the storage node includes one or more storage shards;
[0127] An adjustment unit 902 is used to adjust the partition to which the target storage slice belongs according to the storage slice adjustment information;
[0128] Among them, the storage shard adjustment information is generated by the management node based on the cluster status description information and the cluster load requirement of the message storage cluster, the cluster status description information is generated based on the cluster status indicator data of the message storage cluster, and the cluster status description information is used to indicate the load information of the upper-level nodes of each storage shard in the message storage cluster; the cluster status indicator data of the message storage cluster is obtained by the management node, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node.
[0129] In one embodiment, the storage shard adjustment information indicates that the target storage shard is stored in the target storage node, the target storage shard is included in the first storage node, and the first storage node is included in the message storage cluster; the adjustment unit 902 is used to adjust the partition to which the target storage shard belongs according to the storage shard adjustment information, specifically for performing the following steps:
[0130] Receiving storage shard transfer information sent by the first storage node, wherein the storage shard transfer information includes the target storage shard;
[0131] Storing the target storage slice in the target storage node and generating transfer operation response information;
[0132] The transfer-in operation response information is sent to the first storage node, where the transfer-in operation response information is used to instruct the first storage node to delete the target storage slice contained in the first storage node.
[0133] In one embodiment, the storage shard adjustment information indicates that the target storage shard is stored in a second storage node, the second storage node is included in the message storage cluster, and the target storage shard is included in the target storage node; the adjustment unit 902 is used to adjust the partition to which the target storage shard belongs according to the storage shard adjustment information, specifically for performing the following steps:
[0134] Generate storage shard transfer-out information, and send the storage shard transfer-out information to the second storage node, wherein the storage shard transfer-out information is used to instruct the second storage node to perform storage processing on the target storage shard;
[0135] Receive the transfer-out operation response information sent by the second storage node, and delete the target storage shard stored in the target storage node according to the transfer-out operation response information.
[0136] Fig. 9 The various units in the cluster management device shown can be individually or completely combined into one or several other units to constitute, or one (some) of the units can be further divided into multiple functionally smaller units to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the cluster management device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
[0137] In one embodiment, the program can be executed by running on a general-purpose computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 6 or Figure 7 A computer program for each step involved in part or all of the method shown in the figure is used to construct a computer program for each step involved in part or all of the method shown in the figure. Fig. 9 The cluster management device shown in and the cluster management method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.
[0138] Through the cluster management device provided in the embodiment of the present application, the partition adjustment of the storage shards can be implemented according to the storage shard adjustment information, and then the load status of the storage node can be adjusted, which is conducive to achieving load balancing between the storage nodes in the message storage cluster. There is no need for manual intervention by the administrator, which saves resources and improves the efficiency of the message storage cluster in achieving load balancing, which is conducive to ensuring the stability of the message storage cluster, improving the quality of the services provided by the message storage cluster, and improving the user experience.
[0139] Based on the above method and device embodiments, the present application embodiment provides a computer device. Fig.10 , Fig.10 A structural block diagram of a computer device provided in an embodiment of the present application. Fig.10 The computer device shown may correspond to the above Figure 1 The management node 1031 or storage node in. Fig.10The computer device shown includes at least a processor 1001, an input interface 1002, an output interface 1003, and a computer-readable storage medium 1004. The processor 1001, the input interface 1002, the output interface 1003, and the computer-readable storage medium 1004 may be connected via a bus or other means.
[0140] The computer-readable storage medium 1004 may be stored in a memory of a computer device. The computer-readable storage medium 1004 is used to store a computer program. The computer program includes program instructions (such as Fig.10 The processor 1001 is used to execute the computer program stored in the computer-readable storage medium 1004. The processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing the computer program, and is specifically suitable for loading and executing the computer program to implement the corresponding method flow or corresponding function.
[0141] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the computer device. In addition, a computer program suitable for being loaded and executed by a processor is also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (Non-Volatile Memory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.
[0142] In a specific implementation, the processor 1001 may load and execute a computer program stored in the computer-readable storage medium 1004 to implement the above-mentioned Figure 2 or Figure 6 or Figure 7 The corresponding steps in the cluster management method shown. Fig.10 The computer equipment shown corresponds to the above Figure 1 When the management node in the computer readable storage medium 1004 is used, in a specific implementation, the computer program in the computer readable storage medium 1004 is loaded by the processor 1001 and executes the following steps:
[0143] Acquire cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes load information of each storage shard in the message storage cluster and load information of each storage node;
[0144] Generate cluster state description information of the message storage cluster according to the cluster state indicator data, wherein the cluster state description information is used to indicate load information of upper nodes of each storage shard in the message storage cluster;
[0145] Storage shard adjustment information is generated based on the cluster status description information and the cluster load requirement of the message storage cluster, and the storage shard adjustment information is sent to the target storage node in the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs, so as to adjust the load information of the target storage node.
[0146] In one embodiment, when the computer program in the computer-readable storage medium 1004 is loaded and executed by the processor 1001 to obtain cluster status indicator data of the message storage cluster, it is specifically used to perform the following steps:
[0147] Obtain node status indicator data of each storage node in the message storage cluster, and shard status indicator data of each storage shard; the node status indicator data includes one or more of network traffic data, computing resource occupancy data, node capacity data, and storage resource description data of the corresponding storage node, and the shard status indicator data includes one or more of network traffic data, computing resource occupancy data, and storage resource occupancy data of the corresponding storage shard;
[0148] Generate cluster status indicator data of the message storage cluster, wherein the cluster status indicator data includes node status indicator data of each storage node and shard status indicator data of each storage shard.
[0149] In one embodiment, the message storage cluster further includes a target storage object, in which the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard included in the corresponding storage node are stored; when the computer program in the computer-readable storage medium 1004 is loaded and executed by the processor 1001 to obtain the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard, it is specifically used to perform the following steps:
[0150] Every time an indicator data acquisition cycle is reached, node status indicator data of each storage node in the message storage cluster and shard status indicator data of each storage shard are acquired from the target storage object.
[0151] In one embodiment, when the computer program in the computer-readable storage medium 1004 is loaded and executed by the processor 1001 to generate the cluster state description information of the message storage cluster according to the cluster state indicator data, it is specifically used to perform the following steps:
[0152] Determine the upper node of the storage node to which each storage shard belongs according to the cluster topology information of the message storage cluster, wherein the cluster topology information is used to indicate the hierarchical relationship among the storage racks, storage nodes, and storage shards in the message storage cluster, and the upper node of the storage node includes the storage rack;
[0153] Determine the load information of each upper-layer node according to the load information of the storage nodes and storage slices contained in each upper-layer node;
[0154] The cluster status description information is generated according to the load information of each upper node, the load information of each storage node, and the load information of each storage shard.
[0155] In one embodiment, the cluster load requirement includes at least one of the following: shard partition unification requirement, single-node resource balancing requirement, multi-node resource balancing requirement, and storage resource balancing requirement; the computer program in the computer-readable storage medium 1004 is loaded and executed by the processor 1001 to generate storage shard adjustment information according to the cluster state description information and the cluster load requirement of the message storage cluster, and is specifically used to perform the following steps:
[0156] Taking the cluster load requirement as a target, storage shard adjustment information about the target storage node is generated according to the cluster state description information.
[0157] when Fig.10 The computer equipment shown corresponds to the above Figure 1 When the storage node in the computer readable storage medium 1004 is used, in a specific implementation, the computer program in the computer readable storage medium 1004 is loaded by the processor 1001 and the following steps are executed:
[0158] Receiving storage shard adjustment information sent by a management node to a target storage node; the management node and the target storage node are included in a message storage cluster, the message storage cluster includes multiple storage nodes, and the storage node includes one or more storage shards;
[0159] According to the storage shard adjustment information, the partition to which the target storage shard belongs is adjusted;
[0160] Among them, the storage shard adjustment information is generated by the management node based on the cluster status description information and the cluster load requirement of the message storage cluster, the cluster status description information is generated based on the cluster status indicator data of the message storage cluster, and the cluster status description information is used to indicate the load information of the upper-level nodes of each storage shard in the message storage cluster; the cluster status indicator data of the message storage cluster is obtained by the management node, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node.
[0161] In one embodiment, the storage shard adjustment information indicates that the target storage shard is stored in the target storage node, the target storage shard is included in the first storage node, and the first storage node is included in the message storage cluster; when the computer program in the computer-readable storage medium 1004 is loaded and executed by the processor 1001, the partition to which the target storage shard belongs is adjusted according to the storage shard adjustment information, and is specifically used to perform the following steps:
[0162] Receiving storage shard transfer information sent by the first storage node, wherein the storage shard transfer information includes the target storage shard;
[0163] Storing the target storage slice in the target storage node and generating transfer operation response information;
[0164] The transfer-in operation response information is sent to the first storage node, where the transfer-in operation response information is used to instruct the first storage node to delete the target storage slice contained in the first storage node.
[0165] In one embodiment, the storage shard adjustment information indicates that the target storage shard is stored in a second storage node, the second storage node is included in the message storage cluster, and the target storage shard is included in the target storage node; when the computer program in the computer-readable storage medium 1004 is loaded and executed by the processor 1001 and the partition to which the target storage shard belongs is adjusted according to the storage shard adjustment information, it is specifically used to perform the following steps:
[0166] Generate storage shard transfer-out information, and send the storage shard transfer-out information to the second storage node, wherein the storage shard transfer-out information is used to instruct the second storage node to perform storage processing on the target storage shard;
[0167] Receive the transfer-out operation response information sent by the second storage node, and delete the target storage shard stored in the target storage node according to the transfer-out operation response information.
[0168] Through the computer device provided in the embodiment of the present application, cluster status indicator data of the message storage cluster can be obtained, and cluster status description information can be generated, so as to intuitively reflect the load status of each layer of nodes in the message storage cluster; storage shard adjustment information can be generated according to the cluster status description information, so as to achieve automatic load balancing of the message storage cluster without manual intervention, effectively reducing the time consumed by the load balancing process; storage shard adjustment information can be generated more frequently, so as to achieve rapid elastic expansion of the message storage cluster during business peaks; the situation of a surge in business traffic can be determined through cluster status indicator data, and automated resource management and dynamic scaling strategies can be performed in response to the situation, so as to ensure the stability of the services provided by the message storage cluster during high traffic loads, and effectively ensure that users can still obtain a better service experience during business peaks.
[0169] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored, and when the computer instructions are executed on a computer device, the computer device executes the steps in each method embodiment of the present application to implement the cluster management method provided by the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.
[0170] The embodiment of the present application also provides a computer program product, which includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device executes the steps in the embodiments of the present application to implement the cluster management method provided in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.
[0171] It is appreciated by those skilled in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or in combination with computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technical objects can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit containing the function of the module or unit.
[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technical object familiar with the technical field within the technical scope disclosed in the present application can be easily thought of for changes or replacements, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A cluster management method, characterized in that: The method comprises: Acquire cluster status indicator data of a message storage cluster, wherein the message storage cluster includes a management node and multiple storage nodes, each storage node includes one or more storage shards, and the cluster status indicator data includes load information of each storage shard in the message storage cluster and load information of each storage node; Generate cluster state description information of the message storage cluster according to the cluster state indicator data, wherein the cluster state description information is used to indicate load information of upper nodes of each storage shard in the message storage cluster; Storage shard adjustment information is generated based on the cluster status description information and the cluster load requirement of the message storage cluster, and the storage shard adjustment information is sent to the target storage node in the message storage cluster. The storage shard adjustment information is used to instruct the target storage node to adjust the partition to which the target storage shard belongs, so as to adjust the load information of the target storage node.
2. The method according to claim 1, characterized in that The obtaining of cluster status indicator data of the message storage cluster includes: Obtain node status indicator data of each storage node in the message storage cluster, and shard status indicator data of each storage shard; the node status indicator data includes one or more of network traffic data, computing resource occupancy data, node capacity data, and storage resource description data of the corresponding storage node, and the shard status indicator data includes one or more of network traffic data, computing resource occupancy data, and storage resource occupancy data of the corresponding storage shard; Generate cluster status indicator data of the message storage cluster, wherein the cluster status indicator data includes node status indicator data of each storage node and shard status indicator data of each storage shard.
3. The method according to claim 2, characterized in that The message storage cluster further includes a target storage object, in which the node status indicator data of each storage node in the message storage cluster and the shard status indicator data of each storage shard included in the corresponding storage node are stored; The obtaining of node status indicator data of each storage node in the message storage cluster and shard status indicator data of each storage shard includes: Every time an indicator data acquisition cycle is reached, node status indicator data of each storage node in the message storage cluster and shard status indicator data of each storage shard are acquired from the target storage object.
4. The method according to any one of claims 1 to 3, characterized in that: The generating the cluster state description information of the message storage cluster according to the cluster state indicator data includes: Determine the upper node of the storage node to which each storage shard belongs according to the cluster topology information of the message storage cluster, wherein the cluster topology information is used to indicate the hierarchical relationship among the storage racks, storage nodes, and storage shards in the message storage cluster, and the upper node of the storage node includes the storage rack; Determine the load information of each upper-layer node according to the load information of the storage nodes and storage slices contained in each upper-layer node; The cluster status description information is generated according to the load information of each upper node, the load information of each storage node, and the load information of each storage shard.
5. The method according to any one of claims 1 to 3, characterized in that: The cluster load requirement includes at least one of the following: shard partition unification requirement, single-node resource balancing requirement, multi-node resource balancing requirement, and storage resource balancing requirement; the generating storage shard adjustment information according to the cluster state description information and the cluster load requirement of the message storage cluster includes: Taking the cluster load requirement as a target, storage shard adjustment information about the target storage node is generated according to the cluster state description information.
6. A cluster management method, characterized in that: The method comprises: Receiving storage shard adjustment information sent by a management node to a target storage node; the management node and the target storage node are included in a message storage cluster, the message storage cluster includes multiple storage nodes, and the storage node includes one or more storage shards; According to the storage shard adjustment information, the partition to which the target storage shard belongs is adjusted; Among them, the storage shard adjustment information is generated by the management node based on the cluster status description information and the cluster load requirement of the message storage cluster, the cluster status description information is generated based on the cluster status indicator data of the message storage cluster, and the cluster status description information is used to indicate the load information of the upper-level nodes of each storage shard in the message storage cluster; the cluster status indicator data of the message storage cluster is obtained by the management node, and the cluster status indicator data includes the load information of each storage shard in the message storage cluster and the load information of each storage node.
7. The method according to claim 6, characterized in that The storage shard adjustment information indicates that the target storage shard is stored in a target storage node, the target storage shard is included in a first storage node, and the first storage node is included in the message storage cluster; The step of adjusting the partition to which the target storage shard belongs according to the storage shard adjustment information includes: Receiving storage shard transfer information sent by the first storage node, wherein the storage shard transfer information includes the target storage shard; Storing the target storage slice in the target storage node and generating transfer operation response information; The transfer-in operation response information is sent to the first storage node, where the transfer-in operation response information is used to instruct the first storage node to delete the target storage slice contained in the first storage node.
8. The method according to claim 6 or 7, characterized in that The storage shard adjustment information indicates that the target storage shard is stored in a second storage node, the second storage node is included in the message storage cluster, and the target storage shard is included in the target storage node; The step of adjusting the partition to which the target storage shard belongs according to the storage shard adjustment information includes: Generate storage shard transfer-out information, and send the storage shard transfer-out information to the second storage node, wherein the storage shard transfer-out information is used to instruct the second storage node to perform storage processing on the target storage shard; Receive the transfer-out operation response information sent by the second storage node, and delete the target storage shard stored in the target storage node according to the transfer-out operation response information.
9. A cluster management device, characterized in that: The device comprises a unit for implementing the cluster management method according to any one of claims 1 to 5, or comprises a unit for implementing the cluster management method according to any one of claims 6 to 8.
10. A computer device, characterized in that: The computer device comprises: a processor suitable for implementing a computer program; A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by the processor and implementing the cluster management method according to any one of claims 1 to 5 or implementing the cluster management method according to any one of claims 6 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and implementing the cluster management method according to any one of claims 1 to 5 or the cluster management method according to any one of claims 6 to 8.
12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the cluster management method according to any one of claims 1 to 5 or the cluster management method according to any one of claims 6 to 8 is implemented.
Citation Information
Patent Citations
Load balancing processing method and device, electronic equipment and storage medium
CN111913670A
Load balancing method and device
CN112118314A
Data fragment adjustment method, device and equipment and readable storage medium
CN116955087A
Cross-cluster load balancing method and device, equipment and storage medium
CN117149445A
Kafka high-load node partition weight balancing method based on ant colony algorithm
CN117914698A
