Method for managing a distributed storage cluster and related components

Abstract

The application discloses a kind of management method and related components of distributed storage cluster, applied to storage field, management equipment receives the basic information sent by hard disk in distributed storage cluster, and basic information includes the identification information of hard disk and the identification information of host to which hard disk belongs;According to basic information, determine the storage pool to which hard disk belongs, the fault domain to which it belongs and the cabinet to which it belongs;Topology relationship is constructed;According to topology relationship, distributed storage cluster is drawn into visual display mode.The distributed storage cluster is visualized to facilitate users to determine the relationship between storage pool, fault domain, cabinet, host and hard disk, and the operation is more convenient, and the location of the fault can be quickly determined when the distributed storage cluster fails.

Description

Technical Field

[0001] This invention relates to the field of storage, and in particular to a management method and related components for a distributed storage cluster. Background Technology

[0002] The topology of a distributed storage cluster in a distributed storage system involves five levels: storage pools, fault domains, racks, hosts, and hard drives. Multiple storage pools can share a fault domain, and a fault domain is composed of multiple hard drives from hosts located in different racks. A fault domain to which a storage pool belongs can contain multiple hard drives, so one storage pool can correspond to multiple hard drives, and a single hard drive can belong to multiple fault domains, meaning one hard drive can also correspond to multiple storage pools. Current management software lacks an end-to-end topology for the storage system, i.e., the topology between storage pools, fault domains, racks, hosts, and hard drives. When a storage pool fails or its performance degrades, it's impossible to determine the associated fault domain, rack, host, and hard drive, hindering rapid identification of the cause of the problem. Similarly, when a hard drive fails or is in a suboptimal state, it's impossible to determine the affected hosts, racks, fault domains, and storage pools, preventing timely implementation of necessary measures. Summary of the Invention

[0003] The purpose of this invention is to provide a management method and related components for a distributed storage cluster, which visualizes the distributed storage cluster so that users can determine the relationship between storage pools, fault domains, racks, hosts and hard drives, making operation more convenient. At the same time, it can quickly determine the location of the fault when a fault occurs in the distributed storage cluster.

[0004] To address the aforementioned technical problems, this invention provides a management method for a distributed storage cluster, applied to the processor of a management device, the method comprising:

[0005] Receive basic information sent by the hard disk in the distributed storage cluster, the basic information including the identification information of the hard disk and the identification information of the host to which the hard disk belongs;

[0006] Based on the basic information, determine the storage pool, fault domain, and rack to which the hard drive belongs;

[0007] Construct topology relationships, including cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. The cluster topology relationship includes the correspondence between the distributed storage cluster and the storage pool. The storage pool hierarchical topology relationship includes the correspondence between the storage pool and the fault domain. The fault domain hierarchical topology relationship includes the correspondence between the fault domain and the rack. The rack hierarchical topology relationship includes the correspondence between the rack and the host. The host hierarchical topology relationship includes the correspondence between the host and the hard drive.

[0008] The distributed storage cluster is visualized based on the topological relationships.

[0009] Preferably, before determining the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information, the method further includes:

[0010] Determine the list information of the storage pools, wherein the list information of the storage pools includes the identification information of the storage pools;

[0011] Determine the topological relationships of the fault domains, cabinets, and nodes of the cluster, and generate a dictionary of the topological relationships;

[0012] The fault domain corresponding to the storage pool is determined based on the dictionary.

[0013] Preferably, determining the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information includes:

[0014] Identify all hard disks in the fault domain corresponding to the storage pool, designate the hard disks as leaf nodes, and define the storage pool to which the hard disks belong as the storage pool.

[0015] Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs;

[0016] Traverse upwards from the parent node to determine the rack containing the hard drive.

[0017] Preferably, after determining the storage pool, fault domain, and rack to which the hard drive belongs, the method further includes:

[0018] Add the storage pool to which the hard drive belongs, the fault domain to which it belongs, the rack to which it belongs, and the basic information of the hard drive to the hard drive information table;

[0019] The hard disk information table uses the storage pool identification information and the hard disk identification information of the hard disk as a composite primary key.

[0020] Preferably, a topology relationship is constructed, which includes cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships, including:

[0021] Determine whether the storage pool to which the hard drive belongs exists in the cluster topology, obtained from the hard drive information table of the hard drive;

[0022] If it does not exist in the cluster topology, a new key-value pair is created, where the key is the identification information of the storage pool, the value is an empty dictionary, and the key-value pair represents the storage pool hierarchical topology.

[0023] Determine whether the fault domain of the storage pool exists in the storage pool hierarchical topology;

[0024] If it does not exist in the storage pool hierarchical topology, a new key-value pair is created, where the key is the identifier information of the fault domain, the value is an empty dictionary, and the key-value pair represents the fault domain hierarchical topology.

[0025] Determine whether the rack to which the hard drive belongs exists in the fault domain hierarchical topology, obtained from the hard drive information table of the hard drive;

[0026] If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the identification information of the cabinet, the value is an empty dictionary, and the key-value pair represents the cabinet hierarchical topology.

[0027] Determine whether the host to which the hard drive belongs exists in the rack-level topology, obtained from the hard drive information table of the hard drive;

[0028] If it does not exist in the rack hierarchy topology, a new key-value pair is created, where the key is the host's identification information, the value is an empty dictionary, and the key-value pair represents the host hierarchy topology.

[0029] Determine whether the hard drive's identification information exists in the host-level topology;

[0030] If the host-level topology does not exist, a new key-value pair is created, where the key is the host's identification information and the value is an empty dictionary. The basic information of the hard disk is then filled into the dictionary.

[0031] Preferably, the cluster is drawn into a visual display mode according to the topological relationship, including:

[0032] Traverse the cluster topology dictionary, retrieve all the storage pool hierarchical topology relationships, and draw the storage pools;

[0033] Traverse the storage pool hierarchical topology, extract all the fault domain hierarchical topology, and draw the fault domains and the connections between the storage pools and the fault domains;

[0034] Traverse the fault domain hierarchical topology, extract all the cabinet hierarchical topology, and draw the cabinets and the connections between the fault domains and the cabinets;

[0035] Traverse the rack hierarchy topology, extract all host hierarchy topology, and draw the hosts and the connections between the racks and hosts;

[0036] Traverse the host hierarchy topology, retrieve all the hard drives, and draw the hard drives and the connections between the host and the hard drives.

[0037] To address the aforementioned technical problems, the present invention also provides a management system for a distributed storage cluster, comprising:

[0038] The receiving unit is used to receive basic information sent by the hard disk in the distributed storage cluster. The basic information includes the identification information of the hard disk and the identification information of the host to which the hard disk belongs.

[0039] The determining unit is used to determine the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information.

[0040] A construction unit is used to construct topology relationships, which include cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. The cluster topology relationship includes the correspondence between the distributed storage cluster and the storage pool. The storage pool hierarchical topology relationship includes the correspondence between the storage pool and the fault domain. The fault domain hierarchical topology relationship includes the correspondence between the fault domain and the rack. The rack hierarchical topology relationship includes the correspondence between the rack and the host. The host hierarchical topology relationship includes the correspondence between the host and the hard drive.

[0041] A drawing unit is used to draw the distributed storage cluster into a visual display mode according to the topological relationship.

[0042] To address the aforementioned technical problems, the present invention also provides a management device, comprising:

[0043] Memory, used to store computer programs;

[0044] A processor is used to implement the steps of the above-described distributed storage cluster management method when executing the computer program.

[0045] To address the aforementioned technical problems, the present invention also provides a distributed storage cluster, including the aforementioned management device, as well as multiple cabinets, multiple hosts, and multiple hard drives.

[0046] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned distributed storage cluster management method.

[0047] This invention discloses a management method and related components for a distributed storage cluster, applied in the storage field. The management device receives basic information sent by the hard drives in the distributed storage cluster, including the hard drive's identification information and the host's identification information. Based on the basic information, it determines the storage pool, fault domain, and rack to which the hard drive belongs; constructs a topology; and visualizes the distributed storage cluster based on the topology. Visualizing the distributed storage cluster allows users to understand the relationships between storage pools, fault domains, racks, hosts, and hard drives, making operation more convenient. Furthermore, it allows for quick location of the fault when a fault occurs in the distributed storage cluster. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a distributed storage cluster management method provided by the present invention;

[0050] Figure 2 A schematic diagram illustrating a visualization display mode provided by the present invention;

[0051] Figure 3 This is a schematic diagram of the structure of a distributed storage cluster management system provided by the present invention;

[0052] Figure 4 This is a schematic diagram of the structure of a management device provided by the present invention. Detailed Implementation

[0053] The core of this invention is to provide a management method and related components for a distributed storage cluster, which visualizes the distributed storage cluster so that users can determine the relationship between storage pools, fault domains, racks, hosts and hard drives, making operation more convenient. At the same time, it can quickly determine the location of the fault when a fault occurs in the distributed storage cluster.

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Figure 1 A flowchart of a distributed storage cluster management method provided by the present invention is shown. The method is applied to the processor of a management device and includes:

[0056] S11: Receive basic information sent by the hard disk in the distributed storage cluster. The basic information includes the identification information of the hard disk and the identification information of the host to which the hard disk belongs.

[0057] Considering that the topology of a distributed storage cluster in a distributed storage system involves five levels: storage pools, fault domains, racks, hosts, and hard drives, multiple storage pools can share a fault domain. A fault domain is composed of multiple hard drives from hosts located in different racks. A fault domain belonging to a storage pool can contain multiple hard drives, so one storage pool can correspond to multiple hard drives, and a single hard drive can belong to multiple fault domains, meaning one hard drive can also correspond to multiple storage pools. Currently, management software lacks an end-to-end topology for the storage system—that is, the topology between storage pools, fault domains, racks, hosts, and hard drives. When a storage pool fails or its performance degrades, it's impossible to determine the associated fault domain, rack, host, and hard drive, hindering rapid identification of the cause of the problem. Similarly, when a hard drive fails or is in a suboptimal state, it's impossible to determine the affected hosts, racks, fault domains, and storage pools, preventing timely implementation of necessary measures.

[0058] Once the hard drive's identification information is obtained, the host corresponding to the hard drive and other devices at other levels can be identified.

[0059] Specifically, the hard drive's identification information is its name, and the host's identification information is its name.

[0060] S12: Determine the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information;

[0061] Based on the host computer associated with the hard drive, the corresponding server rack can be determined, and then the corresponding fault domain and storage pool can be identified.

[0062] S13: Construct topology relationships, including cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. Cluster topology relationships include the correspondence between distributed storage clusters and storage pools. Storage pool hierarchical topology relationships include the correspondence between storage pools and fault domains. Fault domain hierarchical topology relationships include the correspondence between fault domains and racks. Rack hierarchical topology relationships include the correspondence between racks and hosts. Host hierarchical topology relationships include the correspondence between hosts and hard drives.

[0063] Based on the host, rack, fault domain, and storage pool to which the hard drive belongs, a topology relationship is generated, which includes the relationship between each adjacent level.

[0064] S14: Draw the distributed storage cluster into a visual display mode based on the topology.

[0065] Figure 2 This is a schematic diagram of a visualization mode provided by the present invention. Taking nine storage pools, four fault domains, three racks and five hosts as an example, nodes are hosts, SSDs and HDDs are hard disks, hard disk resource domains are fault domains and racks are racks.

[0066] The relationships between different levels are Figure 2 This is presented in a way that allows users to determine the relationship between hard drives and storage pools, identify all hard drives corresponding to a storage pool when a storage pool fails, and identify the storage pool corresponding to a hard drive when a hard drive fails.

[0067] This invention discloses a management method for a distributed storage cluster, applied in the storage field. The management device receives basic information sent by the hard drives in the distributed storage cluster, including the hard drive's identification information and the host's identification information. Based on the basic information, the method determines the storage pool, fault domain, and rack to which the hard drive belongs. A topology is constructed. Based on the topology, the distributed storage cluster is visualized. Visualizing the distributed storage cluster allows users to understand the relationships between storage pools, fault domains, racks, hosts, and hard drives, making operation more convenient. Furthermore, it allows for quick location of the fault when a fault occurs in the distributed storage cluster.

[0068] Based on the above embodiments:

[0069] As a preferred embodiment, before determining the storage pool, fault domain, and rack to which the hard drive belongs based on basic information, the process further includes:

[0070] Determine the list information of the storage pools, which includes the identification information of the storage pools;

[0071] Determine the fault domains, racks, and node topology of the cluster, and generate a dictionary of the topology relationships;

[0072] The fault domain corresponding to the storage pool is determined based on the dictionary.

[0073] The method provided in this application is applied to the processor of a management device. After the agent is deployed on the storage host, the host periodically reports basic information about the hard drives through the Telegraf component. The management device saves the basic information about the hard drives to the InfluxDB database. The processor of the management device calls the REST API interface to query the identification information of the storage pool. Specifically, the identification information of the storage pool includes the name of the storage pool.

[0074] Call the REST API interface to determine the fault domains, racks, and node topology of the cluster, and generate a dictionary of the topology relationships.

[0075] The name of the host of the root node of the fault domain is obtained from the faultdomain field of the storage pool, and then the fault domain corresponding to the storage pool is determined based on the name of the host of the root node.

[0076] Obtain information about the storage pool and determine the corresponding fault domain so that the corresponding hard drive can be identified later.

[0077] As a preferred embodiment, determining the storage pool, fault domain, and rack to which the hard drive belongs based on basic information includes:

[0078] Identify all hard drives in the fault domain corresponding to the storage pool, treat the hard drives as leaf nodes, and the storage pool to which the hard drives belong as the storage pool.

[0079] Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs;

[0080] Traverse upwards from the parent node to determine the hard drive rack.

[0081] Get all leaf nodes (i.e. hard disks) of the fault domain. The storage pool to which the hard disk belongs is the current storage pool. Traverse the parent nodes upward from the leaf nodes to get the host to which it belongs. Then traverse the parent nodes upward again to get the rack to which it belongs.

[0082] As a preferred embodiment, after determining the storage pool, fault domain, and rack to which the hard drive belongs, the method further includes:

[0083] Add the storage pool to which the hard drive belongs, the fault domain to which it belongs, the rack to which it belongs, and the basic information of the hard drive to the hard drive information table;

[0084] The hard disk information table uses the storage pool identifier and the hard disk identifier as a composite primary key.

[0085] Based on the hard drive name and its hostname, the InfluxDB hard drive information list is queried to obtain the basic information of the hard drive. The basic information of the hard drive is then updated with its storage pool, fault domain, and rack, and persisted to the MySQL hard drive information table. Table 1 is the MySQL hard drive information table. The MySQL hard drive information table uses a composite primary key consisting of the storage pool field and the hard drive's unique identifier. Due to the many-to-many relationship between storage pools and hard drives, the same hard drive may have multiple records in the database table.

[0086] Table 1

[0087] name type illustrate cluster_id varchar(100) Unique identifier of storage system id varchar(100) Hard drive unique identifier vendor_id varchar(50) Manufacturers product_id varchar(50) model slot_id int(11) Slot number enclosure_id int(11) Chassis number disk_name varchar(150) Hard drive name agent_host varchar(150) Node hostname fault_domain varchar(100) Fault Domain pool_name varchar(100) Storage pool rack varchar(50) Server rack

[0088] As a preferred embodiment, a topology relationship is constructed, including cluster topology relationships, storage pool level topology relationships, fault domain level topology relationships, rack level topology relationships, and host level topology relationships, including:

[0089] Determine whether the storage pool to which the hard drive belongs exists in the cluster topology, obtained from the hard drive information table;

[0090] If it does not exist in the cluster topology, create a new key-value pair. The key is the identification information of the storage pool, and the value is an empty dictionary. The key-value pair represents the storage pool hierarchical topology.

[0091] Determine whether the fault domain of the storage pool exists in the storage pool hierarchy topology;

[0092] If it does not exist in the storage pool hierarchical topology, create a new key-value pair. The key is the identification information of the fault domain, and the value is an empty dictionary. The key-value pair represents the fault domain hierarchical topology.

[0093] Determine whether the rack to which the hard drive belongs exists in the fault domain hierarchy topology, obtained from the hard drive information table.

[0094] If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the rack identification information and the value is an empty dictionary. The key-value pair represents the rack hierarchical topology.

[0095] Determine whether the host to which the hard drive belongs exists in the rack-level topology, obtained from the hard drive information table;

[0096] If it does not exist in the rack-level topology, create a new key-value pair. The key is the host's identification information, and the value is an empty dictionary. The key-value pair represents the host-level topology.

[0097] Determine if the hard drive's identification information exists in the host-level topology;

[0098] If the key does not exist in the host-level topology, a new key-value pair is created, with the key being the host's identification information and the value being an empty dictionary. The basic information of the hard drive is then filled into the dictionary.

[0099] Initialize a dictionary with empty cluster topology relationships. Query all disk information from the MySQL disk information table, which is a list. Iterate through the list of disk information.

[0100] Retrieve the storage pool to which the hard drive belongs, and determine whether the storage pool already exists in the cluster topology. If it does not exist, create a new key-value pair, where the key is the storage pool name and the value is an empty dictionary. If it exists, retrieve the content with the storage pool name as the key from the cluster topology, i.e., the storage pool hierarchical topology.

[0101] Retrieve the fault domain to which the storage pool belongs, and determine whether the fault domain already exists in the storage pool hierarchical topology. If it does not exist, create a new key-value pair, where the key is the fault domain name and the value is an empty dictionary. If it exists, retrieve the content with the fault domain name as the key from the storage pool hierarchical topology, i.e., the fault domain hierarchical topology.

[0102] Retrieve the rack to which the hard drive belongs, and determine whether the rack already exists in the fault domain hierarchical topology. If it does not exist, create a new key-value pair, where the key is the rack name and the value is an empty dictionary. If it exists, retrieve the content with the rack name as the key from the fault domain hierarchical topology, i.e., the rack hierarchical topology.

[0103] Retrieve the host to which the hard drive belongs, and determine whether the host already exists in the rack-level topology. If it does not exist, create a new key-value pair, where the key is the hostname and the value is an empty list. If it exists, retrieve the content with the key being the hostname from the rack-level topology, i.e., the host-level topology.

[0104] Retrieve the hard drive name, check if the hard drive name already exists in the host-level topology, if not, create a new hard drive dictionary and add it to the host list, where the key is the hard drive name and the value is an empty dictionary, and the dictionary is filled with the basic information of the hard drive.

[0105] As a preferred embodiment, the cluster is drawn into a visual display mode according to the topology, including:

[0106] Traverse the cluster topology dictionary, retrieve all storage pool hierarchical topology relationships, and draw the storage pools;

[0107] Traverse the storage pool hierarchical topology, extract all fault domain hierarchical topology, and draw the fault domains and the connections between the storage pools and the fault domains.

[0108] Traverse the fault domain hierarchical topology, extract all rack hierarchical topology, and draw the racks and the connections between fault domains and racks;

[0109] Traverse the rack hierarchy topology, extract all host hierarchy topology, and draw the hosts and the connections between the racks and hosts;

[0110] Traverse the host hierarchy topology, retrieve all hard drives, and draw the hard drives and the connections between the host and the hard drives.

[0111] Figure 3 This is a schematic diagram of the structure of a distributed storage cluster management system provided by the present invention. The system includes:

[0112] The receiving unit 31 is used to receive basic information sent by the hard disk in the distributed storage cluster. The basic information includes the identification information of the hard disk and the identification information of the host to which the hard disk belongs.

[0113] The determination unit 32 is used to determine the storage pool, fault domain, and rack to which the hard drive belongs based on basic information.

[0114] Building unit 33 is used to build topology relationships, which include cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. Cluster topology relationships include the correspondence between distributed storage clusters and storage pools. Storage pool hierarchical topology relationships include the correspondence between storage pools and fault domains. Fault domain hierarchical topology relationships include the correspondence between fault domains and racks. Rack hierarchical topology relationships include the correspondence between racks and hosts. Host hierarchical topology relationships include the correspondence between hosts and hard drives.

[0115] The drawing unit 34 is used to draw the distributed storage cluster into a visual display mode according to the topology.

[0116] The determining unit 32 is also used to determine the list information of the storage pools, which includes the identification information of the storage pools;

[0117] Determine the fault domains, racks, and node topology of the cluster, and generate a dictionary of the topology relationships;

[0118] Determine the fault domain corresponding to the storage pool based on the dictionary;

[0119] Identify all hard drives in the fault domain corresponding to the storage pool, treat the hard drives as leaf nodes, and the storage pool to which the hard drives belong as the storage pool.

[0120] Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs;

[0121] Traverse upwards from the parent node to determine the hard drive rack.

[0122] It also includes an addition unit, which is used to add the storage pool to which the hard drive belongs, the fault domain to which it belongs, and the basic information of the rack and hard drive to the hard drive information table;

[0123] The hard disk information table uses the storage pool identifier and the hard disk identifier as a composite primary key.

[0124] It also includes a judgment unit, used to determine whether the storage pool to which the hard drive belongs exists in the cluster topology relationship obtained from the hard drive information table; whether the fault domain of the storage pool exists in the storage pool level topology relationship; whether the rack to which the hard drive belongs exists in the fault domain level topology relationship obtained from the hard drive information table; whether the host to which the hard drive belongs exists in the rack level topology relationship obtained from the hard drive information table; and whether the hard drive identification information exists in the host level topology relationship.

[0125] It also includes a new unit, which is used to create a key-value pair if it does not exist in the cluster topology relationship. The key is the identification information of the storage pool, the value is an empty dictionary, and the key-value pair is the storage pool hierarchical topology relationship.

[0126] If it does not exist in the storage pool hierarchical topology, create a new key-value pair. The key is the identification information of the fault domain, and the value is an empty dictionary. The key-value pair represents the fault domain hierarchical topology.

[0127] If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the rack identification information and the value is an empty dictionary. The key-value pair represents the rack hierarchical topology.

[0128] If it does not exist in the rack-level topology, create a new key-value pair. The key is the host's identification information, and the value is an empty dictionary. The key-value pair represents the host-level topology.

[0129] If the key does not exist in the host-level topology, a new key-value pair is created, with the key being the host's identification information and the value being an empty dictionary. The basic information of the hard drive is then filled into the dictionary.

[0130] The drawing unit is specifically used to traverse the cluster topology dictionary, retrieve all storage pool hierarchical topology relationships, and draw the storage pools.

[0131] Traverse the storage pool hierarchical topology, extract all fault domain hierarchical topology, and draw the fault domains and the connections between the storage pools and the fault domains.

[0132] Traverse the fault domain hierarchical topology, extract all rack hierarchical topology, and draw the racks and the connections between fault domains and racks;

[0133] Traverse the rack hierarchy topology, extract all host hierarchy topology, and draw the hosts and the connections between the racks and hosts;

[0134] Traverse the host hierarchy topology, retrieve all hard drives, and draw the hard drives and the connections between the host and the hard drives.

[0135] The management system for the distributed storage cluster provided in this application is described in the above embodiments and will not be repeated here.

[0136] Figure 4 This invention provides a schematic diagram of the structure of a management device, which includes:

[0137] Memory 41 is used to store computer programs;

[0138] The processor 42 is used to implement the steps of the above-described distributed storage cluster management method when executing computer programs.

[0139] The specific steps executed by the processor are as follows:

[0140] Receive basic information sent by the hard drives in the distributed storage cluster. The basic information includes the identification information of the hard drive and the identification information of the host to which the hard drive belongs.

[0141] Determine the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information;

[0142] Construct topology relationships, including cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. Cluster topology relationships include the correspondence between distributed storage clusters and storage pools. Storage pool hierarchical topology relationships include the correspondence between storage pools and fault domains. Fault domain hierarchical topology relationships include the correspondence between fault domains and racks. Rack hierarchical topology relationships include the correspondence between racks and hosts. Host hierarchical topology relationships include the correspondence between hosts and hard drives.

[0143] The distributed storage cluster is visualized based on its topology.

[0144] Before determining the storage pool, fault domain, and rack to which the hard drive belongs based on basic information, the following steps are also included:

[0145] Determine the list information of the storage pools, which includes the identification information of the storage pools;

[0146] Determine the fault domains, racks, and node topology of the cluster, and generate a dictionary of the topology relationships;

[0147] The fault domain corresponding to the storage pool is determined based on the dictionary.

[0148] Based on basic information, determine the storage pool, fault domain, and rack to which the hard drive belongs, including:

[0149] Identify all hard drives in the fault domain corresponding to the storage pool, treat the hard drives as leaf nodes, and the storage pool to which the hard drives belong as the storage pool.

[0150] Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs;

[0151] Traverse upwards from the parent node to determine the hard drive rack.

[0152] After determining the storage pool, fault domain, and rack to which the hard drive belongs, the following also includes:

[0153] Add the storage pool to which the hard drive belongs, the fault domain to which it belongs, the rack to which it belongs, and the basic information of the hard drive to the hard drive information table;

[0154] The hard disk information table uses the storage pool identifier and the hard disk identifier as a composite primary key.

[0155] The topology relationships are constructed, including cluster topology relationships, storage pool level topology relationships, fault domain level topology relationships, rack level topology relationships, and host level topology relationships, including:

[0156] Determine whether the storage pool to which the hard drive belongs exists in the cluster topology, obtained from the hard drive information table;

[0157] If it does not exist in the cluster topology, create a new key-value pair. The key is the identification information of the storage pool, and the value is an empty dictionary. The key-value pair represents the storage pool hierarchical topology.

[0158] Determine whether the fault domain of the storage pool exists in the storage pool hierarchy topology;

[0159] If it does not exist in the storage pool hierarchical topology, create a new key-value pair. The key is the identification information of the fault domain, and the value is an empty dictionary. The key-value pair represents the fault domain hierarchical topology.

[0160] Determine whether the rack to which the hard drive belongs exists in the fault domain hierarchy topology, obtained from the hard drive information table.

[0161] If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the rack identification information and the value is an empty dictionary. The key-value pair represents the rack hierarchical topology.

[0162] Determine whether the host to which the hard drive belongs exists in the rack-level topology, obtained from the hard drive information table;

[0163] If it does not exist in the rack-level topology, create a new key-value pair. The key is the host's identification information, and the value is an empty dictionary. The key-value pair represents the host-level topology.

[0164] Determine if the hard drive's identification information exists in the host-level topology;

[0165] If the key does not exist in the host-level topology, a new key-value pair is created, with the key being the host's identification information and the value being an empty dictionary. The basic information of the hard drive is then filled into the dictionary.

[0166] The cluster is visualized based on its topology, including:

[0167] Traverse the cluster topology dictionary, retrieve all storage pool hierarchical topology relationships, and draw the storage pools;

[0168] Traverse the storage pool hierarchical topology, extract all fault domain hierarchical topology, and draw the fault domains and the connections between the storage pools and the fault domains.

[0169] Traverse the fault domain hierarchical topology, extract all rack hierarchical topology, and draw the racks and the connections between fault domains and racks;

[0170] Traverse the rack hierarchy topology, extract all host hierarchy topology, and draw the hosts and the connections between the racks and hosts;

[0171] Traverse the host hierarchy topology, retrieve all hard drives, and draw the hard drives and the connections between the host and the hard drives.

[0172] Please refer to the above embodiments for a description of the management equipment provided in this application, and it will not be repeated here.

[0173] This application also provides a distributed storage cluster, including the aforementioned management device, as well as multiple cabinets, multiple hosts, and multiple hard drives.

[0174] Please refer to the above embodiments for a description of the management equipment provided in this application, and it will not be repeated here.

[0175] This application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described distributed storage cluster management method.

[0176] Receive basic information sent by the hard drives in the distributed storage cluster. The basic information includes the identification information of the hard drive and the identification information of the host to which the hard drive belongs.

[0177] Determine the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information;

[0178] Construct topology relationships, including cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. Cluster topology relationships include the correspondence between distributed storage clusters and storage pools. Storage pool hierarchical topology relationships include the correspondence between storage pools and fault domains. Fault domain hierarchical topology relationships include the correspondence between fault domains and racks. Rack hierarchical topology relationships include the correspondence between racks and hosts. Host hierarchical topology relationships include the correspondence between hosts and hard drives.

[0179] The distributed storage cluster is visualized based on its topology.

[0180] Before determining the storage pool, fault domain, and rack to which the hard drive belongs based on basic information, the following steps are also included:

[0181] Determine the list information of the storage pools, which includes the identification information of the storage pools;

[0182] Determine the fault domains, racks, and node topology of the cluster, and generate a dictionary of the topology relationships;

[0183] The fault domain corresponding to the storage pool is determined based on the dictionary.

[0184] Based on basic information, determine the storage pool, fault domain, and rack to which the hard drive belongs, including:

[0185] Identify all hard drives in the fault domain corresponding to the storage pool, treat the hard drives as leaf nodes, and the storage pool to which the hard drives belong as the storage pool.

[0186] Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs;

[0187] Traverse upwards from the parent node to determine the hard drive rack.

[0188] After determining the storage pool, fault domain, and rack to which the hard drive belongs, the following also includes:

[0189] Add the storage pool to which the hard drive belongs, the fault domain to which it belongs, the rack to which it belongs, and the basic information of the hard drive to the hard drive information table;

[0190] The hard disk information table uses the storage pool identifier and the hard disk identifier as a composite primary key.

[0191] The topology relationships are constructed, including cluster topology relationships, storage pool level topology relationships, fault domain level topology relationships, rack level topology relationships, and host level topology relationships, including:

[0192] Determine whether the storage pool to which the hard drive belongs exists in the cluster topology, obtained from the hard drive information table;

[0193] If it does not exist in the cluster topology, create a new key-value pair. The key is the identification information of the storage pool, and the value is an empty dictionary. The key-value pair represents the storage pool hierarchical topology.

[0194] Determine whether the fault domain of the storage pool exists in the storage pool hierarchy topology;

[0195] If it does not exist in the storage pool hierarchical topology, create a new key-value pair. The key is the identification information of the fault domain, and the value is an empty dictionary. The key-value pair represents the fault domain hierarchical topology.

[0196] Determine whether the rack to which the hard drive belongs exists in the fault domain hierarchy topology, obtained from the hard drive information table.

[0197] If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the rack identification information and the value is an empty dictionary. The key-value pair represents the rack hierarchical topology.

[0198] Determine whether the host to which the hard drive belongs exists in the rack-level topology, obtained from the hard drive information table;

[0199] If it does not exist in the rack-level topology, create a new key-value pair. The key is the host's identification information, and the value is an empty dictionary. The key-value pair represents the host-level topology.

[0200] Determine if the hard drive's identification information exists in the host-level topology;

[0201] If the key does not exist in the host-level topology, a new key-value pair is created, with the key being the host's identification information and the value being an empty dictionary. The basic information of the hard drive is then filled into the dictionary.

[0202] The cluster is visualized based on its topology, including:

[0203] Traverse the cluster topology dictionary, retrieve all storage pool hierarchical topology relationships, and draw the storage pools;

[0204] Traverse the storage pool hierarchical topology, extract all fault domain hierarchical topology, and draw the fault domains and the connections between the storage pools and the fault domains.

[0205] Traverse the fault domain hierarchical topology, extract all rack hierarchical topology, and draw the racks and the connections between fault domains and racks;

[0206] Traverse the rack hierarchy topology, extract all host hierarchy topology, and draw the hosts and the connections between the racks and hosts;

[0207] Traverse the host hierarchy topology, retrieve all hard drives, and draw the hard drives and the connections between the host and the hard drives.

[0208] The description of the computer-readable storage medium provided in this application is given in the above embodiments and will not be repeated here.

[0209] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0210] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0211] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A management method for a distributed storage cluster, characterized in that, The method, applied to a processor of a management device, includes: Receive basic information sent by the hard disk in the distributed storage cluster, the basic information including the identification information of the hard disk and the identification information of the host to which the hard disk belongs; Based on the basic information, the storage pool, fault domain, and rack to which the hard drive belongs are determined; the fault domain consists of several hard drives of hosts located in different racks; the hard drive corresponds to multiple storage pools, and the storage pool corresponds to multiple hard drives; Construct topology relationships, including cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. The cluster topology relationship includes the correspondence between the distributed storage cluster and the storage pool. The storage pool hierarchical topology relationship includes the correspondence between the storage pool and the fault domain. The fault domain hierarchical topology relationship includes the correspondence between the fault domain and the rack. The rack hierarchical topology relationship includes the correspondence between the rack and the host. The host hierarchical topology relationship includes the correspondence between the host and the hard drive. The distributed storage cluster is visualized based on the described topology. Before determining the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information, the process also includes: Determine the list information of the storage pools, wherein the list information of the storage pools includes the identification information of the storage pools; The topological relationships of the fault domains, cabinets, and nodes in the cluster are determined, and a dictionary is generated from these topological relationships; the nodes are hosts. The fault domain corresponding to the storage pool is determined based on the dictionary; The process of determining the storage pool, fault domain, and server rack to which the hard drive belongs based on the basic information includes: Identify all hard disks in the fault domain corresponding to the storage pool, designate the hard disks as leaf nodes, and define the storage pool to which the hard disks belong as the storage pool. Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs; Traverse upwards from the parent node to determine the server rack containing the hard drive; After determining the storage pool, fault domain, and rack to which the hard drive belongs, the process also includes: Add the storage pool to which the hard drive belongs, the fault domain to which it belongs, the rack to which it belongs, and the basic information of the hard drive to the hard drive information table; The hard disk information table uses the identification information of the storage pool to which the hard disk belongs and the identification information of the hard disk as a composite primary key; This includes constructing topology relationships, which encompass cluster topology relationships, storage pool-level topology relationships, fault domain-level topology relationships, rack-level topology relationships, and host-level topology relationships, including: Determine whether the storage pool to which the hard drive belongs exists in the cluster topology, obtained from the hard drive information table of the hard drive; If it does not exist in the cluster topology, a new key-value pair is created, where the key is the identification information of the storage pool, the value is an empty dictionary, and the key-value pair represents the storage pool hierarchical topology. Determine whether the fault domain of the storage pool exists in the storage pool hierarchical topology; If it does not exist in the storage pool hierarchical topology, a new key-value pair is created, where the key is the identifier information of the fault domain, the value is an empty dictionary, and the key-value pair represents the fault domain hierarchical topology. Determine whether the rack to which the hard drive belongs exists in the fault domain hierarchical topology, obtained from the hard drive information table of the hard drive; If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the identification information of the cabinet, the value is an empty dictionary, and the key-value pair represents the cabinet hierarchical topology. Determine whether the host to which the hard drive belongs exists in the rack-level topology, obtained from the hard drive information table of the hard drive; If it does not exist in the rack hierarchy topology, a new key-value pair is created, where the key is the host's identification information, the value is an empty dictionary, and the key-value pair represents the host hierarchy topology. Determine whether the hard drive's identification information exists in the host-level topology; If the host-level topology does not exist, a new key-value pair is created, where the key is the host's identification information and the value is an empty dictionary. The basic information of the hard disk is then filled into the dictionary.

2. The management method for a distributed storage cluster as described in claim 1, characterized in that, Based on the aforementioned topology, the cluster is visualized in a display mode, including: Traverse the cluster topology dictionary, retrieve all the storage pool hierarchical topology relationships, and draw the storage pools; Traverse the storage pool hierarchical topology, extract all the fault domain hierarchical topology, and draw the fault domains and the connections between the storage pools and the fault domains; Traverse the fault domain hierarchical topology, extract all the cabinet hierarchical topology, and draw the cabinets and the connections between the fault domains and the cabinets; Traverse the rack hierarchy topology, extract all host hierarchy topology, and draw the hosts and the connections between the racks and hosts; Traverse the host hierarchy topology, retrieve all the hard drives, and draw the hard drives and the connections between the host and the hard drives.

3. A management system for a distributed storage cluster, characterized in that, include: The receiving unit is used to receive basic information sent by the hard disk in the distributed storage cluster. The basic information includes the identification information of the hard disk and the identification information of the host to which the hard disk belongs. The determining unit is used to determine the storage pool, fault domain, and rack to which the hard drive belongs based on the basic information; the fault domain consists of several hard drives of hosts located in different racks; the hard drive corresponds to multiple storage pools, and the storage pool corresponds to multiple hard drives; A construction unit is used to construct topology relationships, which include cluster topology relationships, storage pool hierarchical topology relationships, fault domain hierarchical topology relationships, rack hierarchical topology relationships, and host hierarchical topology relationships. The cluster topology relationship includes the correspondence between the distributed storage cluster and the storage pool. The storage pool hierarchical topology relationship includes the correspondence between the storage pool and the fault domain. The fault domain hierarchical topology relationship includes the correspondence between the fault domain and the rack. The rack hierarchical topology relationship includes the correspondence between the rack and the host. The host hierarchical topology relationship includes the correspondence between the host and the hard drive. A drawing unit is used to draw the distributed storage cluster into a visual display mode according to the topological relationship; The determining unit is also used to determine the list information of the storage pools, which includes the identification information of the storage pools. Determine the fault domains, racks, and node topology of the cluster, and generate a dictionary of the topology relationships; nodes are hosts. Determine the fault domain corresponding to the storage pool based on the dictionary; Identify all hard drives in the fault domain corresponding to the storage pool, treat the hard drives as leaf nodes, and define the storage pool to which the hard drives belong as the storage pool. Traverse upwards from the leaf nodes to determine the host to which the hard drive belongs; Traverse upwards from the parent node to determine the hard drive rack; It also includes an addition unit, which is used to add the storage pool to which the hard drive belongs, the fault domain to which it belongs, and the basic information of the rack and hard drive to the hard drive information table; The hard disk information table uses the identification information of the storage pool to which the hard disk belongs and the identification information of the hard disk as a composite primary key; The management system of the distributed storage cluster further includes a judgment unit, used to determine whether the storage pool to which the hard drive belongs exists in the cluster topology relationship obtained from the hard drive information table of the hard drive; whether the fault domain of the storage pool exists in the storage pool level topology relationship; whether the rack to which the hard drive belongs exists in the fault domain level topology relationship obtained from the hard drive information table of the hard drive; whether the host to which the hard drive belongs exists in the rack level topology relationship obtained from the hard drive information table of the hard drive; and whether the identification information of the hard drive exists in the host level topology relationship. It also includes a new unit, which is used to create a key-value pair if it does not exist in the cluster topology relationship. The key is the identification information of the storage pool, the value is an empty dictionary, and the key-value pair is the storage pool hierarchical topology relationship. If it does not exist in the storage pool hierarchical topology, create a new key-value pair. The key is the identification information of the fault domain, and the value is an empty dictionary. The key-value pair represents the fault domain hierarchical topology. If the fault domain hierarchical topology does not exist, a new key-value pair is created, where the key is the rack identification information and the value is an empty dictionary. The key-value pair represents the rack hierarchical topology. If it does not exist in the rack-level topology, create a new key-value pair. The key is the host's identification information, and the value is an empty dictionary. The key-value pair represents the host-level topology. If the key does not exist in the host-level topology, a new key-value pair is created, with the key being the host's identification information and the value being an empty dictionary. The basic information of the hard drive is then filled into the dictionary.

4. A management device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the management method for a distributed storage cluster as described in claim 1 or 2 when executing the computer program.

5. A distributed storage cluster, characterized in that, The device includes the management equipment as described in claim 4, and also includes multiple cabinets, multiple hosts, and multiple hard drives.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the management method for the distributed storage cluster as described in claim 1 or 2.

Images