Cloud disk management and control method, cloud disk management and control system, and electronic device and storage medium

Abstract

A cloud disk management and control method, a cloud disk management and control system, and an electronic device and a storage medium, which relate to the technical field of distributed storage. The method comprises: acquiring cloud disk identification information of a target cloud disk, wherein the cloud disk identification information is used for identifying the target cloud disk and a target storage master and a target cluster that correspond to the target cloud disk, the target cluster comprises a plurality of storage masters, the plurality of storage masters comprise the target storage master, and the target storage master is used for creating the target cloud disk on at least one storage server of the target cluster and generating the cloud disk identification information; and determining the target cluster, the target storage master and the target cloud disk on the basis of the cloud disk identification information. By means of the method, the resource utilization rate can be improved, and a large-scale cluster is supported.

Description

Cloud disk management method, system, electronic device and storage medium

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 12, 2023, with application number 202311713410.6 and application name “Cloud disk management method, system, electronic device and storage medium”, the entire contents of which are incorporated by reference in this disclosure. Technical Field

[0002] The present disclosure relates to the field of distributed storage technology, and in particular to a cloud disk management method, system, electronic device, and storage medium. Background Art

[0003] Distributed storage is a data storage technology that distributes data across multiple physical locations on a network. A block storage system is a block-level, randomly distributed storage system that provides low latency, durability, and high reliability. In a block storage system, data is divided into blocks, each of which operates as an independent hard drive. This approach provides high performance and flexibility, making it particularly suitable for processing large amounts of data, such as in databases and large-scale virtual machine environments. In distributed block storage applications, cloud disks provide users with block-level block storage services. They virtualize the storage space of physical hard drives and make it available to users over the network. Users can use cloud disks to read and write data just like local hard drives. Therefore, in block storage systems, it is crucial to achieve efficient management and control of cloud disks and minimize the failure domain of the block storage system.

[0004] Summary of the Invention

[0005] The embodiments of the present disclosure provide a cloud disk management method, system, electronic device, and storage medium to alleviate or solve one or more technical problems existing in the prior art.

[0006] In a first aspect, an embodiment of the present disclosure provides a cloud disk management and control method, including: obtaining cloud disk identification information of a target cloud disk; the cloud disk identification information is used to identify the target cloud disk and the target storage management and control node and target cluster corresponding to the target cloud disk; the target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; the target storage management and control node is used to create the target cloud disk on at least one storage service node of the target cluster and generate the cloud disk identification information; and determine the target cluster, the target storage management and control node and the target cloud disk based on the cloud disk identification information.

[0007] In the second aspect, an embodiment of the present disclosure provides a cloud disk management and control method, including: in response to a cloud disk creation request, creating a target cloud disk on at least one storage service node of a target cluster; the target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; generating cloud disk identification information of the target cloud disk; the cloud disk identification information is used to identify the target cloud disk, the target storage management and control node and the target cluster.

[0008] In the third aspect, an embodiment of the present disclosure provides a cloud disk management and control system, including: a cluster management and control node, used to execute the method provided by the embodiment of the first aspect of the present disclosure; at least one cluster, the cluster including multiple storage management and control nodes and multiple storage service nodes, the storage management and control node is used to execute the method provided by the embodiment of the second aspect of the present disclosure.

[0009] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method provided in any embodiment of the present disclosure when executing the computer program.

[0010] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method provided by any embodiment of the present disclosure is implemented.

[0011] In a sixth aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program, which implements the method provided by any embodiment of the present disclosure when executed by a processor.

[0012] According to the cloud disk management and control method of the embodiment of the present disclosure, the storage management and control nodes in a storage cluster are horizontally expanded, that is, a storage cluster includes multiple storage management and control nodes, and each storage management and control node is responsible for the management and control of several groups of cloud disks (for example, a collection of multiple cloud disks). On the one hand, each storage management and control node works simultaneously, avoiding the waste of resources caused by a one-master-multiple-backup architecture, improving resource utilization, and supporting large-scale clusters; on the other hand, when a certain storage management and control node fails, only a part of the cloud disks in the storage cluster is affected, so the explosion radius is much smaller than the explosion radius of the centralized BM architecture. Furthermore, for the cloud disk management and control architecture of distributed storage management and control nodes, an effective cluster resource scheduling solution can be provided to ensure that cloud disk scheduling is not affected by architecture updates.

[0013] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present disclosure will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0015] FIG1 shows an architecture diagram of a cloud disk management and control system provided in the related art;

[0016] FIG2 shows an architecture diagram of a cloud disk management and control system provided by an embodiment of the present disclosure;

[0017] FIG3 shows an example diagram of an application of cloud disk scheduling provided by an embodiment of the present disclosure;

[0018] FIG4 shows another example diagram of an application of cloud disk scheduling provided by an embodiment of the present disclosure;

[0019] FIG5 shows a flow chart of a cloud disk management and control method provided by an embodiment of the present disclosure;

[0020] FIG6 shows a flow chart of a cloud disk management and control method provided by an embodiment of the present disclosure;

[0021] FIG7 shows a block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present disclosure. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0023] To facilitate understanding of the technical solutions of the embodiments of the present disclosure, the relevant technologies of the embodiments of the present disclosure are described below. The following relevant technologies are optional solutions that can be arbitrarily combined with the technical solutions of the embodiments of the present disclosure, and they all fall within the protection scope of the embodiments of the present disclosure.

[0024] The following terms are used in the following explanations:

[0025] Block Storage System: A low-latency, persistent, and highly reliable block-level random access storage system that divides data into blocks, each of which operates as an independent hard drive. This means users can freely create their own file systems on these blocks, giving them greater control over their data environment. This makes it particularly suitable for processing large amounts of data, such as databases and large-scale virtual machine environments.

[0026] Cloud Disk: Based on a distributed storage architecture, it provides block-level storage services. Cloud Disk data is persistently stored in the backend distributed storage system, and failure of any node on the computing side will not affect the Cloud Disk.

[0027] Explosion radius: This represents the size of the fault domain, or the area affected by a node failure. A smaller explosion radius indicates a smaller fault domain; a larger explosion radius indicates a larger fault domain.

[0028] Internet Data Center (IDC): A facility that stores, manages, and distributes large amounts of data. It provides a range of services including server hosting, network connectivity, data storage and backup, and security management.

[0029] Availability Zone: This represents a physical location with its own power supply, network connectivity, and security facilities. Within the same geographic region, there may be one or more availability zones. Each availability zone is independent and isolated from the others to prevent problems in one availability zone from affecting others. For example, if one availability zone experiences problems due to a power outage, network outage, or natural disaster, other availability zones can continue to operate normally. In a cloud computing environment, an availability zone includes one or more Internet Data Centers (IDCs).

[0030] Cluster: A cluster is a group of independent computers connected by a high-speed computer network. These computers form a group and are managed as a single system. The computers or servers in a cluster are typically connected by a high-speed network and share storage, software, or other resources, such as a computing cluster or storage cluster.

[0031] Cluster management node: a service node used to manage each storage cluster, such as the resource management node (Resource Master, RM).

[0032] Storage management and control nodes: In the embodiment of the present disclosure, each storage cluster has several storage management and control nodes, such as a block storage backend control node (Block Master, BM), and multiple storage management and control nodes jointly manage a storage cluster.

[0033] Storage service node: Each storage cluster has multiple storage service nodes. Each storage service node is used to write or read data according to the scheduling of the storage control node, such as the block storage backend service node (Block Server, BS).

[0034] Virtual block device (Device): A virtualized storage device that simulates the functions of a physical hard drive, allowing virtual machines to access storage space just like accessing a physical hard drive, thereby reading or writing data in the cloud disk.

[0035] Create a cloud disk (create disk): The system performs resource scheduling based on the cloud disk size and type specified by the user and the current cluster resource usage, including: creating a new cloud disk on the selected storage service node and setting related metadata, such as the size, type, status, etc. of the storage volume; registering the newly created cloud disk with the cluster's metadata service so that users can find and use it.

[0036] Mount a cloud disk (open a disk): Mount a cloud disk to a server instance, such as an Elastic Compute Service (ECS) instance, and use the cloud disk as the data disk of the server instance.

[0037] Garbage Collection (GC): An automatic memory management technique. In a computer system, when objects are no longer used by a program, the garbage collector automatically reclaims the memory space occupied by these objects for reuse. In a cluster environment, GC is the periodic cleanup of unused data blocks or objects to free up storage space and improve resource utilization.

[0038] In related technologies, storage clusters use a centralized BM architecture. Figure 1 shows the architecture of a cloud disk management and control system in related technologies. As shown in Figure 1, a cloud disk management and control system typically includes a client (Client), a cluster management node RM, and at least one storage cluster (Figure 1 uses storage cluster 1 and storage cluster 2 as examples). The storage cluster includes a storage management node BM and multiple storage service nodes BS (Figure 1 uses BS1, BS2, BS3, etc. as examples).

[0039] On the one hand, in a centralized BM architecture, regardless of the size of the storage cluster, a single BM is deployed for each storage cluster. This single BM manages the metadata of the clients, BSs, and cloud disks associated with the entire storage cluster. It is responsible for scheduling cloud disk shards (i.e., segments, with SEG0, SEG1, SEG2, etc. as examples in Figure 1) and allocating background tasks such as garbage collection, completing all cluster-level management and control tasks. A single BM group adopts a one-master, two-backup service model. That is, under normal operating conditions, a single master BM completes all management and control tasks for the storage cluster. When the master BM fails, the backup BM takes over the management and control of the storage cluster. Therefore, for a single cluster, only one BM is operational, while the others are dormant, resulting in a waste of resources. Moreover, if a BM group in a storage cluster fails, even if the other storage cluster has sufficient machines, it will not be able to support the failed storage cluster.

[0040] On the other hand, this centralized management approach has a relatively simple architecture. The client obtains the location of the segment through the BM and can then read and write data to the corresponding BS. The centralized BM is responsible for the management, scheduling, and operation and maintenance of the entire storage cluster. It can work relatively efficiently when the number of cloud disks in a single cluster is small, the number of segments is limited, and the cluster size is small. However, as the scale increases, a single cluster is expected to need to support more segments and cloud disks in the future. If the number of segments to be supported is calculated to reach 10 million in the future, a single BM will need to provide 360G of memory. Obviously, existing servers cannot support this requirement. Moreover, once the centralized BM goes down, all cluster-level management and control operations will fail until one of the three BMs (one master and two backups) can start and provide services normally. Therefore, the explosion radius of a storage cluster with a centralized BM architecture is all the cloud disks in the entire storage cluster.

[0041] The present disclosure provides a cloud disk management solution based on a distributed BM. Figure 2 shows an architecture diagram of a cloud disk management system 200 according to one embodiment of the present disclosure. As shown in Figure 2, the cloud disk management system 200 includes a cluster management node 201 and one or more clusters 202.

[0042] Cluster management node 201 is a service node that manages each cluster 202. For example, cluster management node 201 may be a resource management node RM. For example, when a compute node creates a corresponding storage resource, cluster management node 201, which manages each storage cluster, determines one of the clusters 202 as the target cluster.

[0043] Cluster 202 is a storage cluster. For a cluster 202, its deployment includes multiple storage management and control nodes and multiple storage service nodes. Exemplarily, the storage management and control node can be a block storage backend control node (BM), as shown in FIG2 , and the multiple storage management and control nodes are, for example, BM1, BM2, BM3, and BM4. The storage service node can be a block storage backend service node (BS), as shown in FIG2 , and the multiple storage service nodes are, for example, BS1, BS2, BS3, BS4, BS5, BS6, BS7, and BS8. It should be noted that the embodiment of the present disclosure does not specifically limit the number of clusters 202, and the number of storage management and control nodes and storage service nodes deployed in each cluster.

[0044] Each storage management node is used to manage multiple cloud disk sets (partitions). For example, BM1 can manage cloud disk set 1 and cloud disk set 8 (for cloud disk set list 1), BM2 can manage cloud disk set 2 and cloud disk set 3 (for cloud disk set list 2), BM3 can manage cloud disk set 4 and cloud disk set 6 (for cloud disk set list 3), and BM4 can manage cloud disk set 5 and cloud disk set 7 (for cloud disk set list 4). For a cluster 202, multiple storage management nodes have a one-to-one mapping relationship with multiple cloud disk set lists (partition lists), that is, cloud disk sets and storage management nodes have a mapping relationship.

[0045] A cloud disk collection is a logical concept, representing a collection of cloud disks. A cloud disk can only belong to one cloud disk collection. After a cloud disk collection is created, it must be loaded by the corresponding storage management node before it can provide corresponding services. For example, to maintain logical simplicity, cloud disk collections (partitions) support dynamic creation, but cannot be modified or deleted.

[0046] The storage control node manages and controls its corresponding cloud disk collection, including but not limited to: creating cloud disks and generating corresponding cloud disk identification information (Volume ID) for the created cloud disks; mounting cloud disks (opening disks), deleting cloud disks, and scheduling cloud disks based on the cloud disk identification information.

[0047] The cloud disk identification information includes the cluster ID, the partition ID, and the cloud disk ID (the location of the cloud disk in the cloud disk collection). The location of the cloud disk in the cloud disk collection can be represented by a cloud disk number or a cloud disk random number. For a cloud disk collection, each cloud disk in the cloud disk collection has a unique cloud disk number or a unique cloud disk random number. For example, the cloud disk identification information may also include a reserved field (or preset field), which may identify the reserved or preset functions of the cloud disk or other information.

[0048] For a cluster 202, its deployment may also include a central control node (Central Master, CM), which is responsible for bypass management and control from the global perspective of the cluster 202, including: managing the online and offline of BM in the cluster 202; managing the online and offline of BS in the cluster 202; allocating and creating cloud disk collections, creating and configuring corresponding resources for them, etc.

[0049] For example, the cloud disk management and control system 200 may further include a client 203, which may be deployed in a computing cluster and interact with the cluster management and control node 201 to obtain cloud disk identification information, and then access the corresponding cloud disk based on the cloud disk identification information, such as mounting a cloud disk. The client 203 may interact with the storage management and control node through an application programming interface (API) to obtain information about the storage service node where the segment is located, and then read and write data on the storage service node.

[0050] The following describes an application example of a dynamic disk creation process using a cloud disk management and control system 200 according to an embodiment of the present disclosure in conjunction with FIG2 . The disk creation process mainly includes:

[0051] (1) The cluster management node 201 selects a target storage management node (e.g., BM1) from multiple storage management nodes in the cluster 201. The selection strategy can be random selection or based on a hash rule.

[0052] (2) Cluster management node 201 sends a cloud disk creation request to BM1;

[0053] (3) BM1 can be configured with a proxy or management component. BM1's proxy selects a target cloud disk set (e.g., cloud disk set 8) from multiple cloud disk sets.

[0054] (4) BM1 completes the creation of the target cloud disk, generates the cloud disk identification information (Volume ID) of the target cloud disk, and returns the disk creation success information and the cloud disk identification information of the target cloud disk to the cluster management node 201 through the original path.

[0055] The following describes an application example of a dynamic disk opening process using a cloud disk management and control system 200 according to an embodiment of the present disclosure in conjunction with FIG2 . The opening process mainly includes:

[0056] (1) The client 203 queries the cluster management node 201 for information about the target cluster, the cloud disk identification information of the target cloud disk, and the information about the target storage management node through the device identification information (DEVID) of the virtual block device;

[0057] (2) The cluster management and control node 201 parses the corresponding cloud disk identification information (Volume ID) based on the device identification information provided by the client 203. By parsing the Volume ID, the cluster identification (Cluster ID), the cloud disk set identification (Partition ID), and the cloud disk identification (the location of the cloud disk on the cloud disk set) are obtained. Then, based on the mapping relationship between the cloud disk set and the storage management and control node, the storage management and control node (for example, BM1) corresponding to the cloud disk set identification (Partition ID) is determined as the target storage management and control node.

[0058] (3) The cluster management node 201 returns the cluster identifier (e.g., cluster name) of the target cluster, the Internet Protocol (IP) address of BM1, and the Volume ID to the client 203;

[0059] (4) Client 203 sends a cloud disk mount request for the target cloud disk to BM1 based on BM1's IP address;

[0060] (5) BM1 responds to the cloud disk mount request and mounts the target cloud disk.

[0061] The following describes an application example of cloud disk scheduling based on the cloud disk management and control system of the embodiment of the present disclosure in conjunction with Figure 3. In this application example, the cloud disk scheduling process mainly includes:

[0062] (1) Each BM in the cluster obtains the traffic information of each BS in the cluster. The acquisition method can be that each BS in the cluster sends a heartbeat to the central control node (CM) to report its own traffic information; the acquisition method can also be that each BM in the cluster sends a slow heartbeat to each BS in the cluster, so that each BM can obtain the traffic information of each BS. The slow heartbeat is sent every 10 seconds, for example.

[0063] (2) Each BM takes turns to perform cloud disk scheduling in a preset order. For example, BM1, BM2, BM3, and BM4 take turns to perform cloud disk scheduling, including traffic balancing scheduling (dynamic or static), thereby scheduling cloud disk segments to multiple storage service nodes. For example, CM is used to control each BM to take turns to perform cloud disk scheduling in a preset order according to the global coordination mechanism from the global perspective of the cluster, and to perform certain interventions. If CM fails, each BM will continue to take turns to perform cloud disk scheduling.

[0064] The following describes another application example of cloud disk scheduling based on the cloud disk management and control system of the embodiment of the present disclosure in conjunction with Figure 4. In this application example, the cloud disk scheduling process mainly includes:

[0065] (1) Each BS in the cluster is divided into multiple groups, forming storage service node groups. Each BM corresponds to each storage service node group one by one. The BM has jurisdiction over the storage service node group it corresponds to, including initiating downtime operations on the BS of the storage service node group. For example: BS1 and BS2 form group 1, BM1 corresponds to group 1 and has jurisdiction over BS1 and BS2; BS3 and BS4 form group 2, BM2 corresponds to group 2 and has jurisdiction over BS3 and BS4; BS5 and BS6 form group 3, BM3 corresponds to group 3 and has jurisdiction over BS5 and BS6; BS7 and BS8 form group 4, BM4 corresponds to group 4 and has jurisdiction over BS7 and BS8;

[0066] (2) The BM sends a fast heartbeat to its corresponding BS, for example, once every 200 milliseconds, to obtain the operating status information of the corresponding BS for fault diagnosis (downtime diagnosis) of the BS;

[0067] (3) If the BM determines that there is a faulty BS in the group, that is, a downtime BS, it will perform a downtime operation on the BS. For example, if BM1 determines that BS2 in group 1 is downtime, BM1 will perform a downtime operation on the BS to implement downtime scheduling (also called fault scheduling or abnormal scheduling).

[0068] The following describes an application example of the heartbeat deployment of the cloud disk management and control system of the embodiment of the present disclosure in conjunction with Figure 2. As shown in Figure 2, within a cluster 202, each BM reports a heartbeat to the RM and reports its own operating status information, that is, confirming that the BM is not faulty; cluster-granular information settings are reported by a BM in the cluster (for example, BM1 by default). The central management and control node reports a heartbeat to each BM, obtains the operating status information of each BM, such as whether it is faulty, and obtains the load status of each cloud disk set on the BM.

[0069] The cloud disk management and control system of the embodiment of the present disclosure provides a cloud disk management and control solution based on distributed storage management and control nodes, which horizontally expands the storage management and control nodes within a storage cluster, that is, a storage cluster includes multiple storage management and control nodes, each of which is responsible for the management and control of several cloud disk sets. On the one hand, each storage management and control node works simultaneously, avoiding the waste of resources caused by the one-master-multiple-backup architecture and improving resource utilization; on the other hand, when a storage management and control node fails, other storage management and control nodes take over the cloud disk set it is responsible for. In this process, only a part of the cloud disks in the storage cluster is affected, so the explosion radius is much smaller than the explosion radius of the centralized BM architecture; on the other hand, it can support large-scale clusters. After simulation, the solution of the embodiment of the present disclosure can support larger clusters of 200 or even 1,000 units; on the other hand, the embodiment of the present disclosure also provides an effective cluster resource scheduling solution for the cloud disk management and control architecture of distributed storage management and control nodes, so that cloud disk scheduling is not affected by architecture updates.

[0070] It should be noted that the above application scenarios provided in the embodiments of the present disclosure are for ease of understanding, and the embodiments of the present disclosure do not specifically limit the application scenarios of the technical solutions. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for users to select or edit authorization or rejection.

[0071] The following specific embodiments are used to describe in detail the technical solution of the present disclosure and how the technical solution of the present disclosure solves the above-mentioned technical problems. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0072] FIG5 shows a flow chart of a cloud disk management method according to an embodiment of the present disclosure. The cloud disk management method can be applied to a cluster management node. As shown in FIG5 , the cloud disk management method includes:

[0073] Step S501: Obtain the cloud disk identification information of the target cloud disk; the cloud disk identification information is used to identify the target cloud disk and the target storage management and control node and target cluster corresponding to the target cloud disk; the target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; the target storage management and control node is used to create a target cloud disk on at least one storage service node of the target cluster and generate cloud disk identification information.

[0074] In an embodiment of the present disclosure, the target cluster is a storage cluster, such as the cluster 202 shown in Figure 2. The target cluster includes multiple storage management and control nodes and multiple storage service nodes. For example, as shown in Figure 2, the storage management and control node can be a block storage back-end control node, and as shown in Figure 2, the multiple storage management and control nodes are, for example, BM1, BM2, BM3 and BM4. The storage service node can be a block storage back-end service node, and as shown in Figure 2, the multiple storage service nodes are, for example, BS1, BS2, BS3, BS4, BS5, BS6, BS7 and BS8. The multiple storage management and control nodes include a target storage management and control node, which is a storage management and control node for creating a target cloud disk and generates cloud disk identification information (Volume ID) of the target cloud disk.

[0075] Step S502: Determine the target cluster, target storage management and control node, and target cloud disk according to the cloud disk identification information.

[0076] Cloud disk identification information includes: information used to identify the target cluster, such as the cluster ID; information used to identify the target storage management and control node; and information used to identify the target cloud disk. Based on the target cloud disk's cloud disk identification information, the target cluster, target storage management and control node, and target cloud disk corresponding to the target cloud disk can be determined. This allows the target storage management and control node to perform corresponding management and control on the target cloud disk, including but not limited to mounting, deleting, and scheduling.

[0077] In one embodiment, each storage management and control node is used to manage multiple cloud disk collections (partitions), and the cloud disk identification information includes a cloud disk identification, a cloud disk collection identification, and a cluster identification. In step S501, the target cluster, the target storage management and control node, and the target cloud disk are determined according to the cloud disk identification information, which may include: using the cluster identification, cloud disk collection identification, and cloud disk identification of the target cloud disk to respectively determine the target cluster, the target cloud disk collection to which the target cloud disk belongs, and the position of the target cloud disk on the target cloud disk collection; based on the mapping relationship between the cloud disk collection and the storage management and control node, determine the storage management and control node corresponding to the target cloud disk collection as the target storage management and control node.

[0078] A partition is a logical concept, representing a collection of cloud disks. A cloud disk can only belong to one partition. After a partition is created, it must be loaded by the corresponding storage management node before it can provide corresponding services. For example, to maintain logical simplicity, partitions can be dynamically created but cannot be modified or deleted.

[0079] Each storage management node manages multiple partitions. For a cluster, partitions are mapped to storage management nodes. Therefore, the partition identifier of the target cloud disk can be used to determine the partition to which the target cloud disk belongs. Based on the mapping between partitions and storage management nodes, the corresponding target storage management node information can be obtained. For example, a partition is mapped to the IP address of a BM. Based on the partition identifier of the target cloud disk, the IP address of the target BM can be determined.

[0080] In one embodiment, the method of the embodiment of the present disclosure may also include: selecting a target storage management and control node from multiple storage management and control nodes; sending a cloud disk creation request to the target storage management and control node, so that the target storage management and control node creates a target cloud disk on at least one storage service node and generates cloud disk identification information; and receiving the cloud disk identification information returned by the target storage management and control node.

[0081] Exemplarily, when a computing node creates a corresponding storage resource, the cluster management node decides one of the clusters as the target cluster; the cluster management node can select one from multiple storage management nodes of the target cluster as the target storage management node. The selection strategy can be a random selection or based on a hash rule, which is not specifically limited in the embodiments of the present disclosure; further, the cluster management node sends a cloud disk creation request to the target storage management node, and the target storage management node completes the creation of the target cloud disk, generates the cloud disk identification information (Volume ID) of the target cloud disk, and returns the Volume ID of the target cloud disk to the cluster management node.

[0082] In one embodiment, the client can query the cluster management node for information about the target cluster, cloud disk identification information of the target cloud disk, and information about the target storage management node through the device identification information (DEVID) of the virtual block device.

[0083] Exemplarily, the cluster management and control node can parse the corresponding Volume ID based on the device identification information, and obtain the cluster identification (Cluster ID), cloud disk collection identification (Partition ID) and cloud disk identification (the location of the cloud disk on the cloud disk collection) by parsing the Volume ID. Then, based on the mapping relationship between the cloud disk collection and the storage management and control node, the IP address of the target storage management and control node corresponding to the cloud disk collection identification (Partition ID) is determined, and the cluster identification (for example, the cluster name) of the target cluster, the IP address of the target storage management and control node and the Volume ID are returned to the client; based on this information, the client can send a management request for the target cloud disk to the target storage management and control node, such as a cloud disk mount request, a cloud disk deletion request, etc.

[0084] FIG6 shows a flow chart of a cloud disk management method according to an embodiment of the present disclosure. The cloud disk management method can be applied to a storage management node. The embodiment of the present disclosure takes the target storage management node as an example, i.e., the storage management node that creates the target cloud disk. As shown in FIG6, the cloud disk management method includes:

[0085] Step S601: In response to a cloud disk creation request, create a target cloud disk on at least one storage service node of a target cluster; the target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include a target storage management and control node;

[0086] Step S602: Generate cloud disk identification information of the target cloud disk; the cloud disk identification information is used to identify the target cloud disk, the target storage management and control node, and the target cluster.

[0087] Among them, each storage management and control node is used to manage multiple cloud disk sets (partitions), and the cloud disk identification information includes cluster identification, partition identification and cloud disk identification; among them, the cluster identification of the target cloud disk is used to determine the target cluster; the partition identification of the target cloud disk is used to identify the target partition to which the target cloud disk belongs, so as to determine the storage management and control node corresponding to the target partition as the target storage management and control node based on the mapping relationship between the partition and the storage management and control node; the cloud disk identification of the target cloud disk is used to determine the position of the target cloud disk on the target partition.

[0088] In step S601, a target cloud disk is created on at least one storage service node of the target cluster, including: based on a preset load strategy, determining a target partition from multiple partitions corresponding to the target storage management and control node; selecting a target cloud disk from the target partition, and creating the target cloud disk on at least one storage service node.

[0089] Exemplarily, in response to a cloud disk creation request, the storage management node selects one of the multiple partitions as the target partition. The selection strategy can be a load balancing strategy, that is, based on the number of cloud disks already created for each partition, the partition with the least load is selected as the target partition. The selection strategy can also be a random selection strategy, or a distance-based strategy, that is, selecting a partition that is closer to the computing node network topology as the target partition. This is not limited in the present embodiment.

[0090] The storage management node can be configured with an agent or management component to select a target partition from multiple partitions and send the corresponding operation request to the corresponding storage management node. For details, please refer to the application example of the disk creation process above and will not be repeated here.

[0091] In one embodiment, the method of the embodiment of the present disclosure may further include: mounting the target cloud disk in response to a cloud disk mount request for the target cloud disk. For details, please refer to the application example of the dynamic disk opening process above, which will not be repeated here.

[0092] According to the cloud disk management and control method of the embodiment of the present disclosure, the storage management and control nodes within a storage cluster are horizontally expanded, that is, a storage cluster includes multiple storage management and control nodes, each of which is responsible for the management and control of several groups of cloud disks (for example, a collection of multiple cloud disks). On the one hand, each storage management and control node works simultaneously, avoiding the resource waste caused by the one-master-multiple-backup architecture, improving resource utilization, and supporting large-scale clusters; on the other hand, in the event of a failure of a storage management and control node, only a portion of the cloud disks in the storage cluster are affected. Therefore, the explosion radius is much smaller than the explosion radius of a centralized BM (one-master-multiple-backup) architecture.

[0093] In one embodiment, multiple storage management and control nodes take turns to execute cloud disk scheduling in a preset order. The method executed by the target storage management and control node in the embodiment of the present disclosure may also include: obtaining traffic information of each storage service node in the target cluster; in response to the target storage management and control node's turn to execute cloud disk scheduling, scheduling at least one cloud disk corresponding to the target storage management and control node based on the traffic information of each storage service node.

[0094] Exemplarily, each storage management and control node in the target cluster obtains the traffic information of each storage service node in the target cluster. The acquisition method can be that each storage service node in the target cluster sends a heartbeat to the central management and control node (CM) to report its own traffic information; the acquisition method can also be that each storage management and control node in the target cluster sends a slow heartbeat to each storage service node in the target cluster respectively, so that the target storage management and control node can obtain the traffic information of each storage service node in the target cluster. Furthermore, each storage management and control node performs cloud disk scheduling in turn according to a preset order. When it is the turn of the target storage management and control node to perform cloud disk scheduling, the target storage management and control node can perform cloud disk scheduling on the cloud disks on each part it controls, including dynamic traffic balancing scheduling or static traffic balancing scheduling.

[0095] In one embodiment, each storage service node of the target cluster is divided into multiple storage service node groups, and multiple storage management and control nodes correspond one-to-one to the multiple storage service node groups. The method executed by the target storage management and control node in the embodiment of the present disclosure may also include: based on the operating status information of each storage service node in the storage service node group corresponding to the target storage management and control node, determining whether there is a faulty storage service node in the storage service node group corresponding to the target storage management and control node; in response to the presence of a faulty storage service node, performing a downtime operation on the faulty storage service node. For details, please refer to the cloud disk scheduling application example in conjunction with Figure 4, which will not be repeated here.

[0096] In one embodiment, the method executed by the target storage management and control node in the disclosed embodiment may further include: in response to a failed storage management and control node in the target cluster, managing the cloud disks corresponding to the failed storage management and control node. In other words, if a storage management and control node fails, other storage management and control nodes will take over the cloud disks for which it is responsible.

[0097] According to the cloud disk management and control method of the embodiment of the present disclosure, an effective cluster resource scheduling solution can be provided for the cloud disk management and control architecture of the distributed storage management and control nodes, so that the cloud disk scheduling is not affected by the architecture update.

[0098] Corresponding to the method provided by the embodiment of the present disclosure, the embodiment of the present disclosure provides a cloud disk management and control device, which is applied to a cluster management and control node and may include: a cloud disk identification information acquisition module, used to obtain the cloud disk identification information of the target cloud disk; the cloud disk identification information is used to identify the target cloud disk and the target storage management and control node and target cluster corresponding to the target cloud disk; the target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; the target storage management and control node is used to create the target cloud disk on at least one storage service node of the target cluster and generate the cloud disk identification information; a target storage management and control node determination module, used to determine the target cluster, the target storage management and control node and the target cloud disk according to the cloud disk identification information.

[0099] In one embodiment, each storage management and control node is used to manage multiple cloud disk collections, and the cloud disk identification information includes a cloud disk identification, a cloud disk collection identification, and a cluster identification. The target storage management and control node determination module is specifically used to: use the cluster identification, cloud disk collection identification, and cloud disk identification of the target cloud disk to respectively determine the target cluster, the target cloud disk collection to which the target cloud disk belongs, and the position of the target cloud disk on the target cloud disk collection; based on the mapping relationship between the cloud disk collection and the storage management and control node, determine the storage management and control node corresponding to the target cloud disk collection as the target storage management and control node.

[0100] In one embodiment, the device may also include: a target storage management and control node selection module, used to select the target storage management and control node from the multiple storage management and control nodes; a cloud disk creation request sending module, used to send a cloud disk creation request to the target storage management and control node, so that the target storage management and control node creates the target cloud disk on the at least one storage service node and generates the cloud disk identification information; a cloud disk identification information receiving module, used to receive the cloud disk identification information returned by the target storage management and control node.

[0101] Corresponding to the method provided in the embodiment of the present disclosure, the embodiment of the present disclosure provides a cloud disk management and control device, which is applied to the target storage management and control node and may include: a target cloud disk creation module, used to create a target cloud disk on at least one storage service node of the target cluster in response to a cloud disk creation request; the target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; a cloud disk identification information generation module, used to generate cloud disk identification information of the target cloud disk; the cloud disk identification information is used to identify the target cloud disk, the target storage management and control node and the target cluster.

[0102] In one embodiment, each storage management and control node is used to manage multiple cloud disk collections, and the cloud disk identification information includes a cluster identification, a cloud disk collection identification, and a cloud disk identification; wherein, the cluster identification of the target cloud disk is used to determine the target cluster; the cloud disk collection identification of the target cloud disk is used to identify the target cloud disk collection to which the target cloud disk belongs, so as to determine the storage management and control node corresponding to the target cloud disk collection as the target storage management and control node based on the mapping relationship between the cloud disk collection and the storage management and control node; the cloud disk identification of the target cloud disk is used to determine the position of the target cloud disk on the target cloud disk collection.

[0103] In one embodiment, the target cloud disk creation module is specifically used to: determine the target cloud disk set from multiple cloud disk sets corresponding to the target storage management and control node based on a preset load strategy; select the target cloud disk from the target cloud disk set, and create the target cloud disk on the at least one storage service node.

[0104] In one embodiment, the device may further include a mounting module for mounting the target cloud disk in response to a cloud disk mounting request for the target cloud disk.

[0105] In one embodiment, the multiple storage management and control nodes perform cloud disk scheduling in turn according to a preset order. The device may also include: a traffic information acquisition module, used to obtain the traffic information of each storage service node in the target cluster; a scheduling module, used to schedule at least one cloud disk corresponding to the target storage management and control node based on the traffic information of each storage service node in response to the target storage management and control node performing cloud disk scheduling.

[0106] In one embodiment, each storage service node of the target cluster is divided into multiple storage service node groups, and the multiple storage management and control nodes correspond one-to-one to the multiple storage service node groups. The device may also include: a judgment module for judging whether there is a faulty storage service node in the storage service node group corresponding to the target storage management and control node based on the operating status information of each storage service node in the storage service node group corresponding to the target storage management and control node; and a downtime operation module for performing a downtime operation on the faulty storage service node in response to the existence of the faulty storage service node.

[0107] In one embodiment, the device further includes a fault management and control module for managing each cloud disk corresponding to the faulty storage management and control node in response to the presence of a faulty storage management and control node in the target cluster.

[0108] The functions of each module in each device of the embodiment of the present disclosure can be referred to the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0109] Figure 7 is a block diagram of an electronic device used to implement embodiments of the present disclosure. As shown in Figure 7 , the electronic device includes a memory 701 and a processor 702. The memory 701 stores a computer program executable by the processor 702. When the processor 702 executes the computer program, the method described in the above embodiments is implemented. The number of memory 701 and processor 702 can be one or more.

[0110] The electronic device further includes a communication interface 703 for communicating with external devices and performing data exchange transmission.

[0111] If the memory 701, processor 702, and communication interface 703 are implemented independently, the memory 701, processor 702, and communication interface 703 can be interconnected via a bus and communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG7 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0112] Optionally, in a specific implementation, if the memory 701 , the processor 702 , and the communication interface 703 are integrated on a chip, the memory 701 , the processor 702 , and the communication interface 703 may communicate with each other through an internal interface.

[0113] An embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, which implements the method provided in any embodiment of the present disclosure when the program is executed by a processor.

[0114] An embodiment of the present disclosure provides a computer program product, including a computer program. When the computer program is executed by a processor, the method provided in any embodiment of the present disclosure is implemented.

[0115] An embodiment of the present disclosure further provides a chip, which includes a processor for calling and executing instructions stored in a memory, so that a communication device equipped with the chip executes a method provided by any embodiment of the present disclosure.

[0116] An embodiment of the present disclosure also provides a chip, including: an input interface, an output interface, a processor and a memory. The input interface, the output interface, the processor and the memory are connected through an internal connection path. The processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in any embodiment of the application.

[0117] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0118] Optionally, the above-mentioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory may include random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example: Static RAM (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Sync Link DRAM (SLDRAM) and Direct Rambus RAM (DRRAM).

[0119] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present disclosure are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0120] It should be noted that the user information (including but not limited to user device information, user personal information, user operation information, etc.) and data (including but not limited to data used for processing, analysis, storage, display, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant information and data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0121] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means two or more, unless otherwise specifically defined.

[0123] Any process or method description described in a flowchart or otherwise herein can be understood to represent a module, segment, or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process. The scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed in a different order than shown or discussed, including in a substantially simultaneous manner or in a reverse order depending on the functions involved.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in conjunction with such instruction execution systems, apparatuses or devices.

[0125] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0126] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the aforementioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0127] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope disclosed in the present disclosure, and such modifications or substitutions should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A cloud disk management and control method, applied to a cluster management and control node, the method comprising: Get the cloud disk identification information of the target cloud disk; The cloud disk identification information is used to identify the target cloud disk and the target storage management and control node and the target cluster corresponding to the target cloud disk; The target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; The target storage management and control node is used to create the target cloud disk on at least one storage service node of the target cluster and generate the cloud disk identification information; The target cluster, the target storage management and control node, and the target cloud disk are determined according to the cloud disk identification information.

2. The method according to claim 1, wherein: Each storage management and control node is used to manage multiple cloud disk sets, and the cloud disk identification information includes a cloud disk identification, a cloud disk set identification, and a cluster identification. Determining the target cluster, the target storage management and control node, and the target cloud disk according to the cloud disk identification information includes: Using the cluster identifier, the cloud disk set identifier, and the cloud disk identifier of the target cloud disk, respectively determine the target cluster, the target cloud disk set to which the target cloud disk belongs, and the location of the target cloud disk on the target cloud disk set; Based on the mapping relationship between the cloud disk set and the storage management and control node, a storage management and control node corresponding to the target cloud disk set is determined as the target storage management and control node.

3. The method according to claim 1, further comprising: Selecting the target storage management and control node from the multiple storage management and control nodes; Sending a cloud disk creation request to the target storage management and control node, so that the target storage management and control node creates the target cloud disk on the at least one storage service node and generates the cloud disk identification information; Receive the cloud disk identification information returned by the target storage management and control node.

4. A cloud disk management and control method, applied to a target storage management and control node, the method comprising: In response to the cloud disk creation request, create a target cloud disk on at least one storage service node of the target cluster; The target cluster includes multiple storage management and control nodes, and the multiple storage management and control nodes include the target storage management and control node; Generate cloud disk identification information of the target cloud disk; The cloud disk identification information is used to identify the target cloud disk, the target storage management and control node, and the target cluster.

5. The method according to claim 4, wherein: Each storage management and control node is used to manage multiple cloud disk sets, and the cloud disk identification information includes a cluster identification, a cloud disk set identification and a cloud disk identification; wherein the cluster identification of the target cloud disk is used to determine the target cluster; the cloud disk set identification of the target cloud disk is used to identify the target cloud disk set to which the target cloud disk belongs, so as to determine the storage management and control node corresponding to the target cloud disk set as the target storage management and control node based on the mapping relationship between the cloud disk set and the storage management and control node; the cloud disk identification of the target cloud disk is used to determine the position of the target cloud disk on the target cloud disk set.

6. The method according to claim 5, wherein: Create a target cloud disk on at least one storage service node in the target cluster, including: Based on a preset load strategy, determining the target cloud disk set from a plurality of cloud disk sets corresponding to the target storage management and control node; The target cloud disk is selected from the target cloud disk set, and the target cloud disk is created on the at least one storage service node.

7. The method according to claim 4, wherein: Also includes: In response to a cloud disk mount request for the target cloud disk, the target cloud disk is mounted.

8. The method according to any one of claims 4 to 7, wherein: The plurality of storage control nodes perform cloud disk scheduling in turn according to a preset order, and the method further includes: Obtaining traffic information of each storage service node in the target cluster; In response to the target storage control node's turn to execute cloud disk scheduling, at least one cloud disk corresponding to the target storage control node is scheduled based on traffic information of each storage service node.

9. The method according to any one of claims 4 to 7, wherein: The storage service nodes of the target cluster are divided into a plurality of storage service node groups, the plurality of storage management and control nodes correspond one-to-one to the plurality of storage service node groups, and the method further includes: Based on the running status information of each storage service node in the storage service node group corresponding to the target storage management and control node, it is determined whether there is a faulty storage service node in the storage service node group corresponding to the target storage management and control node.

10. The method according to claim 9, further comprising: In response to the existence of the faulty storage service node, a downtime operation is performed on the faulty storage service node.

11. The method according to any one of claims 4 to 7, further comprising: In response to a faulty storage management and control node in the target cluster, each cloud disk corresponding to the faulty storage management and control node is managed and controlled.

12. A cloud disk management and control system, comprising: A cluster management and control node, used to execute the method described in any one of claims 1 to 3; At least one cluster, the cluster comprising a plurality of storage management and control nodes and a plurality of storage service nodes, the storage management and control nodes being used to execute the method described in any one of claims 4 to 11.

13. The cloud disk management and control system according to claim 12, wherein: Each cluster also includes a central management and control node, which is used to send the traffic information of each storage service node in the cluster to each storage management and control node, and control each storage management and control node in the cluster to perform cloud disk scheduling in turn according to a preset order; and / or, the central management and control node is used to allocate and create multiple cloud disk collections for each storage management and control node.

14. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 11 when executing the computer program.

15. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

16. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images