Method, system and equipment for online upgrading of storage system and medium

Abstract

The invention relates to a method, a system and equipment for online upgrading of a storage system and a medium, and belongs to the technical field of distributed storage, and the method comprises the following steps: determining at least one node to be upgraded in the storage system; any node to be upgraded is selected as a control node to receive the mirror image file package, and a mirror image source is created based on the mirror image file package; and determining at least one node cluster according to the node cluster category to which the to-be-upgraded node belongs, determining the upgrading sequence of each node cluster according to the upgrading priority of the node cluster, and simultaneously upgrading the plurality of to-be-upgraded nodes which belong to the same node cluster and belong to the same storage pool. Through the upgrading method provided by the invention, rapid and reliable upgrading iteration can be realized on the premise of not influencing service operation.

Description

Technical Field

[0001] The present application relates to the field of distributed storage, and more specifically, to a method, system, device and medium for online upgrading of a storage system. Background Art

[0002] After a distributed storage cluster is launched, the storage system of the distributed storage cluster needs to be upgraded regularly to fix vulnerabilities, enhance performance, or experience new version features.

[0003] Traditional distributed storage systems do not support mirroring system kernel versions at the same time. The upgrade methods are mostly offline upgrades or serial online upgrades. These methods require interrupting business operations during the upgrade process and basically only support partial content upgrades of distributed storage systems. The upgrade efficiency is low and the upgrade effect is poor. For industries such as finance and communications that have high requirements for business continuity and upgrade timeliness, this obviously cannot meet market demand. Summary of the invention

[0004] To solve the above problems, the present application provides a method, system, device and medium for online upgrading of a storage system.

[0005] A first aspect of an embodiment of the present application provides a method for online upgrading of a storage system, wherein the storage system includes at least one storage pool, each storage pool includes one or more nodes, including:

[0006] Determining at least one node to be upgraded in the storage system;

[0007] Select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package;

[0008] Determine at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster according to the upgrade priority of the node cluster. The upgrade priorities of the nodes to be upgraded in the same node cluster are the same, and the upgrade priorities of the nodes to be upgraded in different node clusters are different.

[0009] According to the mirror source, the nodes to be upgraded of each node cluster are upgraded in the upgrade order of each node cluster. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously;

[0010] After all nodes to be upgraded are upgraded, the upgraded storage system is obtained.

[0011] Optionally, determining at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determining the upgrade order of each node cluster according to the upgrade priority of the node cluster specifically includes:

[0012] Classifying at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster and a storage monitoring node cluster;

[0013] Create a first-order upgrade queue between classes based on the data node cluster, metadata node cluster, and storage monitoring node cluster;

[0014] Creating a first concurrent upgrade queue for nodes in the data node cluster according to a value of a storage pool in the storage system and a value of a fault domain in the storage pool;

[0015] Creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain;

[0016] Creating a third concurrent upgrade queue of nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain;

[0017] The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue and the third concurrent upgrade queue are determined as the upgrade sequence of the node clusters.

[0018] Optionally, creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain specifically includes: obtaining a maximum upgrade quantity in the storage pool according to the value of the storage pool;

[0019] Obtaining the concurrent number of a single upgrade in the storage pool according to the value of the fault domain;

[0020] A second concurrent upgrade queue for nodes in the metadata node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0021] Optionally, the step of upgrading the nodes to be upgraded of each node cluster according to the image source and in the upgrade order of each node cluster specifically includes:

[0022] Upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source;

[0023] Simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source;

[0024] Simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source;

[0025] The nodes in the storage monitoring node cluster in the third concurrent upgrade queue are upgraded simultaneously according to the mirror source.

[0026] Optionally, it also includes:

[0027] When an upgrade failure occurs in the node to be upgraded, the node is upgraded again through the first fault handling mode. When the upgrade in the first fault handling mode succeeds, the upgrade of the faulty node is completed;

[0028] When the first fault handling mode upgrade fails, the second fault handling mode is used to upgrade the faulty node; or, when the first fault handling mode upgrade fails, the second fault handling mode and the third fault handling mode are used to continue upgrading the faulty node to complete the faulty node upgrade.

[0029] Optionally, before the storage system is upgraded online, the following steps are also performed:

[0030] Backing up system data in the storage system;

[0031] Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes with normal node status;

[0032] Check the working status of the hardware device in the storage system to determine that the working status belongs to a preset upgrade status.

[0033] Optionally, before creating the image source based on the image file package, the method further includes:

[0034] Receive an image file package and a signature file corresponding to the image file package;

[0035] The image file package is verified according to the signature file.

[0036] Optionally, the upgrading the node to be upgraded specifically includes:

[0037] At least one of the kernel, driver, and storage version of the node to be upgraded is upgraded according to the image source.

[0038] Optionally, it also includes:

[0039] After the upgrade is completed, all nodes of the upgraded storage system are verified to determine whether the kernel version, driver version, and storage version of all nodes meet the version conditions for a successful upgrade; and the services of all nodes of the upgraded storage system are verified to determine whether the services of all nodes meet the service conditions for a successful upgrade;

[0040] When the kernel versions, driver versions, and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful.

[0041] A second aspect of an embodiment of the present application provides a distributed storage system, the distributed storage system comprising at least one storage pool, each storage pool comprising one or more nodes, including:

[0042] A first selection module, configured to determine at least one node to be upgraded in the storage system;

[0043] A second selection module is used to select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package;

[0044] A creation module is used to determine at least one node cluster according to the node category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster according to the upgrade priority of the node cluster. The upgrade priority of the nodes to be upgraded in the same node cluster is the same, and the upgrade priority of the nodes to be upgraded in different node clusters is different;

[0045] The upgrade module is used to upgrade the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; after the upgrade of all nodes to be upgraded is completed, an upgraded storage system is obtained.

[0046] A third aspect of an embodiment of the present application provides an electronic device, including a memory and a processor, wherein:

[0047] The memory is used to store programs;

[0048] The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in a storage system online upgrade method of any of the above solutions.

[0049] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of a storage system online upgrade method of any of the above-mentioned schemes.

[0050] The fifth aspect is a computer program product. The fifth aspect of the embodiment of the present invention provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the method for online upgrading of the storage system as described in the first aspect of the embodiment of the present invention are implemented.

[0051] By applying the technical solution provided in the embodiments of the present application, during the storage system upgrade process, the upgrade order of each node to be upgraded is determined through the upgrade order of each node cluster, and the upgrade tasks of multiple nodes in the same storage pool are started simultaneously according to the upgrade order, so as to achieve concurrent upgrades and effectively improve the upgrade efficiency. Based on the upgrade order of each node cluster, it is always ensured that there are nodes in the system that provide services for business services, so that the storage system upgrade method of the present application can achieve fast and reliable upgrade iterations without affecting business operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 It is a flowchart of the steps of a method for online upgrading of a storage system provided in an embodiment of the present application;

[0054] Figure 2 is a flowchart of the steps of creating an upgrade sequence provided in an embodiment of the present application;

[0055] Figure 3 It is a structural diagram of the upgrade sequence provided in the embodiment of the present application;

[0056] Figure 4 It is a schematic diagram of a metadata node provided in an embodiment of the present application;

[0057] Figure 5 It is a flowchart of the steps of the method for creating a second concurrent upgrade queue of nodes in a metadata node cluster provided in an embodiment of the present application;

[0058] Figure 6 It is a flowchart of the steps of the process of upgrading the node to be upgraded provided in the embodiment of the present application;

[0059] Figure 7 It is a data node structure diagram provided in an embodiment of the present application;

[0060] Figure 8 It is a schematic diagram of a storage monitoring node provided in an embodiment of the present application;

[0061] Fig. 9 It is a schematic diagram of the entire upgrade process provided in the embodiment of the present application;

[0062] Fig.10 It is a schematic diagram of the main upgrade process provided in the embodiment of the present application;

[0063] Fig.11 is a structural block diagram of a distributed storage system provided in an embodiment of the present application;

[0064] Fig.12 It is a hardware structure block diagram of an electronic device provided in each embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] The present application relates to the field of distributed storage technology, and is a method for online upgrading of the kernel and storage version of a distributed storage system, which aims to enable the storage system to quickly and safely complete the upgrade of the kernel and storage version without interrupting business operations, and supports upgrade pause and resume functions to improve system availability and maintenance efficiency.

[0067] The kernel is the core of an operating system. It is the first layer of software expansion based on hardware, providing the most basic functions of the operating system. It is the basis for the operation of the operating system. It is responsible for managing the system's processes, memory, device drivers, files, and network systems, and determines the performance and stability of the system. Device drivers are software used in the operating system to control and manage hardware devices. They allow the operating system to communicate with hardware devices and perform necessary operations.

[0068] In modern operating system design, in order to reduce the system's own overhead, some modules closely related to hardware, such as interrupt handlers, device drivers, etc., basic, common, high-frequency modules (such as clock management modules, process scheduling modules, etc.) and other data structures are often separated and made resident in memory, and they are protected. This part is usually called the kernel of the operating system.

[0069] Reference Figure 1 As shown, a flowchart of a method for online upgrading of a storage system is shown. The method can be applied to a distributed storage system, such as Figure 1 As shown, the following steps may be specifically included:

[0070] Step S101, determining at least one node to be upgraded in the storage system; in one embodiment, the storage system may be a distributed storage system.

[0071] Step S102, select any node to be upgraded as the control node to receive the image file package, and create an image source based on the image file package; for example, the image file package can be an ISO (International Organization for Standardization) image file package, iso is one of the storage formats of CD mirror on the computer, because it is a file stored according to ISO*9660 CD-ROM file system standard, so it is usually named with the suffix .iso in the computer. It has only one file in form, which can truly reflect the content of the CD, and can be created by burning software or image file creation tools. The image source refers to the server address that provides software package download services, and the server address is created based on the iso image package. The address contains the kernel, driver and storage version of the storage system required for the upgrade.

[0072] Step S103, determining at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determining the upgrade order of each node cluster according to the upgrade priority of the node cluster. The upgrade priorities of the nodes to be upgraded in the same node cluster are the same, and the upgrade priorities of the nodes to be upgraded in different node clusters are different;

[0073] Step S104, upgrading the nodes to be upgraded of each node cluster according to the upgrade order of each node cluster according to the mirror source, and during the upgrade process, upgrading multiple nodes to be upgraded belonging to the same node cluster and the same storage pool at the same time;

[0074] In a certain embodiment, the kernel of the node to be upgraded is first upgraded according to the image source, and after the kernel upgrade is completed, the driver and storage versions are upgraded in sequence.

[0075] Step S105, after all nodes to be upgraded are upgraded, an upgraded storage system is obtained.

[0076] In this embodiment, before upgrading, the cluster status, network status and hardware equipment of the nodes of the storage system are checked to ensure that the upgrade proceeds normally and to eliminate nodes that do not meet the upgrade conditions. At least one node to be upgraded is determined in the storage system. Usually, there are multiple storage nodes that need to be upgraded.

[0077] The storage system selects a central control node from multiple nodes to be upgraded to receive the image file package and signature file uploaded on the interface. The signature file is a file that signs the image file package. The image file package is verified by the signature file to verify the integrity and correctness of the image file. The image source is created through the image file package and saved locally on the control node. When other nodes to be upgraded are upgraded, they download the image source from the control node for upgrade.

[0078] According to the mirror source, the nodes to be upgraded of each node cluster are upgraded in the upgrade order of each node cluster. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; during the upgrade process, business requests are processed through nodes in a non-upgraded state or upgraded nodes; after executing the upgrade of all nodes to be upgraded, an upgraded storage system is obtained.

[0079] During the storage system upgrade process, the upgrade queue upgrade technology is adopted to determine the upgrade order of each node to be upgraded, and the upgrade tasks of multiple nodes in the same storage pool are started simultaneously according to the upgrade order, so as to realize concurrent upgrade and effectively improve the upgrade efficiency. Based on the execution order of the upgrade queue, it is always ensured that there are service-providing nodes in the system for business services, so that the storage system upgrade method based on the upgrade queue of the present application can realize fast and reliable upgrade iteration without affecting business operations.

[0080] As an implementation method of this application, Figure 2 As shown, a flowchart of the steps of creating an upgrade queue is shown, which may specifically include:

[0081] Step S201, classifying at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster and a storage monitoring node cluster;

[0082] Step S202, creating a first order upgrade queue between classes according to the data node cluster, the metadata node cluster and the storage monitoring node cluster;

[0083] Step S203, creating a first concurrent upgrade queue of nodes in the data node cluster according to the value of the storage pool in the storage system and the value of the fault domain in the storage pool;

[0084] Step S204, creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain;

[0085] Step S205, creating a third concurrent upgrade queue of nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain;

[0086] Step S206: determining the first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue and the third concurrent upgrade queue as the upgrade sequence of the node clusters.

[0087] In this embodiment, if Figure 3As shown, a structural diagram of the upgrade queue is shown. According to the underlying data distribution structure of the node cluster, the storage nodes are classified, and different types of nodes are identified, such as data nodes, metadata nodes, and storage monitoring nodes, to obtain at least one node cluster; wherein, the at least one node cluster includes: a data node cluster, a metadata node cluster, and a storage monitoring node cluster.

[0088] According to the data node cluster, metadata node cluster and storage monitoring node cluster, a first-order upgrade queue is created between classes; the upgrade priorities of the data node cluster, metadata node cluster and storage monitoring node cluster decrease in sequence, and the upgrade order is: data node cluster, metadata node cluster and storage monitoring node cluster.

[0089] Based on the value of the storage pool, it is determined how many nodes can be upgraded at the same time in the storage pool. Combined with the value of the fault domain of the storage pool, it is determined how many nodes can be upgraded at the same time and the business services can be guaranteed not to be affected and the load of the storage system is normal. In an example, Figure 4 The figure shows the structure of the metadata node. When the storage pool value is 4 and the fault domain value is 3, it means that nodes 1, 2, and 3 can be upgraded at the same time. When the storage pool value is 4 and the fault domain value is 2, it means that any two nodes among nodes 1, 2, and 3 can be upgraded at the same time. After upgrading at least one of them, the third node can be upgraded. When the number of concurrent upgrades exceeds the fault domain value, it will affect business services. The single concurrent number is less than or equal to the fault domain value. The value of the storage pool ≥ the fault domain value ≥ the single concurrent number.

[0090] According to the value of the storage pool and the value of the fault domain, in combination with the above method, a second concurrent upgrade queue of nodes in the metadata node cluster is created;

[0091] The creation methods of the first concurrent upgrade queue, the second concurrent upgrade queue and the third concurrent upgrade queue are the same and will not be described in detail.

[0092] The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue and the third concurrent upgrade queue are determined as the upgrade queues.

[0093] The first-order upgrade queue created by node classification, storage pool value and fault domain value in the storage pool ensures that there are always metadata nodes to provide services during the upgrade process, and realizes fast and safe upgrade of the storage system without interrupting business operations. Through multiple concurrent upgrade queues, the upgrade process of each storage node is coordinated to control the storage business from exceeding the fault domain and not affecting the normal use of the storage business, thereby realizing concurrent upgrades and effectively improving the speed of storage system upgrades.

[0094] As an implementation method of this application, Figure 5As shown, a flowchart of the steps of creating a second concurrent upgrade queue of nodes in a metadata node cluster is shown, which may specifically include:

[0095] Step S501, obtaining the maximum upgrade quantity in the storage pool according to the value of the storage pool;

[0096] Step S502, obtaining the concurrent number of a single upgrade in the storage pool according to the value of the fault domain;

[0097] Step S503: Create a second concurrent upgrade queue for nodes in the metadata node cluster according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0098] For example, the concurrent number of a single upgrade in the storage pool is obtained according to the value of the fault domain;

[0099] A first concurrent upgrade queue of nodes in the data node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0100] As another example, the concurrent number of a single upgrade in the storage pool is obtained according to the value of the fault domain;

[0101] A third concurrent upgrade queue of nodes in the storage monitoring node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0102] In this embodiment, according to the value of the storage pool and the value of the fault domain, based on the value of the storage pool, that is, how many nodes can be upgraded at the same time in the storage pool; combined with the value of the fault domain of the storage pool, it can be determined how many nodes can be upgraded at the same time and the business service can be guaranteed not to be affected and the load of the storage system is normal, according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool, a second concurrent upgrade queue of the nodes in the metadata node cluster is created. In one example, as Figure 4 As shown in the figure, when the storage pool value is 4 and the fault domain value is 3, it means that node 1, node 2 and node 3 can be upgraded at the same time; when the storage pool value is 4 and the fault domain value is 2, it means that two nodes among node 1, node 2 and node 3 can be upgraded at the same time, and the third node can be upgraded after at least one of them is upgraded.

[0103] As an implementation mode of the present application, after creating a second concurrent upgrade queue of nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain, it also includes: the value of the storage pool is greater than or equal to the value of the fault domain;

[0104] Determine whether the value of the fault domain is greater than the value of the number of nodes in the metadata node cluster, and if not, upgrade the nodes in the metadata node cluster according to the second concurrent upgrade queue;

[0105] If so, the second concurrent upgrade queue is divided into a first sub-concurrent upgrade queue and a second sub-concurrent upgrade queue, and the nodes in the metadata node cluster are upgraded in the order of the first sub-concurrent upgrade queue and the second sub-concurrent upgrade queue.

[0106] In this embodiment, when the value of the fault domain is less than or equal to the value of the number of nodes in the metadata node cluster, it means that in the storage pool, the current load capacity of the current storage system can simultaneously upgrade some nodes in the metadata node cluster, and there are still idle nodes (nodes in non-upgraded state) to provide business services.

[0107] When the value of the fault domain is greater than the value of the number of nodes in the metadata node cluster, it means that in the storage pool, the current load capacity of the current storage system can upgrade all the nodes in the metadata node cluster at the same time, and there are no idle nodes to provide services. Therefore, through more refined regulation, the nodes in the metadata node cluster are further updated in batches, and the nodes in the metadata node cluster are upgraded in the order of the first sub-concurrent upgrade queue and the second sub-concurrent upgrade queue, thereby ensuring the upgrade efficiency of the metadata nodes and further ensuring the integrity of the business services.

[0108] As an implementation method of this application, Figure 6 As shown, a flowchart of the steps of upgrading the node to be upgraded is shown, which may specifically include:

[0109] Step S601, upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source;

[0110] Step S602, simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source;

[0111] Step S 603, simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source;

[0112] Step S 604 : upgrading the nodes in the storage monitoring node cluster in the third concurrent upgrade queue simultaneously according to the mirror source.

[0113] In this embodiment, if Figure 3 As shown, the first order upgrade queue is the data node cluster, the metadata node cluster and the storage monitoring node cluster; according to the upgrade order of the data node, the metadata node and the storage monitoring node, different types of nodes are upgraded according to the mirror source.

[0114] The first concurrent upgrade queue is represented by: Figure 7 As shown, the structure of the data node is shown, wherein the multiple data nodes include: node 1, node 2, node 3 and node 4, and the upgrade order of node 1, node 2, node 3 and node 4 represents the first concurrent upgrade queue. Node 1, node 2, node 3 and node 4 in the data node select to upgrade node 1, node 2, node 3 and node 4 simultaneously according to the fault domain value of the storage pool and the storage system load capacity. In the case of insufficient load capacity, the number of single concurrent upgrades is reduced by combining simultaneous upgrades with sequential upgrades. For example, if the fault domain value is 2 and the number of single concurrent upgrade nodes is 2, the upgrades of node 1, node 2, node 3 and node 4 need to be completed through two concurrent upgrades.

[0115] The second concurrent upgrade queue is represented by: Figure 4 As shown, the structure of the metadata node is shown, wherein multiple metadata nodes include: node 1 node 2 and node 3, the upgrade order of node 1, node 2 and node 3 represents the second concurrent upgrade queue, and node 1, node 2 and node 3 in the metadata node select to upgrade node 1, node 2 and node 3 simultaneously according to the fault domain value of the storage pool and the storage system load capacity.

[0116] The third concurrent upgrade queue indicates: Figure 8 As shown, the structure of the storage monitoring node is shown, wherein the multiple storage monitoring nodes include: node 1, node 2 and node 3, and the upgrade order of node 1, node 2 and node 3 represents the third concurrent upgrade queue. Node 1, node 2 and node 3 in the storage monitoring node select to upgrade node 1, node 2 and node 3 simultaneously according to the fault domain value of the storage pool and the storage system load capacity.

[0117] In one embodiment, the upgrade queue generator formulates an upgrade strategy based on the load of the cluster, that is, based on the current business load, controls the number of single upgrades in the storage pool, that is, the fault domain, and the node cluster type and other factors.

[0118] The concurrent upgrade technology is used to start the upgrade tasks of multiple nodes at the same time based on multiple concurrent upgrade queues. The first-order upgrade queue is used to ensure that the system always has nodes providing business services, ensuring that the cluster business of the storage system is not affected during the upgrade process.

[0119] As an implementation mode of the present application, when an upgrade failure occurs in the node to be upgraded, the node is re-upgraded through the first fault handling mode. When the upgrade in the first fault handling mode is successful, the upgrade of the faulty node is completed. In an embodiment, the first fault handling mode can be a rollback re-upgrade.

[0120] When the upgrade in the first fault handling mode fails, the upgrade is performed through the second fault handling mode to complete the upgrade of the faulty node; in one embodiment, the second fault handling mode may be a manual recovery upgrade; or, when the upgrade in the first fault handling mode fails, the faulty node is continuously upgraded through the second fault handling mode and the third fault handling mode to complete the upgrade of the faulty node. In one embodiment, the third fault handling mode may be an automatic filling upgrade.

[0121] In this embodiment, when an upgrade failure occurs in the node to be upgraded, the upgrade is automatically rolled back and re-upgraded. If the rollback and re-upgrade are successful, the upgrade of the failed node is completed;

[0122] When the rollback and re-upgrade fails, the upgrade can be manually restored, and if necessary, the node can be manually re-upgraded multiple times to complete the upgrade of the failed node; or, when the rollback and re-upgrade fails, the failed node can be manually restored and upgraded and automatically filled to complete the upgrade of the failed node. In one embodiment, when the rollback and re-upgrade fails, the node upgrade is completed by manually restoring the upgrade once or multiple times, and still fails.

[0123] In one embodiment, when an upgrade failure occurs in the node to be upgraded, the nodes in the storage pool wait for the upgrade of the failed node to be completed. The storage pools are upgraded normally without waiting for the failed node.

[0124] By supporting the upgrade pause and resume functions, there is no intermediate state for nodes to upgrade. Each node is in the upgrade completion state, ensuring the consistency of the node's storage version and solving the coexistence conflict problem of versions before and after the upgrade.

[0125] Through dynamic adjustment and fault-tolerant mechanisms, the reliability and stability of the upgrade process are ensured, and the overall reliability of the system is improved.

[0126] As an implementation mode of the present application, before the storage system is upgraded online, the following steps are further included:

[0127] Backing up system data in the storage system;

[0128] Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes with normal node status;

[0129] Check the working status of the hardware device in the storage system to determine that the working status belongs to a preset upgrade status.

[0130] In one embodiment, the pre-upgrade inspection includes: checking the cluster status, network and hardware equipment, specifically including: checking the service status of the cluster node. The node will be upgraded only if the service status is normal. If it is in a high-pressure state, a fault state or a warning state, it cannot be upgraded. These abnormal nodes are eliminated and normal nodes are upgraded. Checking the hardware equipment includes: the network and hardware equipment, checking whether the network port status is normal, and ensuring that the normal network does not affect the upgrade efficiency. Checking whether the system disk and network card status are normal to prevent abnormal situations from affecting the upgrade. The smooth progress of the upgrade is guaranteed by the pre-upgrade inspection. Among them, the preset upgrade status includes: the network port status is normal, the system disk is normal, and the network card status is normal.

[0131] Through backup, data security is guaranteed for the storage system; through node status inspection, faulty nodes are eliminated to ensure the normal progress of the upgrade process. The system hardware equipment is inspected to ensure that the network is normal during the upgrade process and does not affect the upgrade speed.

[0132] As an implementation manner of the present application, the creating of the mirror source based on the mirror file package also includes:

[0133] Receive an image file package and a signature file corresponding to the image file package; in one embodiment, the signature file is a technical means to ensure the integrity, authenticity and traceability of the file by embedding or appending a digital signature in the file.

[0134] The image file package is verified according to the signature file.

[0135] As an implementation mode of the present application, the upgrade content of the node to be upgraded may include:

[0136] At least one of the kernel, driver, and storage version of the node to be upgraded is upgraded according to the image source.

[0137] For example, the kernel, kernel, driver and storage version of the node to be upgraded are upgraded according to the mirror source. The kernel of the node to be upgraded is upgraded first according to the mirror source. After the kernel upgrade is completed, the driver and storage version are upgraded in sequence based on the kernel upgrade result. Through the line concurrent upgrade technology based on the upgrade queue, the kernel, driver and storage version of the storage system are quickly and safely upgraded without interrupting the business operation. The storage version is upgraded while the kernel of the mirror system is upgraded, which improves the upgrade function and efficiency.

[0138] As an implementation mode of the present application, after the upgrade is completed, all nodes of the upgraded storage system are verified to determine whether the kernel version, driver version and storage version of all nodes meet the version conditions for successful upgrade; and the services of all nodes of the upgraded storage system are verified to determine whether the services of all nodes meet the service conditions for successful upgrade;

[0139] When the kernel versions, driver versions and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful. In one embodiment, the storage version refers to the storage version of the storage system.

[0140] As an implementation method of this application, Fig. 9 As shown in the figure, the whole upgrade process includes:

[0141] Preparation before upgrading: including backing up the current system data, checking the cluster status, uploading and decompressing the new version installation package, etc.

[0142] Concurrent upgrade execution: According to the generated upgrade queue, the upgrade tasks of multiple nodes are started at the same time. During the upgrade process, the upgrade controller monitors the upgrade status of each node in real time and makes dynamic adjustments as needed. In one embodiment, the upgrade controller monitors the upgrade status of each node in real time. When an upgrade failure occurs in the node to be upgraded, it automatically rolls back and re-upgrades. If the rollback and re-upgrade are successful, the upgrade of the faulty node is completed; when the rollback and re-upgrade fails, the upgrade is manually restored. If necessary, the node can be manually re-upgraded multiple times to complete the upgrade of the faulty node; or, when the rollback and re-upgrade fails, the faulty node is continued to be upgraded through manual recovery and automatic filling to complete the upgrade of the faulty node. During the normal upgrade process, the upgrade order of data nodes, metadata nodes and storage monitoring nodes is controlled by the upgrade controller according to the upgrade queue. Control the data nodes to be upgraded first, and the storage monitoring nodes to be upgraded last.

[0143] Upgrade the main process, such as Fig.10 As shown: the main upgrade process of the upgraded node is as follows: remove the drivers that conflict with the kernel upgrade, upgrade the kernel, reinstall the drivers removed in the previous step, install the driver under the new kernel, upgrade the storage version, restart the node, verify and notify the management software interface of the soft upgrade progress.

[0144] Post-upgrade verification: After the upgrade is completed, the cluster status is verified to ensure that the upgrade is successful and data consistency is not affected. In one embodiment, it is verified whether all node driver versions, kernel versions, and storage versions are consistent, and whether the node service is normal.

[0145] After the upgrade, the upgrade process report is supported, with clear content, start time, and end time of each step. It also includes printing of upgrade exceptions, alarms and other information, and supports report download.

[0146] Improve upgrade efficiency by upgrading the kernel and storage version online through the distributed storage system: Through the online concurrent upgrade algorithm, fast and safe upgrades are achieved without interrupting business operations. The image system kernel is upgraded while the storage version is upgraded, which improves the upgrade function and efficiency. Reduce operation and maintenance costs: Reduce business interruption time caused by upgrades, reduce losses and operation and maintenance costs caused by business interruptions. Enhance system reliability: Through dynamic adjustment and fault-tolerant mechanisms, the reliability and stability of the upgrade process are ensured, and the overall reliability of the system is improved.

[0147] The online concurrent algorithm includes: (1) Node classification and upgrade queue generation: First, according to the underlying data distribution structure of the cluster, the storage nodes are classified to identify different types of nodes (such as data nodes, metadata nodes, etc.). Then, according to the actual situation of the current cluster, an optimized upgrade queue is generated to determine the upgrade order of each node.

[0148] (2) Concurrent upgrade execution: During the upgrade process, concurrent upgrade technology is used to start the upgrade tasks of multiple nodes at the same time. The upgrade controller monitors the upgrade status of each node in real time to ensure that the cluster business is not affected during the upgrade process.

[0149] (3) Dynamic adjustment and fault-tolerance mechanism: The upgrade controller dynamically adjusts the upgrade order of each node according to the cluster service status. If an upgrade fails, it can automatically roll back and retry the upgrade to ensure the reliability and stability of the upgrade process.

[0150] The online concurrent upgrade algorithm enables fast and safe upgrades without interrupting business operations. The method includes steps such as node classification, upgrade queue generation, concurrent upgrade execution, dynamic adjustment and fault tolerance mechanism, and supports kernel and storage version upgrades at the same time, which has the beneficial effects of improving upgrade efficiency, reducing operation and maintenance costs, and enhancing system reliability.

[0151] Based on the same inventive concept, an embodiment of the present application provides a distributed storage system. Fig.11 , Fig.11 1 is a block diagram of a distributed storage system 1100 proposed in one embodiment of the present application. Fig.11 As shown, the system includes: the distributed storage system includes at least one storage pool, and each storage pool includes one or more nodes;

[0152] A first selection module 1101 is used to determine at least one node to be upgraded in the storage system;

[0153] The second selection module 1102 is used to select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package;

[0154] A creation module 1103 is used to determine at least one node cluster according to the node category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster according to the upgrade priority of the node cluster. The upgrade priority of the nodes to be upgraded in the same node cluster is the same, and the upgrade priority of the nodes to be upgraded in different node clusters is different;

[0155] The upgrade module 1104 is used to upgrade the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; after the upgrade of all nodes to be upgraded is completed, an upgraded storage system is obtained.

[0156] As an implementation mode of the present application, the creation module 1103 is specifically used to classify at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster and a storage monitoring node cluster;

[0157] Create a first-order upgrade queue between classes based on the data node cluster, metadata node cluster, and storage monitoring node cluster;

[0158] Creating a first concurrent upgrade queue for nodes in the data node cluster according to a value of a storage pool in the storage system and a value of a fault domain in the storage pool;

[0159] Creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain;

[0160] Creating a third concurrent upgrade queue of nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain;

[0161] The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue and the third concurrent upgrade queue are determined as the upgrade sequence of the node clusters.

[0162] As an implementation mode of the present application, the creation module 1103 is specifically used to obtain the maximum upgrade quantity in the storage pool according to the value of the storage pool;

[0163] Obtaining the concurrent number of a single upgrade in the storage pool according to the value of the fault domain;

[0164] A second concurrent upgrade queue for nodes in the metadata node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0165] As an implementation of the present application, the upgrade module 1104 is specifically used to:

[0166] Upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source;

[0167] Simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source;

[0168] Simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source;

[0169] The nodes in the storage monitoring node cluster in the third concurrent upgrade queue are upgraded simultaneously according to the mirror source.

[0170] As an implementation mode of the present application, it further includes: a fault processing module, which is used to re-upgrade the node to be upgraded through the first fault processing mode when an upgrade failure occurs in the node to be upgraded, and when the upgrade in the first fault processing mode is successful, the upgrade of the faulty node is completed;

[0171] When the first fault handling mode upgrade fails, the second fault handling mode is used to upgrade the faulty node; or, when the first fault handling mode upgrade fails, the second fault handling mode and the third fault handling mode are used to continue upgrading the faulty node to complete the faulty node upgrade.

[0172] As an implementation mode of the present application, it further includes: an upgrade check module, which is used to back up the system data in the storage system;

[0173] Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes with normal node status;

[0174] Check the working status of the hardware device in the storage system to determine that the working status belongs to a preset upgrade status.

[0175] As an implementation mode of the present application, it also includes: a verification module, which is used to receive an image file package and a signature file corresponding to the image file package;

[0176] The image file package is verified according to the signature file.

[0177] As an implementation manner of the present application, the upgrade module 1104 is specifically configured to upgrade at least one of the kernel, driver, and storage version of the node to be upgraded according to the image source.

[0178] As an implementation mode of the present application, it also includes: a verification module, which is used to verify all nodes of the upgraded storage system after the upgrade is completed, and determine whether the kernel version, driver version and storage version of all nodes meet the version conditions for successful upgrade; and verify the services of all nodes of the upgraded storage system to determine whether the services of all nodes meet the service conditions for successful upgrade;

[0179] When the kernel versions, driver versions, and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful.

[0180] The distributed storage system provided in the above embodiment can implement the technical solution described in the above embodiment of the method for online upgrading of a storage system. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the method for online upgrading of a storage system, which will not be repeated here.

[0181] Based on the same inventive concept, the present application also provides an electronic device 1200. Fig.12 As shown, the electronic device 1200 includes a processor 1201 , a memory 1202 and a display 1203 . Fig.12 Only some components of the electronic device 1200 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0182] In some embodiments, the memory 1202 may be an internal storage unit of the electronic device 1200, such as a hard disk or memory of the electronic device 1200. In other embodiments, the memory 1202 may also be an external storage device of the electronic device 1200, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 1200.

[0183] Furthermore, the memory 1202 may include both an internal storage unit of the electronic device 1200 and an external storage device. The memory 1202 is used to store application software installed in the electronic device 1200 and various data.

[0184] In some embodiments, the processor 1201 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 1202, such as a method for online upgrading of a storage system in the present application.

[0185] In some embodiments, the display 1203 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 1203 is used to display information on the electronic device 1200 and to display a visual user interface. The components 1201-1203 of the electronic device 1200 communicate with each other via a system bus.

[0186] In some embodiments of the present application, when the processor 1201 executes the storage system online upgrade program in the memory 1202, the steps of the storage system online upgrade method may be implemented:

[0187] It should be understood that: when the processor 1201 executes the storage system online upgrade program in the memory 1202, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0188] Furthermore, the embodiment of the present application does not specifically limit the type of the electronic device 1200 mentioned, and the electronic device 1200 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer (1aptop) with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present application, the electronic device 1200 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0189] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the online upgrade method of the storage system provided by the above methods.

[0190] Based on the same inventive concept, another embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the method for online upgrading of a storage system as described in any of the above embodiments of the present invention. Those skilled in the art can understand that all or part of the process of implementing the above embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0191] The above is a detailed introduction to a method, system, device and medium for online upgrading of a storage system provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for online upgrading of a storage system, characterized in that: The storage system includes at least one storage pool, each storage pool includes one or more nodes, and the method includes: Determining at least one node to be upgraded in the storage system; Select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package; Determine at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster according to the upgrade priority of the node cluster. The upgrade priorities of the nodes to be upgraded in the same node cluster are the same, and the upgrade priorities of the nodes to be upgraded in different node clusters are different. According to the mirror source, the nodes to be upgraded of each node cluster are upgraded in the upgrade order of each node cluster. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; After all nodes to be upgraded are upgraded, the upgraded storage system is obtained.

2. A method for online upgrading of a storage system according to claim 1, characterized in that: The step of determining at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determining the upgrade order of each node cluster according to the upgrade priority of the node cluster specifically includes: Classifying at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster and a storage monitoring node cluster; Create a first-order upgrade queue between classes based on the upgrade priorities of data node clusters, metadata node clusters, storage monitoring node clusters, and node clusters; Creating a first concurrent upgrade queue for nodes in the data node cluster according to a value of a storage pool in the storage system and a value of a fault domain in the storage pool; Creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain; Creating a third concurrent upgrade queue of nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain; The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue and the third concurrent upgrade queue are determined as the upgrade sequence of the node clusters.

3. A method for online upgrading of a storage system according to claim 2, characterized in that: The step of creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain specifically includes: Obtaining a maximum upgrade quantity in the storage pool according to the value of the storage pool; Obtaining the concurrent number of a single upgrade in the storage pool according to the value of the fault domain; A second concurrent upgrade queue for nodes in the metadata node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

4. A method for online upgrading of a storage system according to claim 2 or 3, characterized in that: The step of upgrading the nodes to be upgraded of each node cluster according to the image source and in the upgrade order of each node cluster specifically includes: Upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source; Simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source; Simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source; The nodes in the storage monitoring node cluster in the third concurrent upgrade queue are upgraded simultaneously according to the mirror source.

5. A method for online upgrading of a storage system according to claim 4, characterized in that: Also includes: When an upgrade failure occurs in the node to be upgraded, the node is upgraded again through the first fault handling mode. When the upgrade in the first fault handling mode succeeds, the upgrade of the faulty node is completed; When the upgrade in the first fault handling mode fails, the upgrade is performed in the second fault handling mode to complete the upgrade of the faulty node; Or, when the upgrade in the first fault processing mode fails, the faulty node is continuously upgraded through the second fault processing mode and the third fault processing mode to complete the upgrade of the faulty node.

6. A method for online upgrading of a storage system according to claim 1, characterized in that: Before the storage system is upgraded online, the following steps are also required: Backing up system data in the storage system; Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes with normal node status; Check the working status of the hardware device in the storage system to determine that the working status belongs to a preset upgrade status.

7. A method for online upgrading of a storage system according to claim 1 or 6, characterized in that: Before creating the image source based on the image file package, the method further includes: Receive an image file package and a signature file corresponding to the image file package; The image file package is verified according to the signature file.

8. A method for online upgrading of a storage system according to claim 1 or 6, characterized in that: The step of upgrading the node to be upgraded specifically includes: At least one of the kernel, driver, and storage version of the node to be upgraded is upgraded according to the image source.

9. A method for online upgrading of a storage system according to claim 1 or 6, characterized in that: Also includes: After the upgrade is completed, all nodes of the upgraded storage system are verified to determine whether the kernel version, driver version, and storage version of all nodes meet the version conditions for a successful upgrade; and the services of all nodes of the upgraded storage system are verified to determine whether the services of all nodes meet the service conditions for a successful upgrade; When the kernel versions, driver versions, and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful.

10. A distributed storage system, characterized in that: The distributed storage system includes at least one storage pool, each storage pool includes one or more nodes, including: A first selection module, configured to determine at least one node to be upgraded in the storage system; A second selection module is used to select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package; A creation module is used to determine at least one node cluster according to the node category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster according to the upgrade priority of the node cluster. The upgrade priority of the nodes to be upgraded in the same node cluster is the same, and the upgrade priority of the nodes to be upgraded in different node clusters is different; The upgrade module is used to upgrade the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; after the upgrade of all nodes to be upgraded is completed, an upgraded storage system is obtained.

11. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the method for online upgrading of a storage system as described in any one of claims 1 to 9 above.

12. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the method for online upgrading of a storage system as described in any one of claims 1 to 9 above.

Images