A method, system, device and medium for online upgrade of a storage system

Abstract

The application relates to a method, system, device and medium for online upgrading of a storage system, and belongs to the technical field of distributed storage. The method comprises the following steps: determining at least one node to be upgraded in the storage system; selecting any node to be upgraded as a control node to receive an image file package, and creating an image source based on the image file package; determining at least one node cluster according to the category of the node cluster to which the node to be upgraded belongs, determining the upgrading sequence of each node cluster according to the upgrading priority of the node cluster, and simultaneously upgrading multiple nodes to be upgraded belonging to the same node cluster and the same storage pool. The upgrading method can realize fast and reliable upgrading iteration without affecting the operation of the business.

Description

Technical Field

[0001] The present application relates to the field of distributed storage, and more specifically, to a method, system, device, and medium for online upgrading of a storage system. Background Art

[0002] After a distributed storage cluster is launched, its storage system needs to be regularly upgraded to fix vulnerabilities, enhance performance, or experience new version features.

[0003] Traditional distributed storage systems don't support mirroring system kernel versions simultaneously. Upgrades typically rely on offline or serial online upgrades. These methods require business interruption during the upgrade process and only support partial upgrades within the distributed storage system, resulting in low upgrade efficiency and poor results. For industries like finance and communications, which place high demands on business continuity and upgrade timeliness, these methods clearly cannot meet market demand. Summary of the Invention

[0004] To solve the above problems, the present application provides a method, system, device and medium for online upgrading of a storage system.

[0005] A first aspect of an embodiment of the present application provides a method for online upgrading a storage system, wherein the storage system includes at least one storage pool, each storage pool includes one or more nodes, including:

[0006] Determining at least one node to be upgraded in the storage system;

[0007] Select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package;

[0008] Determine at least one node cluster based on the node cluster category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster based on the upgrade priority of the node cluster. The upgrade priorities of nodes to be upgraded in the same node cluster are the same, while the upgrade priorities of nodes to be upgraded in different node clusters are different.

[0009] According to the mirror source, the nodes to be upgraded of each node cluster are upgraded in the upgrade order of each node cluster. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously.

[0010] After all nodes to be upgraded are upgraded, the upgraded storage system is obtained.

[0011] Optionally, determining at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determining the upgrade order of each node cluster according to the upgrade priority of the node cluster specifically includes:

[0012] Classifying at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster, and a storage monitoring node cluster;

[0013] Create a first-order upgrade queue between classes based on the data node cluster, metadata node cluster, and storage monitoring node cluster;

[0014] Creating a first concurrent upgrade queue for nodes in the data node cluster according to a value of a storage pool in the storage system and a value of a fault domain in the storage pool;

[0015] Creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain;

[0016] Creating a third concurrent upgrade queue for nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain;

[0017] The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue, and the third concurrent upgrade queue are determined as the upgrade sequence of the node clusters.

[0018] Optionally, creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain specifically includes: obtaining a maximum upgrade quantity in the storage pool according to the value of the storage pool;

[0019] Obtaining the concurrent number of single upgrades in the storage pool according to the value of the fault domain;

[0020] A second concurrent upgrade queue for nodes in the metadata node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0021] Optionally, upgrading the nodes to be upgraded in each node cluster according to the image source and in the upgrade order of each node cluster specifically includes:

[0022] Upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source;

[0023] Simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source;

[0024] Simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source;

[0025] The nodes in the storage monitoring node cluster in the third concurrent upgrade queue are simultaneously upgraded according to the mirror source.

[0026] Optionally, it also includes:

[0027] When an upgrade failure occurs in the node to be upgraded, the upgrade is performed again using the first fault handling mode. When the upgrade is successful in the first fault handling mode, the upgrade of the faulty node is completed.

[0028] When the first fault handling mode upgrade fails, the second fault handling mode is used to upgrade the faulty node, or when the first fault handling mode upgrade fails, the second and third fault handling modes are used to continue upgrading the faulty node, so that the faulty node upgrade is completed.

[0029] Optionally, before the storage system is upgraded online, the following steps may also be performed:

[0030] Backing up system data in the storage system;

[0031] Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes in normal node status;

[0032] Checking the working status of the hardware devices in the storage system to determine whether the working status belongs to a preset upgrade status.

[0033] Optionally, before creating the mirror source based on the mirror file package, the method further includes:

[0034] Receive an image file package and a signature file corresponding to the image file package;

[0035] The image file package is verified according to the signature file.

[0036] Optionally, upgrading the node to be upgraded specifically includes:

[0037] At least one of the kernel, driver, and storage version of the node to be upgraded is upgraded according to the image source.

[0038] Optionally, it also includes:

[0039] After the upgrade is complete, verify all nodes of the upgraded storage system to determine whether the kernel version, driver version, and storage version of all nodes meet the version conditions for a successful upgrade; and verify the services of all nodes of the upgraded storage system to determine whether the services of all nodes meet the service conditions for a successful upgrade;

[0040] When the kernel versions, driver versions, and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful.

[0041] A second aspect of an embodiment of the present application provides a distributed storage system, the distributed storage system including at least one storage pool, each storage pool including one or more nodes, including:

[0042] A first selection module is configured to determine at least one node to be upgraded in the storage system;

[0043] A second selection module is used to select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package;

[0044] A creation module is used to determine at least one node cluster based on the node category to which the node to be upgraded belongs, and to determine the upgrade order of each node cluster based on the upgrade priority of the node cluster. The upgrade priority of nodes to be upgraded in the same node cluster is the same, and the upgrade priority of nodes to be upgraded in different node clusters is different.

[0045] The upgrade module is used to upgrade the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; after the upgrade of all nodes to be upgraded is completed, the upgraded storage system is obtained.

[0046] A third aspect of an embodiment of the present application provides an electronic device, including a memory and a processor, wherein:

[0047] The memory is used to store programs;

[0048] The processor is coupled to the memory and is configured to execute the program stored in the memory to implement the steps of a storage system online upgrade method in any of the above solutions.

[0049] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided for storing computer-readable programs or instructions, which, when executed by a processor, can implement the steps of a storage system online upgrade method of any of the above-mentioned schemes.

[0050] The fifth aspect is a computer program product. The fifth aspect of the embodiment of the present invention provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the method for online upgrading of the storage system as described in the first aspect of the embodiment of the present invention are implemented.

[0051] By applying the technical solution provided in the embodiments of the present application, during the storage system upgrade process, the upgrade order of each node to be upgraded is determined through the upgrade order of each node cluster, and the upgrade tasks of multiple nodes in the same storage pool are started simultaneously according to the upgrade order, thereby realizing concurrent upgrades and effectively improving the upgrade efficiency. Based on the upgrade order of each node cluster, it is always ensured that there are nodes in the system that provide services for business services, so that the storage system upgrade method of the present application can achieve fast and reliable upgrade iterations without affecting business operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 This is a flowchart of the steps of a method for online upgrading of a storage system provided in an embodiment of the present application;

[0054] Figure 2 This is a flowchart of the steps for creating an upgrade sequence provided in an embodiment of the present application;

[0055] Figure 3 This is a diagram of the upgrade sequence structure provided in the embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of a metadata node provided in an embodiment of the present application;

[0057] Figure 5 It is a flowchart of the steps of the method for creating a second concurrent upgrade queue of nodes in a metadata node cluster provided in an embodiment of the present application;

[0058] Figure 6 This is a flowchart of the steps of the process of upgrading the node to be upgraded provided in the embodiment of the present application;

[0059] Figure 7 This is a data node structure diagram provided in an embodiment of the present application;

[0060] Figure 8 This is a schematic diagram of a storage monitoring node provided in an embodiment of the present application;

[0061] Figure 9 It is a schematic diagram of the entire upgrade process provided in the embodiment of the present application;

[0062] Figure 10 This is a schematic diagram of the main upgrade process provided in the embodiment of the present application;

[0063] Figure 11 This is a structural block diagram of a distributed storage system provided in an embodiment of the present application;

[0064] Figure 12 This is a hardware structure block diagram of an electronic device provided in each embodiment of the present application. DETAILED DESCRIPTION

[0065] To make the above-mentioned purposes, features, and advantages of this application more clearly understood, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of this application.

[0066] The present application relates to the field of distributed storage technology and is a method for online upgrading the kernel and storage versions of a distributed storage system. The method aims to enable the storage system to quickly and safely complete the upgrade of the kernel and storage versions without interrupting business operations, and supports upgrade pause and resume functions to improve system availability and maintenance efficiency.

[0067] The kernel is the core of an operating system. It is the first layer of software that extends the hardware and provides the most basic operating system functionality. It is responsible for managing the system's processes, memory, device drivers, files, and network systems, and determines system performance and stability. Device drivers are software within the operating system that controls and manages hardware devices. They allow the operating system to communicate with hardware devices and perform necessary operations.

[0068] In modern operating system design, to reduce system overhead, components closely related to hardware, such as interrupt handlers and device drivers, as well as basic, common, and frequently used modules (such as clock management and process scheduling) and other data structures, are often separated and placed in memory to maintain permanent memory and protected. This component is often referred to as the operating system kernel.

[0069] Reference Figure 1 As shown, a flowchart of a method for online upgrading of a storage system is shown. The method can be applied to a distributed storage system, such as Figure 1 As shown, the following steps may be specifically included:

[0070] Step S101: determining at least one node to be upgraded in the storage system; in one embodiment, the storage system may be a distributed storage system.

[0071] In step S102, any node to be upgraded is selected as the control node to receive the image file package, and a mirror source is created based on the image file package. For example, the image file package can be an ISO (International Organization for Standardization) image file package. ISO is one of the storage formats for CD-ROM images on computers. Because it is stored according to the ISO*9660 standard for CD-ROM file systems, it is usually named with the suffix .iso on computers. It is a single file that can truly reflect the contents of the CD and can be created by burning software or an image file creation tool. The mirror source refers to the server address that provides software package download services. The server address is created based on the ISO image package and contains the kernel, driver, and storage version of the storage system required for the upgrade.

[0072] Step S103: Determine at least one node cluster based on the node cluster category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster based on the upgrade priority of the node cluster. Nodes to be upgraded in the same node cluster have the same upgrade priority, while nodes to be upgraded in different node clusters have different upgrade priorities.

[0073] Step S104: upgrading the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously.

[0074] In one embodiment, the kernel of the node to be upgraded is first upgraded according to the image source, and after the kernel upgrade is completed, the driver and storage versions are upgraded in sequence.

[0075] Step S105: After all nodes to be upgraded are upgraded, an upgraded storage system is obtained.

[0076] In this embodiment, before the upgrade, the cluster status, network status and hardware equipment of the nodes of the storage system are checked to ensure that the upgrade proceeds normally and to eliminate nodes that do not meet the upgrade conditions. At least one node to be upgraded is determined in the storage system. Usually, there are multiple storage nodes that need to be upgraded.

[0077] The storage system selects a central control node from among the multiple nodes to be upgraded. This node receives the image file package and signature file uploaded via the interface. The signature file is used to verify the image file package's integrity and correctness. An image source is created from the image file package and stored locally on the control node. Other nodes to be upgraded download the image source from the control node during the upgrade process.

[0078] According to the mirror source, the to-be-upgraded nodes of each node cluster are upgraded in an upgrading order of each node cluster, and in the upgrading process, multiple to-be-upgraded nodes belonging to the same node cluster and the same storage pool are upgraded simultaneously; in the upgrading process, a non-upgraded state or an upgraded node that has completed upgrading processes a service request; after the upgrading of all to-be-upgraded nodes is completed, an upgraded storage system is obtained.

[0079] In the storage system upgrading process, the upgrading technology of the upgrading queue is used to determine the upgrading order of each to-be-upgraded node, and the upgrading tasks of multiple nodes in the same storage pool are started simultaneously according to the upgrading order, so that concurrent upgrading is realized to effectively improve the upgrading efficiency. Based on the execution order of the upgrading queue, a node providing service is always present in the system for business service, so that the storage system upgrading method based on the upgrading queue of the present application can realize fast and reliable upgrading iteration without affecting the operation of the business.

[0080] As an embodiment of the present application, as shown in Figure 2 The step flow chart of creating the upgrading queue is shown, and specifically can include:

[0081] Step S201, at least one to-be-upgraded node is classified to obtain at least one node cluster; wherein the at least one node cluster includes a data node cluster, a metadata node cluster, and a storage monitoring node cluster;

[0082] Step S202, a first order upgrading queue between classes is created according to the data node cluster, the metadata node cluster, and the storage monitoring node cluster;

[0083] Step S203, a first concurrent upgrading queue of nodes in the data node cluster is created according to the value of a storage pool in the storage system and the value of a failure domain in the storage pool;

[0084] Step S204, a second concurrent upgrading queue of nodes in the metadata node cluster is created according to the value of the storage pool and the value of the failure domain;

[0085] Step S205, a third concurrent upgrading queue of nodes in the storage monitoring node cluster is created according to the value of the storage pool and the value of the failure domain;

[0086] Step S206, the first order upgrading queue, the first concurrent upgrading queue, the second concurrent upgrading queue, and the third concurrent upgrading queue are determined as the upgrading order of each node cluster.

[0087] In the present embodiment, as Figure 3As shown, a structural diagram of the upgrade queue is shown. According to the underlying data distribution structure of the node cluster, the storage nodes are classified, and different types of nodes are identified, such as: data nodes, metadata nodes and storage monitoring nodes, to obtain at least one node cluster; wherein, the at least one node cluster includes: data node cluster, metadata node cluster and storage monitoring node cluster.

[0088] Create a first-order upgrade queue between classes based on the data node cluster, metadata node cluster, and storage monitoring node cluster; the upgrade priorities of the data node cluster, metadata node cluster, and storage monitoring node cluster decrease in sequence, and the upgrade order is: data node cluster, metadata node cluster, and storage monitoring node cluster.

[0089] Based on the value of the storage pool, determine how many nodes can be upgraded simultaneously in the storage pool. Combined with the value of the storage pool's fault domain, it can be determined how many nodes can be upgraded simultaneously while ensuring that business services are not affected and the storage system load is normal. In an example, Figure 4 The figure shows the structure of the metadata node. When the storage pool value is 4 and the fault domain value is 3, nodes 1, 2, and 3 can be upgraded simultaneously. When the storage pool value is 4 and the fault domain value is 2, any two of nodes 1, 2, and 3 can be upgraded simultaneously, and at least one of them can be upgraded before the third. If the number of concurrent upgrades exceeds the fault domain value, service will be affected. The single concurrent upgrade number is less than or equal to the fault domain value. The storage pool value ≥ the fault domain value ≥ the single concurrent upgrade number.

[0090] Based on the values ​​of the storage pool and the fault domain, and in combination with the above method, a second concurrent upgrade queue for nodes in the metadata node cluster is created;

[0091] The methods for creating the first concurrent upgrade queue, the second concurrent upgrade queue, and the third concurrent upgrade queue are the same and will not be described in detail.

[0092] The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue, and the third concurrent upgrade queue are determined as the upgrade queues.

[0093] The first-order upgrade queue created by node classification, storage pool value and fault domain value in the storage pool ensures that there are always metadata nodes to provide services during the upgrade process, and realizes fast and safe upgrade of the storage system without interrupting business operations. Through multiple concurrent upgrade queues, the upgrade process of each storage node is coordinated, and the storage business is controlled not to exceed the fault domain, without affecting the normal use of the storage business, thus realizing concurrent upgrades and effectively improving the speed of storage system upgrades.

[0094] As an embodiment of the present application, Figure 5FIG. 1 shows a flowchart of steps for creating a second concurrent upgrade queue for nodes in a metadata node cluster, which may specifically include:

[0095] Step S501, obtaining the maximum upgrade quantity in the storage pool according to the value of the storage pool;

[0096] Step S502: obtaining the concurrent number of single upgrades in the storage pool according to the value of the fault domain;

[0097] Step S503: Create a second concurrent upgrade queue for nodes in the metadata node cluster according to the maximum upgrade quantity in the storage pool and the concurrent quantity of single upgrades in the storage pool.

[0098] For example, the concurrent number of single upgrades in the storage pool is obtained according to the value of the fault domain;

[0099] A first concurrent upgrade queue for nodes in the data node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0100] As another example, the concurrent number of single upgrades in the storage pool is obtained according to the value of the fault domain;

[0101] A third concurrent upgrade queue for nodes in the storage monitoring node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of single upgrades in the storage pool.

[0102] In this embodiment, based on the value of the storage pool and the value of the fault domain, based on the value of the storage pool, that is, how many nodes can be upgraded simultaneously in the storage pool at one time; combined with the value of the fault domain of the storage pool, that is, how many nodes can be upgraded simultaneously at one time while ensuring that business services are not affected and the load of the storage system is normal, a second concurrent upgrade queue for the nodes in the metadata node cluster is created according to the maximum number of upgrades in the storage pool and the number of concurrent single upgrades in the storage pool. In one example, Figure 4 As shown in the figure, when the storage pool value is 4 and the fault domain value is 3, it means that Node 1, Node 2, and Node 3 can be upgraded at the same time. When the storage pool value is 4 and the fault domain value is 2, it means that two nodes among Node 1, Node 2, and Node 3 can be upgraded at the same time, and the third node can be upgraded after at least one of them is upgraded.

[0103] As an implementation manner of the present application, after creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain, the method further includes: the value of the storage pool is greater than or equal to the value of the fault domain;

[0104] Determine whether the value of the fault domain is greater than the number of nodes in the metadata node cluster, and if not, upgrade the nodes in the metadata node cluster according to the second concurrent upgrade queue;

[0105] If so, the second concurrent upgrade queue is divided into a first sub-concurrent upgrade queue and a second sub-concurrent upgrade queue, and the nodes in the metadata node cluster are upgraded in the order of the first sub-concurrent upgrade queue and the second sub-concurrent upgrade queue.

[0106] In this embodiment, when the value of the fault domain is less than or equal to the number of nodes in the metadata node cluster, it means that in the storage pool, the current load capacity of the current storage system can simultaneously upgrade some nodes in the metadata node cluster, and there are still idle nodes (nodes in non-upgraded state) to provide business services.

[0107] When the value of the fault domain is greater than the number of nodes in the metadata node cluster, it means that in the storage pool, the current load capacity of the current storage system can upgrade all the nodes in the metadata node cluster at the same time, and there are no idle nodes to provide services. Therefore, through more refined regulation, the nodes in the metadata node cluster are further updated in batches, and the nodes in the metadata node cluster are upgraded in the order of the first sub-concurrent upgrade queue and the second sub-concurrent upgrade queue, which ensures the upgrade efficiency of the metadata nodes and further ensures the integrity of the business services.

[0108] As an embodiment of the present application, Figure 6 FIG. 1 is a flowchart showing the steps of upgrading a node to be upgraded, which may include:

[0109] Step S601, upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source;

[0110] Step S602: simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source;

[0111] Step S603: simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source;

[0112] Step S604: simultaneously upgrade the nodes in the storage monitoring node cluster in the third concurrent upgrade queue according to the mirror source.

[0113] In this embodiment, if Figure 3 As shown, the first order upgrade queue is the data node cluster, metadata node cluster and storage monitoring node cluster; according to the upgrade order of data nodes, metadata nodes and storage monitoring nodes, different types of nodes are upgraded according to the mirror source.

[0114] The first concurrent upgrade queue is represented by: Figure 7 As shown, the structure of the data node is shown, wherein the multiple data nodes include: node 1 node 2, node 3 and node 4. The upgrade order of node 1 node 2, node 3 and node 4 represents the first concurrent upgrade queue. Node 1, node 2, node 3 and node 4 in the data nodes are selected to upgrade nodes 1, node 2, node 3 and node 4 simultaneously according to the fault domain value of the storage pool and storage system load capacity. In the case of insufficient load capacity, the number of single concurrent upgrades is reduced by combining simultaneous upgrades with sequential upgrades. For example, if the fault domain value is 2 and the number of single concurrent upgrade nodes is 2, the upgrade of nodes 1, node 2, node 3 and node 4 needs to be completed through two concurrent upgrades.

[0115] The second concurrent upgrade queue is represented by: Figure 4 As shown, the structure of the metadata node is shown, wherein multiple metadata nodes include: node 1 node 2 and node 3, and the upgrade order of node 1, node 2 and node 3 represents the second concurrent upgrade queue. Node 1, node 2 and node 3 in the metadata node select to upgrade node 1, node 2 and node 3 simultaneously according to the fault domain value of the storage pool and storage system load capacity.

[0116] The third concurrent upgrade queue is represented by: Figure 8 Figure 1 shows the structure of a storage monitoring node, where multiple storage monitoring nodes include: Node 1, Node 2, and Node 3. The upgrade order of Node 1, Node 2, and Node 3 represents the third concurrent upgrade queue. Node 1, Node 2, and Node 3 in the storage monitoring node are upgraded simultaneously based on the failure domain values ​​of the storage pool and storage system load capacity.

[0117] In one embodiment, the upgrade queue generator formulates an upgrade strategy based on factors such as cluster load, i.e., current business load, controlling the number of single upgrades in the storage pool, i.e., fault domain, and node cluster type.

[0118] Using concurrent upgrade technology, multiple concurrent upgrade queues are used to simultaneously start upgrade tasks for multiple nodes. The first-order upgrade queue ensures that the system always has nodes providing business services, ensuring that cluster services are not affected during the storage system upgrade process.

[0119] As an implementation method of the present application, when an upgrade failure occurs in the node to be upgraded, the upgrade is performed again using the first fault handling mode. When the upgrade is successful in the first fault handling mode, the upgrade of the faulty node is completed. In one embodiment, the first fault handling mode can be a rollback and re-upgrade.

[0120] When the first failure handling mode upgrade fails, the second failure handling mode is used to upgrade, so that the failure node is upgraded; in an embodiment, the second failure handling mode can be manual recovery upgrade; or, when the first failure handling mode upgrade fails, the second failure handling mode and the third failure handling mode are used to continue upgrading the failure node, so that the failure node is upgraded. In an embodiment, the third failure handling mode can be automatic filling upgrade.

[0121] In the present embodiment, when the node to be upgraded fails to upgrade, automatic rollback re-upgrade is performed, and when the rollback re-upgrade succeeds, the failure node is upgraded.

[0122] When the rollback re-upgrade fails, manual recovery upgrade is performed, and the node can be re-upgraded multiple times manually as necessary, so that the failure node is upgraded; or, when the rollback re-upgrade fails, manual recovery upgrade and automatic filling are used to continue upgrading the failure node, so that the failure node is upgraded. In an embodiment, when the rollback re-upgrade fails, single or multiple manual recovery upgrades are performed, and when they still fail, automatic filling is used to ensure that the node is upgraded.

[0123] In an embodiment, when the node to be upgraded fails to upgrade, the nodes in the storage pool wait for the failure node to be upgraded. Between storage pools, the failure node is not waited for to be upgraded.

[0124] By supporting upgrade suspension and continuing functions, the node does not have an intermediate state during upgrading, each node is in an upgraded state, the consistency of the storage version of the node is ensured, and the problem of version coexistence before and after upgrading is solved.

[0125] By dynamically adjusting and using a fault-tolerant mechanism, the reliability and stability of the upgrading process are ensured, and the overall reliability of the system is improved.

[0126] As an embodiment of the present application, before the storage system is upgraded online, the following steps are further included:

[0127] Backup system data in the storage system;

[0128] Check the node status of the nodes in the storage system, and determine at least one node to be upgraded based on the nodes with normal node status;

[0129] Check the working status of the hardware devices in the storage system, and determine whether the working status belongs to a preset upgrade state.

[0130] In an embodiment, the pre-upgrade check comprises checking cluster status, network and hardware devices, specifically checking service status of cluster nodes, and only when the service status is normal, the node is upgraded, if the service status is high tension, fault or warning, the node cannot be upgraded, and the non-normal nodes are excluded, and the normal nodes are upgraded. The hardware devices are checked, including network and hardware devices, checking whether the network port status is normal to ensure that the normal network does not affect the upgrade efficiency. The system disk and network card status are checked to prevent abnormal conditions from affecting the upgrade. The pre-upgrade check ensures the smooth upgrade. The preset upgrade status comprises normal network port status, normal system disk and normal network card status.

[0131] Through backup, data security of the storage system is ensured, and through node status check, the fault node is excluded to ensure the normal upgrade process. The system hardware devices are checked to ensure the normal network during the upgrade process, and the upgrade speed is not affected.

[0132] As an embodiment of the present application, before the image file package is created based on the image file package, the following steps are further included:

[0133] The image file package and the signature file corresponding to the image file package are received. In an embodiment, the signature file is a technical means to ensure the integrity, authenticity and traceability of the file by embedding or attaching a digital signature in the file.

[0134] The image file package is verified according to the signature file.

[0135] As an embodiment of the present application, the upgrade content of the node to be upgraded can comprise:

[0136] The kernel, driver and storage version of the node to be upgraded are upgraded according to the image source.

[0137] For example, the kernel, driver and storage version of the node to be upgraded are upgraded according to the image source, the kernel of the node to be upgraded is upgraded first according to the image source, after the kernel is upgraded, the driver and storage version are sequentially upgraded based on the kernel upgrade result. Through the line concurrent upgrade technology based on the upgrade queue, the kernel, driver and storage version of the storage system are quickly and safely upgraded without interrupting the business operation. The storage version is upgraded at the same time as the image system kernel, and the upgrade function and efficiency are improved.

[0138] As an embodiment of the present application, after the upgrade is completed, all nodes of the upgraded storage system are verified to determine whether the kernel version, the driver version and the storage version of all nodes meet the version condition of successful upgrade; and the services of all nodes of the upgraded storage system are verified to determine whether the services of all nodes meet the service condition of successful upgrade.

[0139] When the kernel version, the driver version and the storage version of all nodes meet the version condition of successful upgrade, and the services of all nodes meet the service condition of successful upgrade, it is determined that the storage system is successfully upgraded. In an embodiment, the storage version represents the storage version of the storage system.

[0140] As an embodiment of the present application, as shown in Figure 9 the upgrade full process includes:

[0141] Preparation before upgrade: including backup of current system data, checking of cluster state, uploading and decompression of new version installation package, etc.

[0142] Concurrent upgrade execution: according to the generated upgrade queue, the upgrade tasks of multiple nodes are started at the same time. In the upgrade process, the upgrade controller monitors the upgrade state of each node in real time, and dynamically adjusts as needed. In an embodiment, the upgrade controller monitors the upgrade state of each node in real time, and when the node to be upgraded fails to upgrade, it automatically rolls back and re-upgrades. When the roll-back and re-upgrade is successful, the upgrade of the fault node is completed. When the roll-back and re-upgrade fails, the upgrade is manually restored, and the node can be re-upgraded multiple times manually as necessary to complete the upgrade of the fault node. Or, when the roll-back and re-upgrade fails, the upgrade of the fault node is continued through manual restoration of the upgrade and automatic filling to complete the upgrade of the fault node. In the normal upgrade process, the upgrade controller controls the upgrade sequence of the data node, the metadata node and the storage monitoring node according to the upgrade queue. The data node is upgraded first, and the storage monitoring node is upgraded last.

[0143] The upgrade main process, as shown in Figure 10 , is as follows: the kernel is upgraded, the drivers removed in the previous step are reinstalled, the driver programs under the new kernel are installed, the storage version is upgraded, the node is restarted, and the progress of the software upgrade interface is verified and notified to the management software.

[0144] Verification after upgrade: after the upgrade is completed, the cluster state is verified to ensure that the upgrade is successful and the data consistency is not affected. In an embodiment, the consistency of the driver version, the kernel version and the storage version of all nodes is verified, and the normality of the services of the nodes is verified.

[0145] After the upgrade, the upgrade process report is supported, with clear content, start time, and end time of each step. It also includes printing of upgrade exceptions, alarms and other information, and supports report download.

[0146] Improve upgrade efficiency by enabling online kernel and storage version upgrades within a distributed storage system. This approach utilizes an online concurrent upgrade algorithm to enable fast and secure upgrades without disrupting business operations. Upgrading the storage version also upgrades the mirror system kernel, improving upgrade functionality and efficiency. This reduces business interruption time, losses, and maintenance costs associated with upgrades. Enhanced system reliability: Dynamic adjustments and fault-tolerance mechanisms ensure the reliability and stability of the upgrade process, improving overall system reliability.

[0147] The online concurrent algorithm includes: (1) Node classification and upgrade queue generation: First, based on the underlying data distribution structure of the cluster, the storage nodes are classified and different types of nodes (such as data nodes, metadata nodes, etc.) are identified. Then, based on the actual situation of the current cluster, an optimized upgrade queue is generated to determine the upgrade order of each node.

[0148] (2) Concurrent upgrade execution: During the upgrade process, concurrent upgrade technology is used to simultaneously initiate upgrade tasks for multiple nodes. The upgrade controller monitors the upgrade status of each node in real time to ensure that cluster services are not affected during the upgrade process.

[0149] (3) Dynamic adjustment and fault tolerance mechanism: The upgrade controller dynamically adjusts the upgrade order of each node according to the cluster service status. If an upgrade fails, it can automatically roll back and retry the upgrade to ensure the reliability and stability of the upgrade process.

[0150] An online concurrent upgrade algorithm enables fast and secure upgrades without disrupting business operations. This method includes node classification, upgrade queue generation, concurrent upgrade execution, dynamic adjustment, and fault tolerance. It also supports kernel and storage version upgrades, improving upgrade efficiency, reducing maintenance costs, and enhancing system reliability.

[0151] Based on the same inventive concept, an embodiment of the present application provides a distributed storage system. Figure 11 , Figure 11 1 is a block diagram of a distributed storage system 1100 proposed in one embodiment of the present application. Figure 11 As shown, the system includes: the distributed storage system includes at least one storage pool, and each storage pool includes one or more nodes;

[0152] A first selection module 1101 is configured to determine at least one node to be upgraded in the storage system;

[0153] The second selection module 1102 is configured to select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package;

[0154] A creation module 1103 is configured to determine at least one node cluster based on the node category to which the node to be upgraded belongs, and to determine the upgrade order of each node cluster based on the upgrade priority of the node cluster. Nodes to be upgraded in the same node cluster have the same upgrade priority, while nodes to be upgraded in different node clusters have different upgrade priorities.

[0155] The upgrade module 1104 is used to upgrade the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; after the upgrade of all nodes to be upgraded is completed, the upgraded storage system is obtained.

[0156] As an embodiment of the present application, the creation module 1103 is specifically configured to classify at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster, and a storage monitoring node cluster;

[0157] Create a first-order upgrade queue between classes based on the data node cluster, metadata node cluster, and storage monitoring node cluster;

[0158] Creating a first concurrent upgrade queue for nodes in the data node cluster according to a value of a storage pool in the storage system and a value of a fault domain in the storage pool;

[0159] Creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain;

[0160] Creating a third concurrent upgrade queue for nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain;

[0161] The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue, and the third concurrent upgrade queue are determined as the upgrade sequence of the node clusters.

[0162] As an implementation manner of the present application, the creation module 1103 is specifically configured to obtain the maximum upgrade quantity in the storage pool according to the value of the storage pool;

[0163] Obtaining the concurrent number of single upgrades in the storage pool according to the value of the fault domain;

[0164] A second concurrent upgrade queue for nodes in the metadata node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

[0165] As an implementation of the present application, the upgrade module 1104 is specifically configured to:

[0166] Upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source;

[0167] Simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source;

[0168] Simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source;

[0169] The nodes in the storage monitoring node cluster in the third concurrent upgrade queue are simultaneously upgraded according to the mirror source.

[0170] As an embodiment of the present application, the system further includes: a fault handling module, configured to, when an upgrade failure occurs in the node to be upgraded, re-upgrade the node using the first fault handling mode; when the upgrade in the first fault handling mode succeeds, the upgrade of the faulty node is completed;

[0171] When the first fault handling mode upgrade fails, the second fault handling mode is used to upgrade the faulty node, or when the first fault handling mode upgrade fails, the second and third fault handling modes are used to continue upgrading the faulty node, so that the faulty node upgrade is completed.

[0172] As an implementation manner of the present application, it further includes: an upgrade check module for backing up system data in the storage system;

[0173] Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes in normal node status;

[0174] Checking the working status of the hardware devices in the storage system to determine whether the working status belongs to a preset upgrade status.

[0175] As an implementation manner of the present application, it further includes: a verification module, configured to receive an image file package and a signature file corresponding to the image file package;

[0176] The image file package is verified according to the signature file.

[0177] As an implementation manner of the present application, the upgrade module 1104 is specifically configured to upgrade at least one of the kernel, driver, and storage version of the node to be upgraded according to the image source.

[0178] As an embodiment of the present application, the present application further includes: a verification module for verifying, after the upgrade is completed, all nodes of the upgraded storage system to determine whether the kernel version, driver version, and storage version of all nodes meet the version conditions for a successful upgrade; and verifying the services of all nodes of the upgraded storage system to determine whether the services of all nodes meet the service conditions for a successful upgrade;

[0179] When the kernel versions, driver versions, and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful.

[0180] The distributed storage system provided in the above embodiment can implement the technical solution described in the above embodiment of the method for online upgrading of a storage system. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above embodiment of the method for online upgrading of a storage system, which will not be repeated here.

[0181] Based on the same inventive concept, the present application also provides an electronic device 1200. Figure 12 As shown, the electronic device 1200 includes a processor 1201 , a memory 1202 and a display 1203 . Figure 12 Only some of the components of the electronic device 1200 are shown, but it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0182] In some embodiments, the memory 1202 may be an internal storage unit of the electronic device 1200, such as a hard disk or memory of the electronic device 1200. In other embodiments, the memory 1202 may also be an external storage device of the electronic device 1200, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 1200.

[0183] Furthermore, the memory 1202 may include both an internal storage unit of the electronic device 1200 and an external storage device. The memory 1202 is used to store application software installed on the electronic device 1200 and various data.

[0184] In some embodiments, the processor 1201 can be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code stored in the memory 1202 or process data, such as a method for online upgrading of a storage system in this application.

[0185] In some embodiments, the display 1203 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 1203 is used to display information on the electronic device 1200 and to display a visual user interface. Components 1201-1203 of the electronic device 1200 communicate with each other via a system bus.

[0186] In some embodiments of the present application, when the processor 1201 executes the storage system online upgrade program in the memory 1202, the following steps of the storage system online upgrade method may be implemented:

[0187] It should be understood that, when the processor 1201 executes the storage system online upgrade program in the memory 1202 , in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0188] Furthermore, the embodiment of the present application does not specifically limit the type of the electronic device 1200 mentioned. The electronic device 1200 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer (laptop). Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer (1laptop) with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present application, the electronic device 1200 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0189] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the online upgrade method of the storage system provided by the above methods.

[0190] Based on the same inventive concept, another embodiment of the present invention provides a computer program product comprising a computer program / instructions. When executed by a processor, the computer program / instructions implement the steps of the method for online upgrading a storage system as described in any of the above embodiments of the present invention. Those skilled in the art will appreciate that all or part of the process of implementing the above-described method can be accomplished by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory, etc.

[0191] The above describes in detail the method, system, device and medium for online upgrading of a storage system provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for online upgrading of a storage system, characterized in that: The storage system includes at least one storage pool, each storage pool includes one or more nodes, and the method includes: Determining at least one node to be upgraded in the storage system; Select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package; Determine at least one node cluster based on the node cluster category to which the node to be upgraded belongs, and determine the upgrade order of each node cluster based on the upgrade priority of the node cluster. The upgrade priorities of nodes to be upgraded in the same node cluster are the same, while the upgrade priorities of nodes to be upgraded in different node clusters are different. According to the mirror source, the nodes to be upgraded of each node cluster are upgraded in the upgrade order of each node cluster. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously. After all nodes to be upgraded are upgraded, the upgraded storage system is obtained.

2. A method for online upgrading of a storage system according to claim 1, characterized in that: The step of determining at least one node cluster according to the node cluster category to which the node to be upgraded belongs, and determining the upgrade order of each node cluster according to the upgrade priority of the node cluster, specifically includes: Classifying at least one node to be upgraded to obtain at least one node cluster; wherein the at least one node cluster includes: a data node cluster, a metadata node cluster, and a storage monitoring node cluster; Create a first-order upgrade queue between classes based on the upgrade priorities of data node clusters, metadata node clusters, storage monitoring node clusters, and node clusters; Creating a first concurrent upgrade queue for nodes in the data node cluster according to a value of a storage pool in the storage system and a value of a fault domain in the storage pool; Creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain; Creating a third concurrent upgrade queue for nodes in the storage monitoring node cluster according to the value of the storage pool and the value of the fault domain; The first sequential upgrade queue, the first concurrent upgrade queue, the second concurrent upgrade queue, and the third concurrent upgrade queue are determined as the upgrade sequence of the node clusters.

3. The method for online upgrading of a storage system according to claim 2, characterized in that: The step of creating a second concurrent upgrade queue for nodes in the metadata node cluster according to the value of the storage pool and the value of the fault domain specifically includes: Obtaining the maximum upgrade quantity in the storage pool according to the value of the storage pool; Obtaining the concurrent number of single upgrades in the storage pool according to the value of the fault domain; A second concurrent upgrade queue for nodes in the metadata node cluster is created according to the maximum upgrade quantity in the storage pool and the concurrent quantity of a single upgrade in the storage pool.

4. A method for online upgrading of a storage system according to claim 2 or 3, characterized in that: The step of upgrading the nodes to be upgraded in each node cluster according to the image source and in the upgrade order of each node cluster specifically includes: Upgrading the nodes to be upgraded of each node cluster in the first sequential upgrade queue in sequence according to the mirror source; Simultaneously upgrading the nodes in the data node cluster in the first concurrent upgrade queue according to the mirror source; Simultaneously upgrading the nodes in the metadata node cluster in the second concurrent upgrade queue according to the mirror source; The nodes in the storage monitoring node cluster in the third concurrent upgrade queue are simultaneously upgraded according to the mirror source.

5. A method for online upgrading of a storage system according to claim 4, characterized in that: Also includes: When an upgrade failure occurs in the node to be upgraded, the upgrade is performed again using the first fault handling mode. When the upgrade is successful in the first fault handling mode, the upgrade of the faulty node is completed. When the upgrade in the first fault handling mode fails, the upgrade in the second fault handling mode is performed to complete the upgrade of the faulty node; Alternatively, when the upgrade in the first fault handling mode fails, the faulty node is continuously upgraded through the second fault handling mode and the third fault handling mode to complete the upgrade of the faulty node.

6. The method for online upgrading of a storage system according to claim 1, characterized in that: Before the storage system is upgraded online, the following steps are also required: Backing up system data in the storage system; Checking node status of multiple nodes of the storage system, and determining at least one node to be upgraded based on nodes in normal node status; Checking the working status of the hardware devices in the storage system to determine whether the working status belongs to a preset upgrade status.

7. A method for online upgrading of a storage system according to claim 1 or 6, characterized in that: Before creating the mirror source based on the mirror file package, the method further includes: Receive an image file package and a signature file corresponding to the image file package; The image file package is verified according to the signature file.

8. A method for online upgrading of a storage system according to claim 1 or 6, characterized in that: The upgrading of the node to be upgraded specifically includes: At least one of the kernel, driver, and storage version of the node to be upgraded is upgraded according to the image source.

9. A method for online upgrading of a storage system according to claim 1 or 6, characterized in that: Also includes: After the upgrade is complete, verify all nodes of the upgraded storage system to determine whether the kernel version, driver version, and storage version of all nodes meet the version conditions for a successful upgrade; and verify the services of all nodes of the upgraded storage system to determine whether the services of all nodes meet the service conditions for a successful upgrade; When the kernel versions, driver versions, and storage versions of all nodes meet the version conditions for successful upgrade, and the services of all nodes meet the service conditions for successful upgrade, it is determined that the storage system upgrade is successful.

10. A distributed storage system, characterized in that: The distributed storage system includes at least one storage pool, each of which includes one or more nodes, including: A first selection module is configured to determine at least one node to be upgraded in the storage system; A second selection module is used to select any node to be upgraded as a control node to receive the image file package, and create an image source based on the image file package; A creation module is used to determine at least one node cluster based on the node category to which the node to be upgraded belongs, and to determine the upgrade order of each node cluster based on the upgrade priority of the node cluster. The upgrade priority of nodes to be upgraded in the same node cluster is the same, and the upgrade priority of nodes to be upgraded in different node clusters is different. The upgrade module is used to upgrade the nodes to be upgraded in each node cluster according to the upgrade order of each node cluster based on the mirror source. During the upgrade process, multiple nodes to be upgraded belonging to the same node cluster and the same storage pool are upgraded simultaneously; after the upgrade of all nodes to be upgraded is completed, the upgraded storage system is obtained.

11. An electronic device, characterized in that: comprising a memory and a processor, wherein, The memory is used to store programs; The processor is coupled to the memory and is configured to execute the program stored in the memory to implement the steps in the method for online upgrading of a storage system as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the method for online upgrading of a storage system as described in any one of claims 1 to 9.

Images