Distributed storage system online upgrading method

Abstract

The invention provides an online upgrading method for a distributed storage system, and relates to the technical field of distributed storage, and the method comprises the following steps: determining an upgrading sequence of each node in the system; creating a logic unit LUN used for caching a write IO request, and excluding the node to be upgraded from the copy; switching a storage service link of the to-be-upgraded node to a specified node; marking the state of the node to be upgraded, and redirecting and pre-writing log records according to the read-write request; and after the nodes are upgraded and restarted, synchronizing data from the LUN and playing back a WAL log, marking that node upgrading is completed after data recovery is completed, recovering a storage link, and sequentially carrying out subsequent node upgrading. According to the method, one LUN is used for storing the IO to the node in the restarting process of the rolling upgrading node, online rolling upgrading of the distributed storage system is achieved, the problems of service interruption and data inconsistency in the upgrading process are avoided, the availability, safety and performance of the system are improved, and the continuous service requirement in the high-load environment is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed storage, and particularly relates to an online upgrading method of a distributed storage system. BACKGROUND

[0002] The distributed storage system plays an important role in modern computer systems, and its online upgrading is crucial for maintaining the availability and security of the system. However, traditional upgrading methods often need to stop the system or affect the performance of the system, which may cause serious economic losses and security risks. Especially in large-scale distributed storage systems, due to the large number of nodes and large amount of data, traditional upgrading methods will face great challenges. Therefore, how to realize the online upgrading of the distributed storage system while ensuring the availability, security and performance of the system has become a problem to be solved.

[0003] Currently, the online upgrading of the distributed storage system mainly faces two challenges: first, how to upgrade without stopping the system; second, how to ensure data consistency and integrity during the upgrading process. In order to solve these challenges, researchers have proposed various methods, including rolling upgrade and data replication. However, these methods also have some limitations, such as large resource consumption, long upgrading time, etc. In addition, the online upgrading of the distributed storage system also needs to consider the scalability, reliability and security of the system. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0005] To this end, the first object of the present application is to propose an online upgrading method of a distributed storage system.

[0006] The second object of the present application is to propose an online upgrading device of a distributed storage system.

[0007] The third object of the present application is to propose an electronic device.

[0008] The fourth object of the present application is to propose a computer-readable storage medium.

[0009] The fifth object of the present application is to propose a computer program product.

[0010] To achieve the above objects, the first aspect of the present application proposes an online upgrading method of a distributed storage system, comprising:

[0011] determining the upgrading order of all nodes in the distributed storage system, the upgrading order being arranged from low to high according to the business load of each node;

[0012] create a logical unit (LUN) for caching write IO requests in the system;

[0013] According to the upgrade sequence, switch the storage service link of the node to be upgraded to the node with the lowest load in the last node in the upgrade sequence or the upgraded node;

[0014] Mark the state of the node to be upgraded as to be upgraded, and process IO requests, perform node upgrade and restart operations, wherein read requests are redirected to non-upgrade nodes, write requests are written to LUN and recorded in the pre-write log (WAL) format if they involve upgrade nodes, and other write requests are normally written to replicas;

[0015] After upgrading and restarting the node to be upgraded, synchronize data from the LUN and replay the WAL log, mark the node as upgraded after all log replay is completed, delete the LUN and restore the original link, and sequentially perform the next node upgrade until all are completed.

[0016] Optionally, the LUN for caching write IO requests in the system includes:

[0017] Collect the identity information of the current node to be upgraded, and select n nodes as LUN replica storage targets in the remaining N-1 non-upgrade nodes according to the replica scheduling strategy;

[0018] The LUN is created in a thin provisioning configuration mode, and the LUN only applies for storage space and data objects as needed when actually receiving write IO requests;

[0019] Enable the write-ahead log function in the LUN, sequentially record and shard store write IO to ensure write sequence consistency and fault recovery capability.

[0020] Optionally, according to the upgrade sequence, switch the storage service link of the node to be upgraded to the node with the lowest load in the last node in the upgrade sequence or the upgraded node, including:

[0021] Identify the last un-upgraded node in the upgrade sequence based on the current upgrade state of the system, and if the node is not the current node to be upgraded, switch the storage service link of the node to be upgraded to the node;

[0022] If the current node to be upgraded is the last node in the upgrade sequence, select the node with the lowest business load from the nodes that have completed upgrade as the target node.

[0023] Optionally, the node to be upgraded is marked as to be upgraded, and IO requests are processed, and node upgrade and restart operations are performed, including:

[0024] Before the node to be upgraded enters the upgrade process, the upgrade state change notification is broadcast to all nodes in the cluster, and the state of the node is uniformly marked as to be upgraded;

[0025] After all nodes in the cluster receive the state, the local metadata and the replica availability table are updated, and the IO scheduling strategy is dynamically adjusted;

[0026] For read requests, the nearest or lightest load replica is selected from non-upgrade nodes for data access according to the replica location and upgrade state; for write requests, it is judged whether the target replica contains the upgrade node, if yes, the write request is converted into a log write operation, and is written into the corresponding replica location in the LUN in the WAL format; for write requests not involving the upgrade node, the corresponding replica node is directly written according to the original replica mechanism, and the normal write path is maintained;

[0027] The node to be upgraded starts to replace the upgrade package, and after the replacement of the upgrade package is completed, the service is restarted.

[0028] Optionally, the service restart of the node supports two service restart modes: only restarting the service without restarting the physical host, or restarting the service by restarting the physical host.

[0029] Optionally, after the node to be upgraded is upgraded and restarted, the data is synchronized from the LUN and the WAL log is played back, the node is marked as upgraded after all log playback is completed, the LUN is deleted and the original link is restored, and the next node upgrade is performed in sequence until all nodes are upgraded, including:

[0030] After the node is upgraded and restarted online, the LUN replica location related to the node is automatically identified, and a data synchronization task is started to synchronize the write IO data written to the LUN during the upgrade to the local;

[0031] During the data synchronization process, the write operation records in the WAL log are extracted and loaded into the memory, and are played back in chronological order to reconstruct the data write not completed during the upgrade;

[0032] For write requests not completed during the playback process, the WAL mechanism is continued to be used for writing according to the access hit condition and the consistency state is maintained;

[0033] After the playback is completed, the node state is switched from to be upgraded to upgraded, and the cluster restores the normal use of the node in replica scheduling and data access;

[0034] After it is confirmed that the WAL log has been completely processed, the LUN replica corresponding to the node is deleted, the storage service link is migrated from the switching node to the node, and the next node is prepared to enter the upgrade process according to the upgrade sequence;

[0035] After all nodes are upgraded, the distributed storage system is upgraded.

[0036] To achieve the above object, the second aspect of the present application proposes an online upgrade device of a distributed storage system, comprising:

[0037] An upgrade sequence determination module is configured to determine an upgrade sequence of all nodes in the distributed storage system, wherein the upgrade sequence is arranged from low to high according to the business load of each node;

[0038] A LUN creation module is configured to create a logical unit LUN for caching write IO requests in the system;

[0039] An upgrade state marker is configured to switch the storage service link of a node to be upgraded to the node with the lowest load among the last node in the upgrade sequence or the upgraded nodes according to the upgrade sequence;

[0040] An IO processing module and a node restart module are configured to mark the state of a node to be upgraded as to be upgraded, process IO requests, and perform node upgrade and restart operations, wherein read requests are redirected to non-upgraded nodes, write requests involving upgraded nodes are written into the LUN and recorded in the write-ahead log WAL format, and other write requests are normally written into the replica;

[0041] A data recovery module and an upgrade promotion module are configured to synchronize data from the LUN and replay the WAL log after upgrading and restarting the node to be upgraded, mark the node as upgraded after all log replay is completed, delete the LUN and restore the original link, and sequentially perform the next node upgrade until all nodes are upgraded.

[0042] To achieve the above object, the third aspect of the present application proposes an electronic device, comprising a processor and a memory in communication connection with the processor;

[0043] The memory stores computer execution instructions;

[0044] The processor executes the computer execution instructions stored in the memory to implement the method of any one of the first aspect.

[0045] To achieve the above object, the fourth aspect of the present application proposes a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of the first aspect.

[0046] To achieve the above object, the fifth aspect of the present application proposes a computer program product, wherein the computer program is executed by the processor to implement the method of any one of the first aspect.

[0047] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0049] Figure 1 A flow chart of a distributed storage system online upgrade method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a LUN-based upgrade provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of IO processing during a LUN-based upgrade process provided in an embodiment of the present application;

[0052] Figure 4 Schematic diagram of data synchronization and IO processing after the upgrade node restart provided in the embodiment of the present application;

[0053] Figure 5 This is a structural diagram of an online upgrade device for a distributed storage system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0055] In order to solve the problems existing in the current solution, the embodiment of the present application provides a distributed storage system online upgrade method. Figure 1 This is a flow chart of a distributed storage system online upgrade method provided by an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0056] Step 101: Determine the upgrade order of all nodes in the distributed storage system. The upgrade order is arranged from low to high according to the service load of each node.

[0057] In the embodiment of the present application, step 101 is configured to determine the upgrade sequence of all nodes in the distributed storage system. Specifically, the present application sorts all nodes from low to high according to the business load of each node to form the sequence of upgrade. The sorting strategy is based on the current running load of the node, and the node with lower load is selected for upgrade in priority to reduce the impact on the overall performance and business response of the system during the upgrade process.

[0058] In the embodiment of the present application, the determined upgrade sequence remains unchanged during the entire upgrade process, ensuring the coherence and controllability of the upgrade process and avoiding the uncertainty caused by adjusting the sequence during the upgrade process. In addition, by fixing the upgrade sequence, it is helpful for the system administrator to plan the upgrade in advance and ensure the smooth implementation of the upgrade operation.

[0059] It should be noted that although the sorting method from low to high load is used as the basis for determining the upgrade sequence in the embodiment of the present application, other reasonable strategies can also be used according to actual application requirements, such as sorting based on factors such as node hardware configuration, failure rate statistics or business importance, which are all within the protection scope of the present application.

[0060] Step 102, creating a logical unit LUN for caching write IO requests in the system.

[0061] In the embodiment of the present application, step 102 involves creating a logical unit (LUN) for caching write IO requests in the distributed storage system. Specifically, the system first collects the identity information of the current node to be upgraded, and selects n nodes from the remaining N-1 non-upgrade nodes as the replica storage targets of the LUN according to the replica scheduling strategy. This selection process fully considers system load balancing and data security to ensure that the LUN replicas can stably and efficiently bear the write requests during the upgrade period.

[0062] In order to improve the efficiency of resource utilization, the present application creates the LUN by using the simplified preparation configuration method. That is, the LUN is not pre-allocated with a fixed storage space, but dynamically applies for storage space and data objects when actually receiving write IO requests, thereby effectively reducing the waste of storage resources and improving the overall performance and flexibility of the system.

[0063] In addition, the LUN enables the Write-Ahead Logging (WAL) function, which sequentially records and sharded stores all write IO operations to ensure the strict consistency of the write order. This not only ensures the integrity of the data during the upgrade process, but also greatly improves the recovery ability of the system under abnormal conditions, ensuring the data security and stable operation of the distributed storage system during online upgrade.

[0064] The application realizes efficient caching and management of write IO requests during node upgrading through the above steps, and lays a solid foundation for online rolling upgrade of a distributed storage system.

[0065] In step 103, the storage service link of the node to be upgraded is switched to the node with the lowest load in the last node or the upgraded node in the upgrade sequence.

[0066] In the embodiment of the application, step 103 is used to perform the storage service link switching operation of the node to be upgraded according to the determined upgrade sequence. Specifically, the system first identifies the last node that has not completed upgrading in the upgrade sequence based on the current upgrade state. If the last node to be upgraded is different from the current node to be upgraded, the system switches the storage service link of the node to be upgraded to the node, realizing seamless transfer of business requests.

[0067] If the current node to be upgraded is the last node in the upgrade sequence, the system will select the node with the lowest business load from the nodes that have completed upgrading as the target node of the storage service link switching. By selecting the node with the lowest load as the destination of the link switching, the pressure on the target node can be effectively reduced, ensuring the performance and stability of the system as a whole during upgrading.

[0068] The link switching operation ensures that the original storage service of the node to be upgraded can be timely taken over by other normally running nodes during upgrading and restarting of the node to be upgraded, avoiding business interruption and data access delay, and improving the online upgrading capability of the system and user experience.

[0069] In a possible embodiment, as shown in Figure 2 , the storage link is switched from node A to node B. If the node to be upgraded is the last node in the upgrade sequence, the node with the lowest business load is selected from the upgraded nodes as the destination node of the storage service link switching of the node to be upgraded.

[0070] The above strategy is used in the embodiment of the application to realize flexible scheduling and dynamic switching of the storage service link, providing a key guarantee for high-availability online upgrading of a distributed storage system

[0071] In step 104, the state of the node to be upgraded is marked as "to be upgraded", and IO requests are processed, and node upgrading and restarting operations are performed, wherein read requests are redirected to non-upgraded nodes, write requests involving upgraded nodes are written to LUN and recorded in the pre-write log WAL format, and other write requests are normally written to replicas.

[0072] In the embodiment of the application, referring to Figure 3 , step 104 involves marking the state of the node to be upgraded as "to be upgraded", and processing IO requests accordingly, while performing node upgrading and restarting operations.

[0073] Specifically, before the node to be upgraded formally enters the upgrade process, the system broadcasts the state change notification of the node to all nodes in the cluster, ensuring that all nodes update the state of the node to be upgraded to "to be upgraded" in synchronization. After receiving the notification, each node updates the local metadata and the replica availability table in time, and dynamically adjusts its IO scheduling strategy to adapt to the state change caused by node upgrade.

[0074] In this state, for read requests, the system selects the nearest or the replica node with lighter business load among the non-upgrade nodes according to the distribution position of the replica and the upgrade state of the node to perform data access, ensuring that the read request is not affected by the upgrade node and improving the read performance and availability of the system. For write requests, the system first judges whether the target replica contains the upgrade node. If it does, the write request is converted into a log write operation, which is written to the corresponding replica position of the LUN in the write-ahead log (WAL) format to ensure the integrity and consistency of the write operation. If the write request does not involve the upgrade node, the write request is normally written to the corresponding replica node according to the original replica write mechanism, maintaining the normal operation of the write operation path.

[0075] After the above IO request processing preparation is completed, the node to be upgraded starts the replacement operation of the upgrade package. After the replacement of the upgrade package is completed, the system supports two restart modes: one is to restart only the storage service without involving the physical host restart, and the other is to restart the service by restarting the physical host. The support of the two modes provides flexibility to meet different operation and maintenance scenarios and upgrade needs.

[0076] Through this step in the embodiment of the application, efficient scheduling and security of read and write requests during the upgrade process are realized, ensuring the continuity and stability of the distributed storage system upgrade.

[0077] Step 105, after upgrading and restarting the node to be upgraded, synchronizing data from the LUN and replaying the WAL log, marking the node as upgraded after all log replay is completed, deleting the LUN and restoring the original link, and sequentially upgrading the next node until all nodes are upgraded.

[0078] In the embodiment of the application, with reference to Figure 4 , step 105 involves synchronizing data from the logical unit LUN and replaying the write-ahead log (WAL) after the node to be upgraded completes the upgrade and restarts, to realize consistency recovery of data and switching of the node state, finally completing the upgrade process of the node, and sequentially starting the upgrade of the next node until all nodes are upgraded.

[0079] Specifically, after the node to be upgraded completes the upgrade and is back online, the system automatically identifies the LUN copy location associated with the node, starts a data synchronization task, and synchronizes all write IO data forwarded to the node during the upgrade from the LUN to the local storage. Since the LUN adopts a thin provisioning mode, the actual amount of data to be synchronized only includes the write request data generated during the service restart process, avoiding unnecessary data redundancy and synchronization overhead. This not only speeds up the data synchronization, but also significantly reduces the resource occupation during system upgrade, thereby reducing the impact on the overall business performance.

[0080] During the data synchronization process, the system extracts the write operation logs recorded in the WAL format in the LUN, loads them into the node memory, and replays them one by one in chronological order to reconstruct the write operations that have not been completed during the upgrade, ensuring data integrity and consistency. This replay process ensures that all write requests are applied to the data copy in the correct order, avoiding data loss or inconsistency caused by upgrades and restarts. For write requests that have not been completed during the replay process, the system continues to use the WAL mechanism for writing based on access hits, ensuring consistent data state during the replay process, and the system can respond to new write requests in real time, ensuring business continuity and reliability.

[0081] When all WAL logs are replayed, the node state is switched from "to be upgraded" to "upgrade completed", and a request to join the cluster is sent to other nodes in the cluster. After receiving the request, the cluster updates the node state and restores its normal role in replica scheduling and data access, allowing read and write requests to the node to be processed normally. At this time, the node re-joins the cluster and assumes its data storage and service responsibilities in the distributed storage system, ensuring the overall consistency and high availability of the system.

[0082] After confirming that all WAL logs have been successfully processed and replayed, the system deletes the LUN copy corresponding to the node, completes the release of resources, and avoids wasting storage resources. At the same time, the storage service link is switched back from the previously switched target node to the current node, achieving normal recovery of the storage link and ensuring the correctness and efficiency of the data access path. This switching operation is automatically completed by the system without human intervention, further improving the automation and intelligence level of the upgrade.

[0083] Finally, according to the predetermined upgrade sequence, the system prepares to start the upgrade process of the next node. Repeat the above steps until all nodes are upgraded, and finally realize the complete online upgrade of the distributed storage system. Through this rolling upgrade method, the system can gradually complete the software version update of the entire system without downtime, maximizing the continuity of business and the safety of data.

[0084] Through the embodiments of the present application, the system realizes efficient synchronization and log playback of upgraded node data, guarantees data consistency and system stability, and improves the availability and security of online upgrade of the distributed storage system. In addition, the embodiments of the present application also provide a flexible upgrade strategy, which can dynamically adjust the upgrade rhythm and sequence according to the actual business load and performance requirements, further optimize system resource utilization and upgrade efficiency, and meet the needs of diversified application scenarios.

[0085] In addition, it should be noted that if the speed of upgrading is improved, the upgrade of the next node can be performed as soon as the synchronization of the LUN data is completed. However, according to the IO request processing strategy defined in the embodiments of the present application, since the WAL log of the previous node has not been completely synchronized, it will have a certain impact on performance. In the business scenario that is not sensitive to performance, this upgrade scheme can be adopted to speed up the upgrade speed.

[0086] In order to realize the above-mentioned embodiments, the present application further provides a distributed storage system online upgrade device. Figure 5 A structural schematic diagram of a distributed storage system online upgrade device provided by the embodiments of the present application is shown in FIG. 1. As shown in the figure, the device comprises: Figure 5

[0087] An upgrade sequence determination module 100 is configured to determine the upgrade sequence of all nodes in the distributed storage system, wherein the upgrade sequence is arranged from low to high according to the business load of each node;

[0088] A LUN creation module 200 is configured to create a logical unit LUN for caching write IO requests in the system;

[0089] An upgrade state marking 300 is configured to switch the storage service link of the node to be upgraded to the node with the lowest load in the last node or the upgraded node in the upgrade sequence according to the upgrade sequence;

[0090] An IO processing module and node restart module 400 is configured to mark the state of the node to be upgraded as to be upgraded, process IO requests, and perform node upgrade and restart operations, wherein the read request is redirected to the non-upgraded node, the write request is written into the LUN and recorded in the pre-write log WAL format if it involves the upgraded node, and other write requests are normally written into the replica;

[0091] A data recovery module and upgrade promotion module 500 is configured to synchronize data from the LUN and play back the WAL log after the node to be upgraded is upgraded and restarted, mark the node as upgraded after all log playback is completed, delete the LUN and restore the original link, and sequentially perform the next node upgrade until all nodes are upgraded.

[0092] ​With regard to the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0093] To achieve the above-described embodiments, the present application further provides an electronic device, comprising: a processor, and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to realize the method provided by the foregoing embodiments.

[0094] To achieve the above-described embodiments, the present application further provides a computer readable storage medium, the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to realize the method provided by the foregoing embodiments.

[0095] To achieve the above-described embodiments, the present application further provides a computer program product, comprising a computer program, the computer program is executed by a processor to realize the method provided by the foregoing embodiments.

[0096] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the present application comply with relevant laws and regulations and do not violate public order and good customs.

[0097] It should be noted that personal information from users should be collected for legitimate and reasonable purposes, and should not be shared or sold outside these legitimate uses. In addition, such collection / sharing should be carried out after the user's informed consent is received, including but not limited to informing the user to read the user agreement / user notice before the user uses the function, and signing the agreement / authorization including authorization of relevant user information. In addition, any necessary steps should be taken to protect and secure access to such personal information data, and to ensure that other people with access to personal information data comply with their privacy policies and processes.

[0098] The present application is expected to provide embodiments in which users can selectively prevent the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, the risk is minimized by limiting data collection and deleting data. In addition, such personal information is de-identified, as applicable, to protect the privacy of users.

[0099] In the foregoing detailed description, reference is made to descriptive terms such as "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. for describing various embodiments of the application. These descriptive terms are used for the purpose of the description and are not meant to limit or restrict the scope of the application. The use of these terms does not imply that the application is comprised of at least the described embodiments, or that the described embodiments are the only embodiments the application is comprised of. The scope of the application is not limited to the described embodiments, but is rather defined by the appended claims. In the description of the embodiments of the application, reference is made to the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. which are meant to describe a particular feature, structure, material or characteristic included in at least one embodiment of the application. The illustrative description of these terms does not imply that the application is comprised of at least the described embodiments or that the described embodiments are the only embodiments the application is comprised of. In the description of the embodiments of the application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example described previously. Moreover, the described features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples of the application. Furthermore, the described embodiments and features of the described embodiments can be combined with other embodiments or features of the described embodiments, unless such a combination is clearly not possible, or is technically infeasible.

[0100] Furthermore, the terms "first", "second", etc. are used herein only to describe a certain feature, structure, material, etc. and do not imply a relative importance or a specific order. Thus, a feature defined with "first" or "second" can implicitly or explicitly comprise at least one of this feature. In the description of the application, the meaning of "a plurality" is at least two, such as for example two, three or four, unless expressly specified otherwise.

[0101] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments which can be managed as one or more modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions or steps, and the various embodiments of the application can include additional or fewer functions, steps, or codes. Moreover, the various embodiments of the application can be implemented in software, hardware, firmware, or any combination thereof.

[0102] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, a processor-based system, or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or a combination thereof. For the purposes of this specification, a "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus or device. The computer readable medium can specifically be, but is not limited to, the following: an electronic connection (electronic apparatus) having one or more wires, a portable computer diskette (magnetic apparatus), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disk read-only memory (CDROM). In addition, the computer readable medium can even be paper or other suitable medium upon which the program can be printed, because the program can be electronically obtained, for example, by optically scanning the paper or other medium, then

[0103] It should be understood that portions of the application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware and in another embodiment, any of the following technologies, or a combination thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0104] Those of skill in the art will understand that the steps carried out by the above-mentioned embodiments can be implemented by programs instructing relevant hardware to complete all or part of the steps, and the programs can be stored in a computer readable storage medium. When the programs are executed, they include one of the steps of the method embodiments or a combination thereof.

[0105] In addition, each of the function units in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0106] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

[0107] It should be understood that various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0108] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A distributed storage system online upgrade method, characterized in that: The following steps are involved: Determine an upgrade order for all nodes in the distributed storage system, wherein the upgrade order is arranged from low to high according to the service load of each node; Create a logical unit LUN in the system for caching write IO requests; According to the upgrade sequence, the storage service link of the node to be upgraded is switched to the last node in the upgrade sequence or the node with the lowest load among the upgraded nodes; Mark the node to be upgraded as pending upgrade, process I / O requests, and perform node upgrade and restart operations. Read requests are redirected to non-upgraded nodes. Write requests involving upgraded nodes are written to the LUN and recorded in the write-ahead log (WAL) format. Other write requests are written to the replica normally. After upgrading and restarting the node to be upgraded, synchronize data from the LUN and replay the WAL log. After all logs are replayed, mark the node as upgraded. Delete the LUN and restore the original link. Then upgrade the next node in sequence until all are completed.

2. The method according to claim 1, characterized in that The step of creating a logical unit LUN for caching write IO requests in the system includes: Collect the identity information of the node to be upgraded and select n nodes from the remaining N-1 nodes not to be upgraded as LUN replica storage targets according to the replica scheduling policy; The LUN is created using a thin provisioning configuration method, and the LUN only applies for storage space and data objects on demand when a write IO request is actually received; The write-ahead log function is enabled in the LUN to sequentially record and fragment the write IO to ensure write order consistency and fault recovery capability.

3. The method according to claim 2, characterized in that The step of switching the storage service link of the node to be upgraded to the last node in the upgrade sequence or the node with the lowest load among the upgraded nodes according to the upgrade sequence includes: Based on the current system upgrade status, the last non-upgraded node in the upgrade sequence is identified. If this node is not the node to be upgraded, the storage service link of the node to be upgraded is switched to this node. If the node to be upgraded is the last node in the upgrade sequence, the node with the lowest service load is selected from the nodes that have completed the upgrade as the target node.

4. The method according to claim 3, characterized in that The process of marking the node to be upgraded as pending upgrade, processing IO requests, and performing node upgrade and restart operations includes: Before the node to be upgraded enters the upgrade process, the upgrade status change notification is broadcast to all nodes in the cluster, and the node status is uniformly marked as pending upgrade; After receiving this status, all nodes in the cluster update local metadata and replica availability tables and dynamically adjust the IO scheduling strategy; For read requests, the closest or lightest-loaded replica is selected from non-upgraded nodes for data access based on the replica location and upgrade status. For write requests, the target replica is determined to include an upgraded node. If so, the write request is converted into a log write operation and written to the corresponding replica location in the LUN in WAL format. For write requests that do not involve upgraded nodes, the data is directly written to the corresponding replica node according to the original replication mechanism, maintaining the normal write path. The node to be upgraded starts replacing the upgrade package, and restarts the service after the upgrade package replacement is completed.

5. The method according to claim 4, characterized in that Node service restart supports two service restart methods: restarting only the service without restarting the physical host, or restarting the service by restarting the physical host.

6. The method according to claim 5, characterized in that After the upgrade and restart of the node to be upgraded, data is synchronized from the LUN and the WAL log is replayed. After all logs are replayed, the node is marked as upgraded. The LUN is deleted and the original link is restored. The next node is upgraded in sequence until all are completed, including: After the node is upgraded and restarted, the LUN replica location associated with the node is automatically identified and a data synchronization task is started to synchronize the write IO data written to the LUN during the upgrade to the local machine. During data synchronization, write operation records in the WAL log are extracted and loaded into memory, and replayed in chronological order to reconstruct data written during the upgrade. For write requests that have not been completed during the playback process, the WAL mechanism is used to continue writing based on the access hit situation and maintain consistency; After the replay is complete, the node status switches from pending upgrade to upgrade completed, and the cluster resumes normal use of the node in replica scheduling and data access; After confirming that all WAL logs have been processed, delete the LUN copy corresponding to the node and migrate the storage service link from the switching node back to the current node. Prepare to enter the upgrade process of the next node according to the upgrade sequence. Once all nodes have been upgraded, the distributed storage system is upgraded.

7. A distributed storage system online upgrade device, characterized in that: include: An upgrade sequence determination module is used to determine the upgrade sequence of all nodes in the distributed storage system, wherein the upgrade sequence is arranged from low to high according to the service load of each node; A LUN creation module is used to create a logical unit LUN in the system for caching write IO requests; An upgrade status flag is used to switch the storage service link of the node to be upgraded to the last node in the upgrade sequence or the node with the lowest load among the upgraded nodes according to the upgrade sequence; The IO processing module and node restart module are used to mark the status of the node to be upgraded as pending upgrade, process IO requests, and perform node upgrade and restart operations. Among them, read requests are redirected to non-upgraded nodes. If the write request involves the upgraded node, it is written to the LUN and recorded in the write-ahead log (WAL) format. Other write requests are written to the replica normally. The data recovery module and upgrade promotion module are used to synchronize data from the LUN and replay the WAL log after upgrading and restarting the node to be upgraded. After all logs are replayed, the node is marked as upgraded, the LUN is deleted, and the original link is restored. The next node is upgraded in sequence until all are completed.

8. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images