Metadata node upgrading method and device in distributed cluster, equipment and medium

Abstract

The present disclosure relates to a metadata node upgrading method and device in a distributed cluster, equipment and medium. The method comprises: in response to an upgrading instruction of a metadata node to be upgraded, sending a first loading instruction to a target metadata node; the first loading instruction is used to instruct the target metadata node to load read-only segment metadata in the metadata node to be upgraded at a first loading rate; determining whether the read-only segment metadata is loaded completely, and in the case of determining that the loading is completed, sending a second loading instruction to the target metadata node; the second loading instruction is used to instruct the target metadata node to load read-write segment metadata in the metadata node to be upgraded at a second loading rate, the first loading rate is less than the second loading rate; after receiving a loading completion notification of the read-write segment metadata, upgrading the metadata node to be upgraded. The present disclosure can realize smooth upgrading of the metadata node without affecting the business performance, and improve the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of distributed storage systems, and in particular to a method, apparatus, device, and medium for upgrading metadata nodes in a distributed cluster. Background Technology

[0002] In a distributed cloud storage system, metadata nodes are core components responsible for maintaining key metadata such as data location and attributes. This information resides in memory to ensure a fast response to query requests from computing nodes, thereby guiding them to access the data nodes that actually store the data.

[0003] In related technologies, metadata node upgrades are usually achieved through direct redirection. However, direct redirection often causes system resource conflicts during the upgrade process, resulting in a severe decline in the service performance of the metadata node. This makes it impossible to achieve a smooth upgrade without the user's awareness while ensuring business continuity. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and medium for upgrading metadata nodes in a distributed cluster.

[0005] According to a first aspect of this disclosure, a method for upgrading a metadata node in a distributed cluster is provided, applied to a management node in the distributed cluster, the method comprising:

[0006] In response to an upgrade instruction for the metadata node to be upgraded, a first loading instruction is sent to the target metadata node; the first loading instruction is used to instruct the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a first loading rate;

[0007] Determine whether the read-only segment metadata has been loaded completely, and if it is determined that the loading has been completed, send a second loading instruction to the target metadata node; the second loading instruction is used to instruct the target metadata node to load the read-write segment metadata in the metadata node to be upgraded at a second loading rate, wherein the first loading rate is less than the second loading rate;

[0008] After receiving a notification that the loading of the read-write segment metadata is complete, the metadata node to be upgraded is upgraded.

[0009] Furthermore, the step of upgrading the metadata node to be upgraded after receiving the notification that the loading of the read-write segment metadata is complete includes:

[0010] Upon receiving a notification that the loading of the read-write segment metadata is complete, the service traffic is switched from the metadata node to be upgraded to the target metadata node, and an upgrade operation is performed on the metadata node to be upgraded.

[0011] Further, determining whether the read-only segment metadata has been fully loaded includes:

[0012] Polling requests are sent to the target metadata node at preset time intervals; wherein, the polling requests are used to request the target metadata node to update the loading progress of the read-only segment metadata;

[0013] Receive a loading status response returned by the target metadata node; wherein the loading status response is used to indicate the loading progress of the target metadata node on the read-only segment metadata;

[0014] Based on the loading status response, determine whether the read-only segment metadata has been loaded completely.

[0015] Furthermore, before sending the first load instruction to the target metadata node, the method further includes:

[0016] The target metadata node is determined from multiple candidate metadata nodes based on business load metrics; wherein the business load metrics include at least one of the following: input load, output load, amount of data already carried, and available resource capacity.

[0017] According to a second aspect of this disclosure, a method for upgrading a metadata node in a distributed cluster is provided, applied to a target metadata node in the distributed cluster, the method comprising:

[0018] In response to the first loading instruction sent by the management node, the read-only segment metadata in the metadata node to be upgraded is loaded according to the first loading rate;

[0019] Receive a second loading instruction returned by the management node; wherein the second loading instruction is sent by the management node after determining that the read-only segment metadata has been loaded;

[0020] The read-write segment metadata in the metadata node to be upgraded is loaded at the second loading rate, and after the read-write segment metadata is loaded, a loading completion notification is returned to the management node; wherein the first loading rate is less than the second loading rate.

[0021] Furthermore, before receiving the second loading instruction returned by the management node, the method further includes:

[0022] Receive polling requests sent by the management node;

[0023] A loading status response is returned to the management node so that the management node can determine whether the read-only segment metadata has been loaded based on the loading status response; wherein, the loading status response is used to indicate the loading progress of the read-only segment metadata.

[0024] Further, loading the read-only segment metadata in the metadata node to be upgraded according to the first loading rate includes:

[0025] The loading quantity is calculated based on the remaining resources of the distributed cluster at the current moment; wherein, the loading quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load for the read-only segment metadata at the current moment;

[0026] The read-only segment metadata is loaded based on the stated loading quantity.

[0027] Further, loading the read-only segment metadata based on the number of loads includes:

[0028] If it is determined that the loading quantity is greater than a preset value, a loading request is generated based on the loading quantity and sent to the data node storing the described data in the distributed cluster; the described data is the data described by the read-only segment metadata, and the loading request is used to instruct the data node to send the described data to the target metadata node according to the loading quantity;

[0029] After the data node completes the loading request, it determines whether all the described data has been loaded.

[0030] If it is determined that the described data has not been fully loaded, the loading quantity is updated. If it is determined that the loading quantity after the update is greater than the preset value, the process returns to continue executing the process of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster until the described data is fully loaded.

[0031] Furthermore, the method also includes:

[0032] If it is determined that the number of loads is less than or equal to the preset value, the number of loads is recalculated after a preset time.

[0033] If it is determined that the recalculated loading quantity is greater than the preset value, the step of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster is executed; otherwise, the step of recalculating the loading quantity after a preset time is returned.

[0034] According to a third aspect of this disclosure, an upgrade device for a metadata node in a distributed cluster is provided, disposed on a management node, the device comprising:

[0035] The first sending module is configured to send a first loading instruction to the target metadata node in response to the upgrade instruction of the metadata node to be upgraded; the first loading instruction is configured to instruct the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a first loading rate;

[0036] The second sending module is used to determine whether the read-only segment metadata has been loaded, and if it is determined that the loading has been completed, to send a second loading instruction to the target metadata node; the second loading instruction is used to instruct the target metadata node to load the read-write segment metadata in the metadata node to be upgraded at a second loading rate, wherein the first loading rate is less than the second loading rate;

[0037] The upgrade module is used to upgrade the metadata node to be upgraded after receiving a notification that the loading of the read-write segment metadata is complete.

[0038] According to a fourth aspect of this disclosure, an upgrade device for a metadata node in a distributed cluster is provided, disposed at a target metadata node, the device comprising:

[0039] The first loading module is used to load read-only segment metadata in the metadata node to be upgraded according to a first loading rate in response to a first loading instruction sent by the management node.

[0040] The first receiving module is configured to receive a second loading instruction returned by the management node; wherein the second loading instruction is sent by the management node after determining that the read-only segment metadata has been loaded.

[0041] The second loading module is used to load the readable and writable segment metadata in the metadata node to be upgraded according to the second loading rate, and after the readable and writable segment metadata is loaded, return a loading completion notification to the management node; wherein, the first loading rate is less than the second loading rate.

[0042] According to a fifth aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0043] According to a sixth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0044] According to a seventh aspect of this disclosure, a computer program product is provided. The computer program product includes a computer program that, when executed by a processor, implements the methods described above.

[0045] This disclosure provides a method, apparatus, device, and medium for upgrading metadata nodes in a distributed cluster. In this embodiment, firstly, in response to an upgrade instruction from the metadata node to be upgraded, a first loading instruction is sent to the target metadata node. The first loading instruction instructs the target metadata node to load read-only segment metadata from the metadata node to be upgraded at a first loading rate. Then, it is determined whether the read-only segment metadata has been loaded completely, and if it is determined that loading is complete, a second loading instruction is sent to the target metadata node. The second loading instruction instructs the target metadata node to load read-write segment metadata from the metadata node to be upgraded at a second loading rate, where the first loading rate is less than the second loading rate. Finally, after receiving a notification that the read-write segment metadata loading is complete, the metadata node to be upgraded is upgraded.

[0046] As described above, the technical solution of this disclosure, after responding to the upgrade instruction of the metadata node to be upgraded, first sends a first loading instruction to the target metadata node, causing the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a lower first loading rate. This process avoids consuming too many resources, allowing the metadata node to be upgraded to still respond normally to the business of the computing node, thereby ensuring business continuity. After confirming that the read-only segment metadata has been loaded, a second loading instruction is sent, causing the target metadata node to load the read-write segment metadata at a faster second loading rate. Taking advantage of the short loading time of the read-write segment metadata, the impact on the business is further reduced. Finally, after the read-write segment metadata has been loaded, the metadata node to be upgraded is upgraded. Compared with the direct redirection of nodes in traditional technologies, the technical solution of this disclosure avoids the decline in business performance caused by not considering the node's carrying capacity, and can control the fluctuation of business performance to a minimum, realizing the continuous operation and stable performance of the business during the upgrade of the metadata node, thereby achieving a smooth upgrade without the user's perception. Attached Figure Description

[0047] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0048] Figure 1 A flowchart illustrating an exemplary embodiment of this disclosure of a method for upgrading metadata nodes in a distributed cluster;

[0049] Figure 2 A schematic diagram illustrating the storage information of metadata nodes provided in an exemplary embodiment of this disclosure;

[0050] Figure 3 A schematic diagram of a distributed cluster architecture provided for an exemplary embodiment of this disclosure;

[0051] Figure 4 A schematic diagram illustrating the upgrade process interaction provided in an exemplary embodiment of this disclosure;

[0052] Figure 5 A flowchart illustrating a method for upgrading metadata nodes in a distributed cluster, provided as another exemplary embodiment of this disclosure;

[0053] Figure 6 A flowchart illustrating a method for upgrading metadata nodes in a distributed cluster, provided as another exemplary embodiment of this disclosure;

[0054] Figure 7 A flowchart illustrating a method for upgrading metadata nodes in a distributed cluster, provided as another exemplary embodiment of this disclosure;

[0055] Figure 8 A schematic diagram of a read-only segment metadata loading process provided for an exemplary embodiment of this disclosure;

[0056] Figure 9 A schematic block diagram of the functional modules of an upgrade device for metadata nodes in a distributed cluster provided as an exemplary embodiment of this disclosure;

[0057] Figure 10 A schematic block diagram of the functional modules of an upgrade device for a metadata node in a distributed cluster provided as another exemplary embodiment of this disclosure;

[0058] Figure 11 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure;

[0059] Figure 12 A structural block diagram of a computer system provided as an exemplary embodiment of this disclosure;

[0060] Figure 13 A structural block diagram of a computer program product provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0061] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0062] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0063] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0064] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0065] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0066] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0067] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0068] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0069] In one embodiment, such as Figure 1 As shown, a method for upgrading metadata nodes in a distributed cluster is provided, which is applied to the management node in the distributed cluster, and includes the following steps:

[0070] Step 101: In response to the upgrade instruction of the metadata node to be upgraded, send the first loading instruction to the target metadata node.

[0071] Here, the management node in the distributed cluster can respond to the upgrade instruction of the metadata node to be upgraded by sending a first loading instruction to the target metadata node. The first loading instruction is used to instruct the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a first loading rate.

[0072] In one possible implementation, the metadata information to be stored by the metadata nodes in the distributed cluster is divided into multiple planes, and the management node evenly distributes these planes across the multiple metadata nodes, such as... Figure 2 As shown, Figure 2 An exemplary diagram illustrating the storage of metadata nodes is provided. Each plane consists of multiple segments. The last segment is in RW (Read-Write) state, meaning the last segment-n (RW) contains read-write segment metadata. The remaining segments are in RO (Read-Only) state, meaning they contain read-only segment metadata. Figure 3 As shown, Figure 3 An exemplary schematic diagram of a distributed cluster architecture is shown, in which metadata nodes 1-N are used to store and manage metadata of the distributed cluster, such as descriptive data such as file paths, permissions, and locations. Data nodes are the nodes that actually store business data. The management node distributes metadata information plane0-plane8 evenly to metadata nodes 1-N.

[0073] In one possible embodiment, such as Figure 4 As shown, Figure 4An exemplary diagram illustrating the upgrade process interaction is provided. When a metadata node storing metadata information needs to be upgraded, this metadata node is the metadata node to be upgraded. At this time, the management node in the distributed cluster responds to the upgrade instruction of the metadata node to be upgraded by sending a first loading instruction, preload, to the target metadata node. This first loading instruction, preload, instructs the target metadata node to load the RO segments (read-only segments) in the metadata node to be upgraded at a first loading rate. In other words, the first loading instruction, preload, contains a list of RO segments to be loaded and explicitly instructs the target metadata node to load the RO segments at a rate-limited manner according to the first loading rate. After receiving the first loading instruction, preload, the target metadata node loads the read-only segment metadata in the metadata node to be upgraded at a rate-limited manner according to the first loading rate. It should be noted that during the process of the target metadata node loading the RO segments at a rate-limited manner according to the first loading rate, the read and write IO services of the metadata node to be upgraded and the computing nodes in the distributed cluster are not affected. That is, when the computing node sends an IO read and write request to the metadata node to be upgraded, the metadata node to be upgraded can normally return an IO read and write response to the computing node.

[0074] Step 102: Determine whether the read-only segment metadata has been loaded, and if it is determined that the loading has been completed, send a second loading instruction to the target metadata node.

[0075] Here, after the management node sends a first loading instruction to the target metadata node in response to the upgrade instruction of the metadata node to be upgraded, it can determine whether the read-only segment metadata has been loaded. If it is determined that the loading has been completed, it sends a second loading instruction to the target metadata node. The second loading instruction is used to instruct the target metadata node to load the read-write segment metadata in the metadata node to be upgraded at a second loading rate, where the first loading rate is less than the second loading rate.

[0076] In one possible embodiment, such as Figure 5 As shown, determining whether the read-only segment metadata has been loaded completely includes the following steps:

[0077] Step 1021: Send polling requests to the target metadata node at preset time intervals.

[0078] Here, after the management node responds to the upgrade instruction of the metadata node to be upgraded and sends the first load instruction to the target metadata node, it can send polling requests to the target metadata node at preset time intervals.

[0079] In one possible embodiment, such as Figure 4As shown, after the management node responds to the upgrade instruction of the metadata node to be upgraded and sends the first loading instruction preload to the target metadata node, it sends polling requests to the target metadata node at preset time intervals. The polling requests are used to request the target metadata node to check the loading progress of the read-only segment metadata RO Segment.

[0080] Step 1022: Receive the loading status response returned by the target metadata node.

[0081] Here, after the management node sends polling requests to the target metadata node at preset time intervals, it can receive the loading status response returned by the target metadata node.

[0082] In one possible embodiment, such as Figure 4 As shown, after the management node sends a polling request to the target metadata node at a preset time interval, the target metadata node sends a load status response of the read-only segment metadata RO Segment to the management node. The management node receives the load status response returned by the target metadata node, wherein the load status response is used to indicate the loading progress of the read-only segment metadata by the target metadata node.

[0083] Step 1023: Determine whether the read-only segment metadata has been loaded based on the loading status response.

[0084] Here, after receiving the load status response from the target metadata node, the management node can determine whether the read-only segment metadata has been loaded based on the load status response.

[0085] In one possible embodiment, such as Figure 4 As shown, after receiving the load status response from the target metadata node, the management node determines the loading status of the read-only segment metadata (RO Segment) based on the load status response. Upon confirming that the RO Segment is loaded, the management node sends a second load instruction to the target metadata node. This second load instruction instructs the target metadata node to load the read-write segment metadata (RW Segment) from the metadata node to be upgraded at a second load rate. It should be noted that the first load rate is lower than the second load rate. The first load rate loads the RO Segment using a rate-limited method, while the second load rate loads the RW Segment using the node's maximum load rate. Because the RW Segment loading time is short, less than 1 second, the impact on IO services can be controlled within 1 second.

[0086] In this embodiment, firstly, the management node sends polling requests to the target metadata node at preset time intervals; wherein, the polling request is used to request the target metadata node to update the loading progress of the read-only segment metadata; then, the management node receives a loading status response returned by the target metadata node; wherein, the loading status response is used to indicate the loading progress of the target metadata node on the read-only segment metadata; finally, the management node determines whether the read-only segment metadata has been loaded based on the loading status response, and if it determines that the loading has been completed, sends a second loading instruction to the target metadata node.

[0087] As described above, in this embodiment, the management node sends polling requests to the target metadata node at preset time intervals to request the progress of loading read-only segment metadata. By receiving the loading status response of the read-only segment metadata loading progress returned by the target metadata node, the loading status of the read-only segment metadata can be accurately and timely grasped. When it is determined based on the loading status response that the read-only segment metadata loading is completed, a second loading instruction is sent to the target metadata node. This ensures that the target metadata node only starts the subsequent loading process after completing the preparatory work of loading read-only segment metadata, effectively avoiding resource waste or performance loss caused by disordered loading process, and ensuring the orderliness and stability of read-only segment metadata loading during the metadata node upgrade process.

[0088] Step 103: After receiving the notification that the loading of the read-write segment metadata is complete, upgrade the metadata node to be upgraded.

[0089] Here, after the management node sends the second load instruction to the target metadata node, it can upgrade the metadata node to be upgraded after receiving the notification that the loading of the read-write segment metadata is complete.

[0090] In one possible embodiment, after receiving a notification that the loading of the read-write segment metadata is complete, the metadata node to be upgraded is upgraded, including the following steps:

[0091] Upon receiving a notification that the loading of the read-write segment metadata is complete, the system triggers a switch of business traffic from the metadata node to be upgraded to the target metadata node and performs the upgrade operation on the metadata node to be upgraded.

[0092] Specifically, after receiving a notification that the loading of the read-write segment metadata is complete, the management node triggers the switching of business traffic from the metadata node to be upgraded to the target metadata node, and performs the upgrade operation on the metadata node to be upgraded.

[0093] In one possible embodiment, such as Figure 4As shown, after receiving the notification from the target metadata node that the read / write segment metadata has been loaded, the management node deletes the plane from the metadata node to be upgraded. At this point, if the metadata node to be upgraded receives a read / write IO from the compute node, it will return "not found" because the plane has been deleted. Upon receiving "not found", the compute node will obtain a new route and resend the read / write IO to the target metadata node. At this point, the compute node can perform normal IO operations with the target metadata node, thus completing the IO service migration. The metadata node to be upgraded then stops its service, and the management node can upgrade it.

[0094] This disclosure provides a method, apparatus, device, and medium for upgrading metadata nodes in a distributed cluster. In this embodiment, firstly, in response to an upgrade instruction from the metadata node to be upgraded, a first loading instruction is sent to the target metadata node. The first loading instruction instructs the target metadata node to load read-only segment metadata from the metadata node to be upgraded at a first loading rate. Then, it is determined whether the read-only segment metadata has been loaded completely, and if it is determined that loading is complete, a second loading instruction is sent to the target metadata node. The second loading instruction instructs the target metadata node to load read-write segment metadata from the metadata node to be upgraded at a second loading rate, where the first loading rate is less than the second loading rate. Finally, after receiving a notification that the read-write segment metadata loading is complete, the metadata node to be upgraded is upgraded.

[0095] As described above, the technical solution of this disclosure, after responding to the upgrade instruction of the metadata node to be upgraded, first sends a first loading instruction to the target metadata node, causing the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a lower first loading rate. This process avoids consuming too many resources, allowing the metadata node to be upgraded to still respond normally to the business of the computing node, thereby ensuring business continuity. After confirming that the read-only segment metadata has been loaded, a second loading instruction is sent, causing the target metadata node to load the read-write segment metadata at a faster second loading rate. Taking advantage of the short loading time of the read-write segment metadata, the impact on the business is further reduced. Finally, after the read-write segment metadata has been loaded, the metadata node to be upgraded is upgraded. Compared with the direct redirection of nodes in traditional technologies, the technical solution of this disclosure avoids the decline in business performance caused by not considering the node's carrying capacity, and can control the fluctuation of business performance to a minimum, realizing the continuous operation and stable performance of the business during the upgrade of the metadata node, thereby achieving a smooth upgrade without the user's perception.

[0096] In one embodiment, before sending the first load instruction to the target metadata node, the management node further includes the following steps:

[0097] The target metadata node is determined from multiple candidate metadata nodes based on business load metrics.

[0098] Specifically, before sending the first load instruction to the target metadata node, the management node can determine the target metadata node from multiple candidate metadata nodes based on business load indicators. The business load indicators include at least one of the following: input load, output load, amount of data already carried, and available resource capacity.

[0099] In one possible embodiment, for example, the management node selects a target node from candidate metadata nodes (node ​​1, node 2, and node 3). The management node first collects the service load indicators of each candidate metadata node. Node 1 has an input load of 700 IOPS, an output load of 600 IOPS, a data volume of 1100GB, and an available resource capacity of 900GB; Node 2 has an input load of 950 IOPS, an output load of 880 IOPS, a data volume of 1400GB, and an available resource capacity of 650GB; Node 3 has an input load of 580 IOPS, an output load of 520 IOPS, a data volume of 850GB, and an available resource capacity of 1100GB. The management node sets a selection strategy based on the upgrade requirements. This selection strategy can be that the node with the largest input load + output load ≤ 1300 IOPS, data volume of ≤ 1200GB, and available resource capacity is selected. After screening, Node 3 meets all the conditions, so the management node determines Node 3 as the target metadata node. This node is used to handle the IO services of the metadata node to be upgraded.

[0100] In one embodiment, such as Figure 6 As shown, a method for upgrading metadata nodes in a distributed cluster is provided, applied to the target metadata node in the distributed cluster, including the following steps:

[0101] Step 601: In response to the first loading instruction sent by the management node, load the read-only segment metadata in the metadata node to be upgraded according to the first loading rate.

[0102] Here, the target metadata node can respond to the first load instruction sent by the management node and load the read-only segment metadata in the metadata node to be upgraded at the first load rate.

[0103] In one possible embodiment, such as Figure 7 As shown, loading the read-only segment metadata in the metadata node to be upgraded according to the first loading rate includes the following steps:

[0104] Step 6011: Calculate the number of loads based on the remaining resources of the distributed cluster at the current moment.

[0105] Here, after receiving the first load instruction sent by the management node, the target metadata node can calculate the load quantity based on the remaining resources of the distributed cluster at the current moment. The load quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load for the read-only segment metadata at the current moment.

[0106] In one possible embodiment, such as Figure 8 As shown, Figure 8 An exemplary diagram of the read-only segment metadata loading process is shown. The target metadata node responds to the first loading instruction preload sent by the management node, collects the remaining resource status of the distributed cluster at the current moment, including the overall remaining CPU utilization, memory free amount, and remaining read / write bandwidth of the data nodes. The target metadata node combines the baseline data obtained from previous tests, which shows that "each concurrent loading of 100GB of read-only segment metadata RO Segment requires 10% CPU, 150GB memory, and 50MB / s data node bandwidth", to calculate that the number of data that can be loaded at the current moment is 400GB, that is, the maximum data token for concurrent loading of read-only segment metadata is 400G.

[0107] Step 6012: Load the read-only segment metadata based on the number of loads.

[0108] Here, the target metadata node calculates the loading quantity based on the remaining resources of the distributed cluster at the current moment, and can load the read-only segment metadata based on the loading quantity.

[0109] In one possible embodiment, loading read-only segment metadata based on the number of loads includes the following steps:

[0110] If it is determined that the number of data to be loaded is greater than the preset value, a loading request is generated based on the number of data to be loaded, and the loading request is sent to the data node in the distributed cluster that stores the data to be described.

[0111] After the data node completes the loading request, it determines whether all the described data has been loaded.

[0112] If it is determined that the described data has not been fully loaded, the loading quantity is updated. If it is determined that the loading quantity after the update is greater than the preset value, the process returns to continue executing the loading request based on the loading quantity and sends the loading request to the data node storing the described data in the distributed cluster until the described data is fully loaded.

[0113] Specifically, the target metadata node calculates the loading quantity based on the remaining resources of the distributed cluster at the current moment. If it determines that the loading quantity is greater than a preset value, it generates a loading request based on the loading quantity and sends the loading request to the data node storing the described data in the distributed cluster. After the data node completes the execution of the loading request, the target metadata node determines whether the described data has been fully loaded. If it determines that the described data has not been fully loaded, it updates the loading quantity. If it determines that the updated loading quantity is greater than the preset value, it returns to continue executing the process of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster, until the described data is fully loaded.

[0114] In one possible embodiment, such as Figure 8 As shown, after the target metadata node calculates the loading quantity token based on the remaining resources of the distributed cluster at the current moment, if it determines that the loading quantity token is greater than a preset value, the target metadata node generates a loading request based on the loading quantity token and sends the loading request to the data node storing the described data in the distributed cluster. Here, the described data is the data described by the read-only segment metadata RO Segment. The loading request is used to instruct the data node to send the described data to the target metadata node according to the loading quantity, that is, the target metadata node sends a loading request of the token size to the data node, and the data node sends the described data to the target metadata node according to the loading quantity token. After the data node completes the execution of the loading request, the target metadata node determines whether the described data has been fully loaded. If it determines that the described data has not been fully loaded, it updates the loading quantity token. If it determines that the updated loading quantity is greater than the preset value, it returns to continue executing the process of generating loading requests based on the loading quantity token and sending loading requests to the data node storing the described data in the distributed cluster, until the described data is fully loaded. When the described data is fully loaded, it means that the read-only segment metadata RO Segment has been fully loaded.

[0115] In one possible embodiment, loading read-only segment metadata based on the number of loads further includes the following steps:

[0116] If the number of items loaded is determined to be less than or equal to the preset value, the number of items loaded will be recalculated after the preset time.

[0117] If the recalculated load quantity is greater than the preset value, the process proceeds to generate a load request based on the load quantity and send the load request to the data node storing the described data in the distributed cluster; otherwise, the process returns to recalculating the load quantity after a preset time.

[0118] Specifically, after the target metadata node calculates the loading quantity based on the remaining resources of the distributed cluster at the current moment, if it determines that the loading quantity is less than or equal to a preset value, it recalculates the loading quantity after a preset time. If it determines that the recalculated loading quantity is greater than the preset value, it executes the step of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster; otherwise, it returns to the step of recalculating the loading quantity after a preset time.

[0119] In one possible embodiment, such as Figure 8 As shown, after the target metadata node calculates the loading quantity token based on the remaining resources of the distributed cluster at the current moment, if it determines that the loading quantity token is less than or equal to a preset value, the target metadata node recalculates the loading quantity token after a preset time. If it determines that the recalculated loading quantity token is greater than the preset value, it executes the step of generating a loading request based on the loading quantity token and sending the loading request to the data node storing the described data in the distributed cluster. Otherwise, it returns to the step of recalculating the loading quantity token after a preset time until the loading quantity token is greater than the preset value.

[0120] As described above, the loading process in this embodiment allows the target metadata node to dynamically adjust its loading pace based on the real-time remaining resources of the distributed cluster. When resources are sufficient, i.e., the number of data to be loaded is greater than a preset value, a loading request can be initiated efficiently and data loading can be continuously advanced until completion. When resources are scarce, i.e., the number of data to be loaded is less than or equal to a preset value, the loading quantity is recalculated at a preset time interval to avoid forcibly loading and occupying resources when resources are insufficient. This effectively avoids interference with the normal business I / O of the cluster during the loading process, ensuring stable business performance, while also ensuring that the data is ultimately loaded completely. This not only meets the performance lossless requirements of cloud storage metadata node upgrade scenarios, but also lays the foundation for smooth business migration and business continuity during metadata node upgrades, making the entire loading process both efficient and with low impact on business.

[0121] Step 602: Receive the second loading instruction returned by the management node.

[0122] Here, after the target metadata node loads the read-only segment metadata in the metadata node to be upgraded at the first loading rate in response to the first loading instruction sent by the management node, it can receive the second loading instruction returned by the management node.

[0123] In one possible embodiment, before receiving the second load instruction returned by the management node, the following steps are also included:

[0124] Receive polling requests sent by the management node;

[0125] Return a load status response to the management node so that the management node can determine whether the read-only segment metadata has been loaded based on the load status response.

[0126] Specifically, first, the target metadata node receives a polling request sent by the management node; then, the target metadata node returns a load status response to the management node, so that the management node can determine whether the read-only segment metadata has been loaded based on the load status response.

[0127] In one possible embodiment, such as Figure 4 As shown, after the target metadata node loads the read-only segment metadata RO segment through preload rate-limited loading, it receives a polling request sent by the management node. The polling request is used to request the target metadata node to check the loading progress of the read-only segment metadata RO segment. The target metadata node sends a loading status response of the read-only segment metadata RO segment to the management node. After receiving the loading status response returned by the target metadata node, the management node determines the loading status of the read-only segment metadata RO segment based on the loading status response. When the management node determines that the read-only segment metadata RO segment has been loaded, it sends a second loading instruction to the target metadata node, and the target metadata node receives the second loading instruction returned by the management node.

[0128] Step 603: Load the readable and writable segment metadata in the metadata node to be upgraded according to the second loading rate, and after the readable and writable segment metadata is loaded, return a loading completion notification to the management node.

[0129] Here, after receiving the second loading instruction returned by the management node, the target metadata node loads the readable and writable segment metadata in the metadata node to be upgraded according to the second loading rate, and after the readable and writable segment metadata is loaded, it returns a loading completion notification to the management node. The first loading rate is less than the second loading rate.

[0130] In one possible embodiment, such as Figure 4As shown, the management node sends a second loading instruction to the target metadata node. This second loading instruction instructs the target metadata node to load the read-write segment metadata (RW Segment) from the metadata node to be upgraded at a second loading rate. The first loading rate is lower than the second loading rate. The first loading rate loads the read-only segment metadata (RO Segment) using a rate-limiting method, while the second loading rate loads the read-write segment metadata (RW Segment) at the node's maximum loading rate. Because the RW Segment loading time is short, it can be completed within 1 second, so the impact on IO services can be controlled within 1 second. After the target metadata node completes loading the read-write segment metadata (RWS Segment), it returns a loading completion notification to the management node, indicating that the target metadata node has completed loading the read-write segment metadata (RW Segment).

[0131] In this embodiment, firstly, the target metadata node responds to the first loading instruction sent by the management node and loads the read-only segment metadata in the metadata node to be upgraded at a first loading rate; then, the target metadata node receives the second loading instruction returned by the management node; finally, the target metadata node loads the read-write segment metadata in the metadata node to be upgraded at a second loading rate, and after the read-write segment metadata is loaded, returns a loading completion notification to the management node.

[0132] As described above, in this embodiment, when the target metadata node loads the read-only segment metadata at a lower first loading rate, it does not consume excessive resources, ensuring the normal operation of the cluster. After the read-only segment metadata is loaded, the target metadata node loads the read-write segment metadata at a higher second loading rate, which can quickly complete the remaining loading work and shorten the overall loading time. At the same time, the target metadata node loads in stages and reports the completion status to the management node, allowing the management node to clearly grasp the loading progress and prepare for subsequent operations such as business switching of nodes to be upgraded. Ultimately, the metadata node upgrade process achieves uninterrupted business and stable performance, allowing customers to upgrade the nodes without any noticeable impact.

[0133] As described above, the technical solution disclosed herein effectively controls the impact of metadata loading on the performance of normal IO services by introducing a preload rate-limiting loading mechanism during the upgrade process. At the same time, it utilizes the characteristic of computing nodes to actively perceive routing changes to achieve smooth migration of business flows. The technical solution disclosed herein not only ensures the continuity and performance stability of IO services during the upgrade process, but also supports flexible target metadata node selection strategies, significantly improving the high availability and user experience of the distributed storage system during maintenance and upgrades, and possesses significant commercial application value and technological advancement.

[0134] By dividing each functional module according to its corresponding function, this disclosure provides an upgrade device for metadata nodes in a distributed cluster, which is applied to the management node in the distributed cluster. The upgrade device for metadata nodes in the distributed cluster can be a server or a chip applied to the server. Figure 9 This is a schematic block diagram illustrating the functional modules of an upgrade apparatus for metadata nodes in a distributed cluster, provided as an exemplary embodiment of this disclosure. Figure 9 As shown, the upgrade device for the metadata node in this distributed cluster includes:

[0135] The first sending module 901 is used to send a first loading instruction to the target metadata node in response to the upgrade instruction of the metadata node to be upgraded; the first loading instruction is used to instruct the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a first loading rate;

[0136] The second sending module 902 is used to determine whether the read-only segment metadata has been loaded, and if it is determined that the loading has been completed, to send a second loading instruction to the target metadata node; the second loading instruction is used to instruct the target metadata node to load the read-write segment metadata in the metadata node to be upgraded at a second loading rate, wherein the first loading rate is less than the second loading rate.

[0137] The upgrade module 903 is used to upgrade the metadata node to be upgraded after receiving a notification that the loading of the read-write segment metadata is complete.

[0138] In one embodiment, the second transmitting module 902 includes:

[0139] The first sending unit is configured to send polling requests to the target metadata node at preset time intervals; wherein the polling requests are used to request the target metadata node to update the loading progress of the read-only segment metadata.

[0140] A receiving unit is configured to receive a loading status response returned by the target metadata node; wherein the loading status response is used to indicate the loading progress of the target metadata node on the read-only segment metadata;

[0141] The determining unit is used to determine whether the read-only segment metadata has been loaded completely based on the loading status response.

[0142] In one embodiment, the upgrade module 903 includes:

[0143] The upgrade unit is used to trigger the switching of service traffic from the metadata node to be upgraded to the target metadata node after receiving a notification that the loading of the read-write segment metadata is complete, and to perform an upgrade operation on the metadata node to be upgraded.

[0144] In one embodiment, the apparatus further includes:

[0145] The determination module is used to determine the target metadata node from multiple candidate metadata nodes based on business load indicators; wherein the business load indicators include at least one of the following: input load, output load, amount of data already carried, and available resource capacity.

[0146] By dividing each functional module according to its corresponding function, this disclosure provides an upgrade device for metadata nodes in a distributed cluster, which is applied to a target metadata node in the distributed cluster. The upgrade device for metadata nodes in the distributed cluster can be a server or a chip applied to a server. Figure 10 This is a schematic block diagram illustrating the functional modules of an upgrade apparatus for metadata nodes in a distributed cluster, provided as an exemplary embodiment of this disclosure. Figure 10 As shown, the upgrade device for the metadata node in this distributed cluster includes:

[0147] The first loading module 1001 is used to load read-only segment metadata in the metadata node to be upgraded according to the first loading rate in response to the first loading instruction sent by the management node.

[0148] The first receiving module 1002 is used to receive the second loading instruction returned by the management node; wherein the second loading instruction is sent by the management node after determining that the read-only segment metadata has been loaded.

[0149] The second loading module 1003 is used to load the readable and writable segment metadata in the metadata node to be upgraded according to the second loading rate, and after the loading of the readable and writable segment metadata is completed, return a loading completion notification to the management node; wherein, the first loading rate is less than the second loading rate.

[0150] In one embodiment, the first loading module 1001 includes:

[0151] The first computing unit is used to calculate the loading quantity based on the remaining resources of the distributed cluster at the current moment; wherein, the loading quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load on the read-only segment metadata at the current moment;

[0152] A loading unit is used to load the read-only segment metadata based on the specified loading quantity.

[0153] In one embodiment, the first loading module 1001 includes:

[0154] The second sending unit is configured to generate a loading request based on the loading quantity if it is determined that the loading quantity is greater than a preset value, and send the loading request to the data node storing the described data in the distributed cluster; the described data is the data described by the read-only segment metadata, and the loading request is used to instruct the data node to send the described data to the target metadata node according to the loading quantity;

[0155] The judgment unit is used to determine whether the described data has been fully loaded after the data node has completed the execution of the loading request;

[0156] An update unit is configured to update the loading quantity when it is determined that the described data has not been fully loaded, and if it is determined that the loading quantity after the update is greater than the preset value, return to continue executing the process of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster until the described data is fully loaded.

[0157] In one embodiment, the first loading module 1001 includes:

[0158] The second calculation unit is used to recalculate the loading quantity after a preset time if it is determined that the loading quantity is less than or equal to the preset value.

[0159] An execution unit is configured to, if it is determined that the recalculated loading quantity is greater than the preset value, execute the step of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster; otherwise, return to the step of recalculating the loading quantity after a preset time.

[0160] In one embodiment, the apparatus further includes:

[0161] The second receiving module is used to receive the polling request sent by the management node;

[0162] The return module is used to return a loading status response to the management node, so that the management node can determine whether the read-only segment metadata has been loaded based on the loading status response; wherein, the loading status response is used to indicate the loading progress of the read-only segment metadata.

[0163] Figure 11 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 11 As shown, the electronic device 1100 includes at least one processor 1101 and a memory 1102 coupled to the processor 1101. The processor 1101 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0164] The processor 1101 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1101 or by software instructions. The processor 1101 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1102, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1101 reads information from the memory 1102 and, in conjunction with its hardware, completes the steps of the method described above.

[0165] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 12 The computer system 1200 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including functions such as those described above. Figure 12 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0166] Computer system 1200 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0167] like Figure 12As shown, the computer system 1200 includes a computing unit 1201, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the computer system 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0168] Multiple components in the computer system 1200 are connected to the I / O interface 1205, including: an input unit 1206, an output unit 1207, a storage unit 1208, and a communication unit 1209. The input unit 1206 can be any type of device capable of inputting information into the computer system 1200. The input unit 1206 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1207 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 1208 may include, but is not limited to, a hard disk and an optical disk. The communication unit 1209 allows the computer system 1200 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, a modem, network card, infrared communication device, wireless communication transceiver, and / or chipset, such as Bluetooth™ device, WiFi device, WiMax device, cellular communication device, and / or the like.

[0169] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via ROM 1202 and / or communication unit 1209. In some embodiments, the computing unit 1201 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0170] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0171] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0172] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0173] Figure 13 A computer program product 1300 is provided as an exemplary embodiment of the present disclosure. The computer program product 1300 includes a computer program 1301, wherein the computer program 1301, when executed by a processor, implements the methods disclosed in the embodiments of the present disclosure.

[0174] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0176] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0177] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0178] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0179] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for upgrading metadata nodes in a distributed cluster, characterized in that, The management node applied in the distributed cluster includes: In response to an upgrade instruction for the metadata node to be upgraded, a first loading instruction is sent to the target metadata node; the first loading instruction is used to instruct the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a first loading rate; Determine whether the read-only segment metadata has been loaded completely, and if it is determined that the loading has been completed, send a second loading instruction to the target metadata node; the second loading instruction is used to instruct the target metadata node to load the read-write segment metadata in the metadata node to be upgraded at a second loading rate, wherein the first loading rate is less than the second loading rate; After receiving a notification that the loading of the read-write segment metadata is complete, the metadata node to be upgraded is upgraded. The target metadata node loads the read-only segment metadata from the metadata node to be upgraded at a first loading rate, including: The loading quantity is calculated based on the remaining resources of the distributed cluster at the current moment; wherein, the loading quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load for the read-only segment metadata at the current moment; The read-only segment metadata is loaded based on the stated loading quantity; The target metadata node loads the readable and writable segment metadata from the metadata node to be upgraded according to the second loading rate, including: The read / write segment metadata is loaded using the maximum loading rate of the metadata node to be upgraded; Determining whether the read-only segment metadata has been fully loaded includes: Polling requests are sent to the target metadata node at preset time intervals; wherein, the polling requests are used to query the loading progress of the target metadata node on the read-only segment metadata; Receive a loading status response returned by the target metadata node; wherein the loading status response is used to indicate the loading progress of the target metadata node on the read-only segment metadata; Based on the loading status response, determine whether the read-only segment metadata has been loaded completely.

2. The method according to claim 1, characterized in that, The step of upgrading the metadata node to be upgraded after receiving the notification that the loading of the read-write segment metadata is complete includes: Upon receiving a notification that the loading of the read-write segment metadata is complete, the service traffic is switched from the metadata node to be upgraded to the target metadata node, and an upgrade operation is performed on the metadata node to be upgraded.

3. The method according to claim 1, characterized in that, Before sending the first load instruction to the target metadata node, the method further includes: The target metadata node is determined from multiple candidate metadata nodes based on business load metrics; wherein the business load metrics include at least one of the following: input load, output load, amount of data already carried, and available resource capacity.

4. A method for upgrading metadata nodes in a distributed cluster, characterized in that, The target metadata nodes applied in the distributed cluster include: In response to the first load command sent by the management node, the read-only segment metadata in the metadata node to be upgraded is loaded according to the first load rate; Receive a second loading instruction returned by the management node; wherein the second loading instruction is sent by the management node after determining that the read-only segment metadata has been loaded; The read-write segment metadata in the metadata node to be upgraded is loaded at the second loading rate, and after the read-write segment metadata is loaded, a loading completion notification is returned to the management node; wherein, the first loading rate is less than the second loading rate; The step of loading the read-only segment metadata in the metadata node to be upgraded according to the first loading rate includes: The loading quantity is calculated based on the remaining resources of the distributed cluster at the current moment; wherein, the loading quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load for the read-only segment metadata at the current moment; The read-only segment metadata is loaded based on the stated loading quantity; The step of loading the readable and writable segment metadata in the metadata node to be upgraded according to the second loading rate includes: The read / write segment metadata is loaded using the maximum loading rate of the metadata node to be upgraded.

5. The method according to claim 4, characterized in that, Before receiving the second load instruction returned by the management node, the method further includes: Receive polling requests sent by the management node; A loading status response is returned to the management node so that the management node can determine whether the read-only segment metadata has been loaded based on the loading status response; wherein, the loading status response is used to indicate the loading progress of the read-only segment metadata.

6. The method according to claim 4, characterized in that, The loading of the read-only segment metadata based on the specified loading quantity includes: If it is determined that the loading quantity is greater than a preset value, a loading request is generated based on the loading quantity and sent to the data node storing the described data in the distributed cluster; the described data is the data described by the read-only segment metadata, and the loading request is used to instruct the data node to send the described data to the target metadata node according to the loading quantity; After the data node completes the loading request, it determines whether all the described data has been loaded. If it is determined that the described data has not been fully loaded, the loading quantity is updated. If it is determined that the loading quantity after the update is greater than the preset value, the process returns to continue executing the process of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster until the described data is fully loaded.

7. The method according to claim 6, characterized in that, The method further includes: If it is determined that the number of loads is less than or equal to the preset value, the number of loads is recalculated after a preset time. If it is determined that the recalculated loading quantity is greater than the preset value, the step of generating a loading request based on the loading quantity and sending the loading request to the data node storing the described data in the distributed cluster is executed; otherwise, the step of recalculating the loading quantity after a preset time is returned.

8. An upgrade device for metadata nodes in a distributed cluster, characterized in that, Configured on the management node, including: The first sending module is configured to send a first loading instruction to the target metadata node in response to the upgrade instruction of the metadata node to be upgraded; the first loading instruction is configured to instruct the target metadata node to load the read-only segment metadata in the metadata node to be upgraded at a first loading rate; The second sending module is used to determine whether the read-only segment metadata has been loaded, and if it is determined that the loading has been completed, to send a second loading instruction to the target metadata node; the second loading instruction is used to instruct the target metadata node to load the read-write segment metadata in the metadata node to be upgraded at a second loading rate, wherein the first loading rate is less than the second loading rate; The upgrade module is used to upgrade the metadata node to be upgraded after receiving a notification that the loading of the read-write segment metadata is complete; The target metadata node loads the read-only segment metadata from the metadata node to be upgraded at a first loading rate, including: The loading quantity is calculated based on the remaining resources of the distributed cluster at the current moment; wherein, the loading quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load for the read-only segment metadata at the current moment; The read-only segment metadata is loaded based on the stated loading quantity; The target metadata node loads the readable and writable segment metadata from the metadata node to be upgraded according to the second loading rate, including: The read / write segment metadata is loaded using the maximum loading rate of the metadata node to be upgraded; The second sending module is further configured to: Polling requests are sent to the target metadata node at preset time intervals; wherein, the polling requests are used to query the loading progress of the target metadata node on the read-only segment metadata; Receive a loading status response returned by the target metadata node; wherein the loading status response is used to indicate the loading progress of the target metadata node on the read-only segment metadata; Based on the loading status response, determine whether the read-only segment metadata has been loaded completely.

9. An upgrade device for metadata nodes in a distributed cluster, characterized in that, Configured in the target metadata node, including: The first loading module is used to load the read-only segment metadata in the metadata node to be upgraded according to the first loading rate in response to the first loading instruction sent by the management node. The first receiving module is configured to receive a second loading instruction returned by the management node; wherein the second loading instruction is sent by the management node after determining that the read-only segment metadata has been loaded. The second loading module is used to load the readable and writable segment metadata in the metadata node to be upgraded according to the second loading rate, and after the loading of the readable and writable segment metadata is completed, return a loading completion notification to the management node; wherein, the first loading rate is less than the second loading rate; The first loading module is further configured to: The loading quantity is calculated based on the remaining resources of the distributed cluster at the current moment; wherein, the loading quantity is used to indicate the maximum amount of data that the target metadata node can concurrently load for the read-only segment metadata at the current moment; The read-only segment metadata is loaded based on the stated loading quantity; The first receiving module is further configured to: The read / write segment metadata is loaded using the maximum loading rate of the metadata node to be upgraded.

10. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-3, and / or to implement the method as described in any one of claims 4-7.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1-3, and / or to perform the method as described in any one of claims 4-7.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3, and / or the method as described in any one of claims 4-7.

Images