Metadata service management method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of distributed file storage, and provides a metadata service management method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a target server from a server set when the current activation number of metadata services in a cluster is smaller than a target activation number; carrying out availability check on each metadata service which is currently in a backup state on the target server, if available metadata services exist, carrying out activation, and updating the current activation quantity; and if the updated current activation number is smaller than the target activation number, continuing to determine a new target server from the server set until the updated current activation number is equal to the target activation number. According to the method, the server with the largest backup number of the currently owned metadata services is selected as the target server every time, and the available metadata services on the target server are activated, so that the load balance among the servers can be realized, and the metadata service capability with higher performance is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed file storage, and in particular to a method, device, electronic device, and storage medium for managing metadata services. Background Art

[0002] In a file system architecture, metadata access and data access together constitute the core components of system access. As the key information describing the characteristics of file system data, metadata includes file paths, permissions, sizes, creation times, etc., and its access efficiency directly affects the performance of the entire file system. According to statistics, metadata access accounts for more than 40% of the overall file system access, highlighting its important position in system performance optimization. Since metadata access usually presents a random access mode of small I / O (Input / Output), the effects of traditional storage access optimization means are limited, and caching technology, with its characteristic of being able to quickly respond to frequently accessed metadata requests, has become an effective way to improve metadata access performance.

[0003] In a distributed file system, an MDS (Metadata Server) is usually used to provide metadata access services. The traditional MDS configuration adopts a master-slave architecture of "primary-backup", and only one metadata service (hereinafter referred to as "MDS service") is carried on one server. The MDS service has two roles: active (activated) and slave (backup). Among them, the active MDS service can provide metadata request services, while the slave MDS service serves as a real-time backup of the active MDS service and takes over when the active MDS service fails to ensure the high availability of the system.

[0004] However, in order to ensure the reliability of the system, a certain number of slave MDS services usually need to be reserved in the cluster, which results in the number of active MDS services actually providing services being always less than the total number of servers in the cluster, thus limiting the overall metadata processing capacity of the cluster to a certain extent. Summary of the Invention

[0005] The present invention provides a method, device, electronic device, and storage medium for managing metadata services to solve the defect that the metadata processing capacity of the cluster is limited due to a single server carrying a single metadata service in the related art.

[0006] The present invention provides a method for managing metadata services, including: Based on the received metadata service activation request, determining the target activation number of the metadata service and obtaining the current activation number of the metadata service in the cluster; When the current activation quantity is less than the target activation quantity, determine a target server from the server set, where the target server is the server with the largest number of metadata services in the backup state among the currently owned servers; Check the availability of each metadata service currently in the backup state on the target server. If an available metadata service is detected among the metadata services, activate the available metadata service and update the current activation quantity; When the updated current activation quantity is less than the target activation quantity, continue to determine a new target server from the server set and activate the available metadata services on the new target server until the updated current activation quantity is equal to the target activation quantity.

[0007] According to a metadata service management method provided by the present invention, the determining of the target server from the server set includes: Obtain the current backup quantity of the metadata services in the cluster; When the current backup quantity is greater than the preset backup quantity, determine a target server from the server set; When the current backup quantity is less than or equal to the preset backup quantity, determine that the metadata service activation request fails.

[0008] According to a metadata service management method provided by the present invention, each metadata service currently in the backup state on the target server is obtained based on the backup service set of the target server. Correspondingly, the method further includes a metadata service startup process, and the metadata service startup process includes: Start a metadata service on the first server, set the metadata service to the backup state, and update the current backup quantity; When the first server is a newly added server, add the metadata service information of the first server to the server set and add the metadata service to the backup service set of the first server; When the first server is not a newly added server, add the metadata service to the backup service set of the first server.

[0009] According to a metadata service management method provided by the present invention, the determining of the target server from the server set includes: Sort based on the number of metadata services currently in the backup state owned by each server in the server set, and according to the sorting result, determine a candidate server from the servers, where the candidate server is the server with the largest number of metadata services in the backup state among the currently owned servers; In the case where there is one candidate server, use the candidate server as the target server; In the case where there are multiple candidate servers, determine the target server from the candidate servers based on the number of metadata services that are currently in the active state on each candidate server.

[0010] According to a metadata service management method provided by the present invention, the availability check for each metadata service that is currently in the backup state on the target server includes: Obtain the current load of the target server. If the current load exceeds a preset threshold, determine that there is no available metadata service on the target server; If the current load does not exceed the preset threshold, perform an availability check on each metadata service based on the startup order of each metadata service that is currently in the backup state on the target server.

[0011] According to a metadata service management method provided by the present invention, the availability check for each metadata service includes: Obtain the specified state and activation setting of any metadata service; In the case where the specified state is unspecified and the activation setting is activatable, determine that the any metadata service is an available metadata service.

[0012] According to a metadata service management method provided by the present invention, after performing an availability check on each metadata service that is currently in the backup state on the target server, it further includes: If it is checked that there is no available metadata service among the metadata services, sort based on the number of metadata services that are currently in the backup state owned by each server in the server set, and according to the sorting result, determine the next server of the target server from the servers; Use the next server as the new target server, and return to the step of performing an availability check on each metadata service that is currently in the backup state on the target server.

[0013] According to a metadata service management method provided by the present invention, the number of metadata services started on any server is determined based on the hardware resources of the any server and the hardware resources required by the metadata service.

[0014] The present invention also provides a metadata service management device, including: An acquisition unit, configured to determine the target activation number of a metadata service based on a received metadata service activation request, and acquire the current activation number of the metadata service in the cluster; A determination unit, configured to determine a target server from a set of servers when the current activation quantity is less than the target activation quantity, where the target server is the server with the largest number of metadata services in the backup state among the currently owned servers; An activation unit, configured to perform an availability check on each metadata service in the backup state on the target server. If an available metadata service is detected among the metadata services, activate the available metadata service and update the current activation quantity; A loop unit, configured to, when the updated current activation quantity is less than the target activation quantity, continue to determine a new target server from the set of servers and activate the available metadata services on the new target server until the updated current activation quantity is equal to the target activation quantity.

[0015] The present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the computer program, the metadata service management method described in any one of the above is implemented.

[0016] The present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the metadata service management method described in any one of the above is implemented.

[0017] The present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the metadata service management method described in any one of the above is implemented.

[0018] The metadata service management method, device, electronic device, and storage medium provided by the present invention can determine the target activation quantity of the metadata service by receiving a metadata service activation request. According to the difference between the current activation quantity and the target activation quantity, the system can dynamically adjust the quantity of the metadata services in the activated state in the cluster, realizing on-demand resource allocation. When activating each metadata service, the system will select the server with the largest quantity of the metadata services in the backup state currently owned as the target server, so as to activate the available metadata services on the target server. This way of selecting the target server can not only maximize the utilization of backup resources and reduce the overhead of starting new services, but also ensure that each time a metadata service is activated, it is from the server with a relatively large quantity of the metadata services in the backup state currently, that is, the server with a relatively light current load. Thus, load balancing between servers can be achieved, avoiding the situation where some servers are overloaded while other servers are idle. In addition, before activating the metadata service on the target server, an availability check will be performed on it to ensure that the activated metadata service is available, improving the overall reliability of the metadata service. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions in the present invention or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a schematic diagram of metadata service management in related technologies; Figure 2 is one of the schematic flowcharts of the metadata service startup provided by the present invention; Figure 3 is one of the schematic flowcharts of the metadata service activation provided by the present invention; Figure 4 is one of the schematic diagrams of the metadata service startup and activation provided by the present invention; Figure 5 is another schematic diagram of the metadata service startup and activation provided by the present invention; Figure 6 is yet another schematic diagram of the metadata service startup and activation provided by the present invention; Figure 7 is still another schematic diagram of the metadata service startup and activation provided by the present invention; Figure 8 is the schematic flowchart of the metadata service management method provided by the present invention; Figure 9It is the fifth schematic diagram of the startup and activation of the metadata service provided by the present invention; Figure 10 It is the sixth schematic diagram of the startup and activation of the metadata service provided by the present invention; Figure 11 It is the seventh schematic diagram of the startup and activation of the metadata service provided by the present invention; Figure 12 It is the eighth schematic diagram of the startup and activation of the metadata service provided by the present invention; Figure 13 It is the second schematic diagram of the process of starting the metadata service provided by the present invention; Figure 14 It is the second schematic diagram of the process of activating the metadata service provided by the present invention; Figure 15 It is the schematic diagram of the structure of the metadata service management device provided by the present invention; Figure 16 It is the schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0021] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] In the field of distributed file systems, as a component specifically responsible for metadata management, MDS undertakes key tasks such as caching, distributing, and maintaining the consistency of metadata. The traditional MDS configuration adopts a master-slave architecture of "primary-backup", and only one MDS service is carried on one physical server. Here, the MDS service actually refers to the MDS instance, that is, the MDS software unit actually running on the physical server. Each MDS instance has an independent metadata management function, including caching, distributing, and maintaining consistency, etc., for providing metadata access functions.

[0023] Figure 1 It is the schematic diagram of metadata service management in the related art, such as Figure 1As shown in the figure, there are three servers in the cluster (i.e., Server 1 to Server 3), and an MDS service is started on each server. The cluster will number each MDS service in the order of startup of the MDS service, starting from 0. The MDS service has two roles (also known as two states), active and slave. Among them, the active MDS service (hereinafter referred to as "active MDS") can provide metadata request services; the slave MDS service (hereinafter referred to as "slave MDS") does not provide metadata request services but serves as a real-time backup of the active MDS. When the active MDS fails, the slave MDS will replace the corresponding active MDS to provide metadata request services. Each MDS service is in the slave state by default after startup.

[0024] The storage administrator of the file system can adjust the number of active MDSs through the parameter max_mds. The cluster will change the corresponding MDS services from the slave state to the active state in ascending order according to the value of max_mds. As Figure 1 shown in the figure, green in the figure indicates that the MDS service is in the slave state, and blue indicates that the MDS service is in the active state. When the administrator sets max_mds = 2, the cluster will set MDS0 and MDS1 to the active state in sequence according to the numbering order, so that these two MDS services can provide metadata request services, that is, MDS0 and MDS1 can receive metadata access requests initiated by users or applications and return the obtained corresponding metadata to users or applications. The MDS service numbered 2 (i.e., MDS2) remains in the default slave state for backup use.

[0025] Obviously, in the traditional MDS configuration architecture, the number of active MDSs is limited by the total number of servers in the cluster and the reliability configuration requirements, resulting in difficulty in linearly expanding the system's metadata processing capacity by simply adding servers. With the exponential growth of data scale and access volume, the load pressure on a single server continues to climb, becoming the key bottleneck restricting the improvement of system performance. In addition, in scenarios where the metadata performance requirements surge, the traditional architecture needs to purchase more physical servers to increase the number of active MDSs. However, due to reliability constraints, only some of the newly added servers can actually provide services, resulting in low utilization of hardware resources and a significant increase in the overall system deployment and operation and maintenance costs.

[0026] Currently, mainstream storage servers are configured with CPUs (Central Processing Units) with more than 48 cores and memories greater than 256GB. Usually, an MDS service only requires a 4-core CPU and about 48GB of memory to provide services externally. Only in scenarios where the client load is extremely high or during MDS failover will higher memory resources be needed, and the MDS service in the slave state hardly occupies server resources. Therefore, the mode of purchasing a single server to host only one MDS service will cause idle server resources.

[0027] If a single server is used to host multiple MDS services, the above problems can be well solved. However, directly increasing the number of MDS services on the server will bring certain defects. To better understand the defects brought, first, the processes of starting a new MDS service and activating an MDS service are introduced.

[0028] Figure 2 It is one of the schematic diagrams of the process for starting a metadata service provided by the present invention. As Figure 2 shown, the process of starting an MDS service on a server is relatively simple. The cluster maintains a standby_daemons array for storing relevant information (such as numbers) of all MDS services in the slave state in the cluster. When a new MDS service is started on the server, the cluster will set this MDS service to the slave state and add it to the end of the standby_daemons array.

[0029] Figure 3 It is one of the schematic diagrams of the process for activating a metadata service provided by the present invention. As Figure 3 shown, if the capacity of the metadata services currently provided by the cluster is insufficient, the storage administrator can use the max_mds parameter to increase the number of MDS services to be activated. When the system receives an MDS service activation request initiated by the administrator, it can parse this request to obtain the value of the max_mds parameter and determine whether the current activation number active_mds of MDS services in the cluster is less than max_mds. If it is less, it enters the MDS service activation process. Here, active_mds represents the total number of currently active MDS services in the cluster.

[0030] After entering the MDS service activation process, it first checks whether the number of MDS services currently in the backup state in the cluster (i.e., the current backup service number) is greater than the minimum redundant backup number required for the cluster to be stable. If it is less, it indicates that the number of MDS services currently in the backup state in the cluster is insufficient and the MDS service activation cannot be performed. Therefore, the request to activate the MDS service fails. If it is greater, the MDS service that joined the array earliest can be selected from the stanby_daemons array in the order of MDS service addition, set this service to the active state, and delete it from the stanby_daemons array. Since a new MDS service is set to the active state, the number of active MDS services active_mds in the cluster will increase. At this time, the sizes of active_mds and max_mds can be compared again. If active_mds is still less than max_mds, it indicates that the number of active MDS services in the current cluster still does not meet the requirements, and the above process needs to be repeated until active_mds is equal to max_mds.

[0031] Figure 4 is one of the schematic diagrams of the metadata service startup and activation provided by the present invention. As Figure 4 shown, according to the above MDS service startup process, assuming that two MDS services are started simultaneously when each server joins the cluster and there are three servers in the cluster, the MDS services are numbered according to the startup order. Specifically, as Figure 4 (a) shown. At this time, if the administrator sets max_mds = 3, according to the above MDS service activation process, MDS0, MDS1, and MDS2 will be activated in turn according to the startup order of the MDS services (i.e., the numbers of the MDS services). Specifically, as Figure 4 (b) shown. It can be clearly seen from Figure 4 (b) that there are significant differences in the number of active MDSs carried on different servers, which will lead to uneven load among the servers.

[0032] Figure 5 is the second schematic diagram of the metadata service startup and activation provided by the present invention. As Figure 5 shown, to address the above-mentioned load imbalance problem, the administrator can solve it by adding operation and maintenance steps to strictly control the startup order of each MDS service on different servers. For example, for homogeneous servers, the administrator can batch complete the above operations through a relatively simple round-robin algorithm. The numbers of each MDS service after startup are as Figure 5 (a) shown. At this time, the administrator sets max_mds to 6. According to the above MDS service activation process, the result can be obtained as Figure 5(b) The activation result shown.

[0033] Although the problem of uneven load between servers can be solved to a certain extent by increasing the operation and maintenance steps. However, this increases the operation and maintenance time and labor costs. Moreover, in the actual production environment, as time goes by and the enterprise develops, servers are often purchased in batches at different times, which will result in different hardware resources, models or manufacturers among the servers, that is, there are heterogeneous servers in the cluster. In this scenario, even if the operation and maintenance steps are increased, there will still be a situation where low - configuration servers are overloaded or high - configuration servers waste resources. At this time, very complex operation and maintenance intervention is required to solve the problem, greatly increasing the operation and maintenance difficulty and complexity.

[0034] For Figure 2 and Figure 3 the start - up and activation processes of the MDS service shown in, in addition to bringing the above - mentioned problems of uneven load between servers and increasing the operation and maintenance complexity of administrators, there are also defects in being unable to handle transiently high traffic and being unable to cope with the expansion of the number of servers.

[0035] Figure 6 is the third schematic diagram of the start - up and activation of the metadata service provided by the present invention. As Figure 6 shown, the load of the server will change with the access of the client. With different services, sometimes there will be a sudden large increase in access to certain metadata. At this time, the resources occupied by the corresponding MDS service will increase exponentially. Suppose the access volume to MDS0 by the client suddenly increases, resulting in an increase in the resources occupied by MDS0, and even most of the resources of the entire server can be occupied. As Figure 6 (a) shown, the orange in the figure indicates that the MDS0 service occupies a large amount of resources of server 1. At this time, if the storage administrator adjusts max_mds to 4, then according to the above - mentioned activation process of the MDS service, the MDS3 service will be activated. The specific distribution of active MDS in the cluster is as Figure 6 (b) shown.

[0036] Obviously, at this time, server 1 is already under high load. Continuing to activate the MDS3 service will further increase the burden on server 1, easily leading to server 1 becoming the performance bottleneck of the entire cluster. Therefore, for Figure 4 the activation process described above, even if the administrator increases the operation and maintenance steps, the problem of transiently high traffic (i.e., sudden increase in access volume) still cannot be solved.

[0037] Figure 7 is the fourth schematic diagram of the start - up and activation of the metadata service provided by the present invention. As Figure 7As shown in the figure, assume that there are three servers in the cluster, and each server can host a maximum of 3 MDS services. The max_mds is set to 6. According to the above MDS service activation process, the distribution of active MDS is as follows Figure 7 (a). If the metadata service capacity provided by the cluster is insufficient at this time and a server needs to be added to support the business, according to the MDS service startup process, server 4 is added to the cluster, and 3 MDS services are started on this server. At this time, the distribution of MDS services is as follows Figure 7 (b). Subsequently, the storage administrator adjusts max_mds to 8. According to the above MDS service activation process, MDS6 and MDS7 will be activated in sequence by number. After the activation is completed, the distribution of active MDS in the cluster is as follows Figure 7 (c). Obviously, according to such an activation process, the newly activated MDS services are still on the old servers, and the newly added servers do not carry the client access volume, failing to achieve the effect of improving the overall performance by adding servers.

[0038] As can be seen from the above introduction, there are many problems in starting multiple MDS services on a single server, such as uneven load between servers, increased operation and maintenance complexity, inability to handle excessive instantaneous traffic, and inability to cope with the expansion of the number of servers. To solve these problems, the present invention provides a metadata service management method that optimizes the startup process and activation process of MDS services. It can not only enable a single server to host multiple MDS services, but also avoid uneven load between servers without increasing operation and maintenance costs, providing a higher-performance metadata service capacity. The present invention can also handle complex problems such as common load hotspots (i.e., excessive instantaneous traffic), heterogeneous environments, and server number expansion in the production environment well.

[0039] It should be noted that metadata service management refers to the process of managing the startup and activation of MDS services, aiming to ensure that MDS services can be started normally according to requirements, dynamically adjust their activation numbers, etc., to ensure that the metadata service capacity of the cluster meets the requirements. The technical solution provided by the present invention will be introduced in detail below.

[0040] Based on any of the above embodiments, Figure 8 is a schematic flowchart of the metadata service management method provided by the present invention. As shown in Figure 8 the figure, the method includes: Step 810, based on the received metadata service activation request, determine the target activation number of the metadata service, and obtain the current activation number of the metadata service in the cluster.

[0041] It should be noted that the execution subject of the metadata service management method provided by the embodiments of the present invention can be a management node of a distributed file storage system. This management node is responsible for monitoring the status of each server in the cluster and managing the startup and activation of the metadata service (i.e., the MDS service) to ensure that the number of MDS services in the cluster meets the business requirements and system performance requirements.

[0042] Specifically, a metadata service activation request refers to a request initiated by a user or a system, indicating that a certain number of metadata services (i.e., MDS services) need to be activated to meet the current business or system operation's demand for metadata services. The management node can receive the metadata service activation request in various ways. For example, the management node can provide a specific API (Application Programming Interface), and the user or system sends the activation request by calling this interface and passing the corresponding parameters. Another example is to send the activation request in the form of a message to a specified message queue, and the management node can read and process these requests from the message queue. For yet another example, the management node can provide a graphical management interface, and the user initiates the request by entering relevant information on the interface and clicking the activation button.

[0043] After receiving the metadata service activation request, the management node can parse the metadata service activation request to determine the target activation quantity. The specific parsing method depends on the format and transmission method of the request. For example, if the request is JSON-formatted data passed through an API interface, the management node can parse a specific field (such as max_mds) in the JSON data, which contains the number of metadata services that the user or system expects to activate, that is, the target activation quantity. Here, the target activation quantity refers to the number of metadata services expected to be activated in the cluster based on factors such as current business requirements, system load, and performance requirements.

[0044] Meanwhile, the current active count of the MDS service in the cluster can be obtained in the following ways. For example, many cluster management tools (such as Kubernetes, etc.) provide functions to query the service status in the cluster, and the current active count of the MDS service can be obtained through the APIs or command-line interfaces of these tools. Another example is that some monitoring systems can monitor the running status of services in the cluster in real time, including the active count of the MDS service, so that this information can be obtained through the interface or API of the monitoring system. In addition, custom scripts can be written to communicate with each server in the cluster, query the status of the MDS service on each server, and count the currently active count. It should be understood that the current active count refers to the number of MDS services that are currently in the active state (i.e., the active state) in the cluster, and the active state means that these MDS services are running normally and can provide metadata-related services.

[0045] Step 820, in the case where the current active count is less than the target active count, determine a target server from the server set, where the target server is the server with the largest number of metadata services in the backup state currently owned.

[0046] Specifically, after obtaining the target active count and the current active count of the MDS service, the difference between the two can be compared to achieve dynamic activation of the MDS service. If the current active count is less than the target active count, it means that the number of MDS services in the active state in the cluster is insufficient to meet the expected business requirements or system performance requirements. The cluster may not be able to efficiently process the current metadata requests due to the insufficient number of active MDS services, resulting in increased response latency, decreased system throughput, and other problems. Therefore, more MDS services need to be further activated to reach the target active count.

[0047] To activate the MDS service, first, a target server needs to be determined from the server set to activate the MDS service on this target server. Here, when determining the target server from the server set, a specific strategy can be adopted, that is, select the server with the largest number of MDS services in the backup state (i.e., the slave state) currently owned. Specifically, the server set can be traversed to obtain the number of MDS services in the backup state on each server, and then compare them to select the server with the largest number as the target server. Here, the target server refers to the server selected from the server set with the largest number of MDS services in the backup state currently owned, and this server will be used to activate new MDS services to increase the number of active MDS services in the cluster.

[0048] Step 830: Check the availability of each metadata service currently in the backup state on the target server. If an available metadata service is detected among the metadata services, activate the available metadata service and update the current activation count.

[0049] It should be noted that after determining the target server, in order to ensure the availability of the activated MDS services, it is necessary to check the availability of each MDS service currently in the backup state on the target server to screen out the available MDS services for activation. Here, the availability check refers to a series of check operations performed on the MDS services in the backup state to determine whether they can be normally activated and provide the expected services.

[0050] Specifically, when checking the availability of each MDS service currently in the backup state on the target server, it can be achieved in various ways. For example, the current load situation of the target server can be checked. If the load is high, activating the MDS services on this target server will cause the target server to be overloaded, leading to the problem of unbalanced load among servers. Therefore, at this time, it is not appropriate to activate the MDS services on the target server anymore, that is, the MDS services on the target server are considered unavailable, and it is necessary to try the next server for MDS service activation. Another example is that if the current load of the target server is relatively low, it means that the MDS services on the target server can be activated. At this time, according to the startup order of each MDS service on this target server, it can be judged in turn whether each MDS service is available (such as whether it is assigned to other file systems, whether the administrator sets this MDS service as non-activatable, etc.). If an available MDS service is detected, this MDS service can be directly activated.

[0051] It can be understood that when an available metadata service is detected on the target server, it indicates that at least one MDS service in the backup state on this target server meets the conditions for activation. Therefore, the available MDS service can be selected for activation operations to increase the number of activated MDS services in the cluster, thereby gradually approaching the target activation count. Here, when activating the available MDS service, it can be achieved according to the specific system architecture and management mechanism. Generally speaking, an activation instruction can be sent to this MDS service to make it switch from the backup state to the activation state. For example, in a distributed storage system, the activation operation may be triggered by modifying the configuration file of the MDS service or calling the corresponding management interface.

[0052] After completing the activation of each available MDS service, the current activation count can be updated by incrementing the value of the current activation count by 1. Since a new MDS service has been successfully activated, the total number of activated MDS services in the cluster has increased by 1. Specifically, a variable (such as active_mds) can be maintained to record the current activation count, and this variable can be incremented by 1 after the activation operation is successful.

[0053] In addition, if it is detected that there is no available MDS service on the target server, it indicates that all MDS services in the backup state on this server are currently unavailable and cannot be immediately activated to increase the number of activated MDS services in the cluster. In this case, the MDS services in the backup state currently owned by each server in the server set can be sorted in descending order, and based on the sorting result, the next server of the target server can be determined from it, and this next server can be used as the new target server to start a new round of MDS service availability check and activation process, and so on, until the current activation count of the MDS services in the cluster reaches the target activation count. If the MDS services in the backup state on all servers are unavailable, further troubleshooting is required to find the cause of the MDS service unavailability and fix it. After the problem is solved, the activation operation can be performed again.

[0054] Step 840, in the case where the updated current activation count is less than the target activation count, continue to determine a new target server from the server set and activate the available metadata services on the new target server until the updated current activation count is equal to the target activation count.

[0055] It should be noted that this step will continuously determine new target servers from the server set and activate the available metadata services on them until the number of currently activated MDS services in the cluster is equal to the target activation count. This process ensures that the cluster can dynamically adjust the number of MDS services according to actual needs and provide higher-performance metadata service capabilities.

[0056] Specifically, after completing an activation operation on the available metadata services on the target server and updating the current activation count, it is necessary to determine whether the updated current activation count is still less than the target activation count. When the updated current activation count is less than the target activation count, it indicates that although some of the available MDS services have been activated, the total number of activated MDS services in the cluster still has not reached the expected target activation count. At this time, it is necessary to continue to determine a new target server from the set of servers, that is, repeat the above steps 820 and 830 until the updated current activation count is equal to the target activation count, which means that the number of activated MDS services in the cluster has reached the expected target, and the system can operate normally and meet the business requirements and system performance requirements.

[0057] The method provided by the embodiments of the present invention can determine the target activation count of the metadata service by receiving a metadata service activation request, and according to the difference between the current activation count and the target activation count, the system can dynamically adjust the number of metadata services in the activated state in the cluster to achieve on-demand allocation of resources. When activating each metadata service, the system will select the server with the largest number of currently available metadata services in the backup state as the target server, so as to activate the available metadata services on the target server. This way of selecting the target server can not only maximize the utilization of backup resources and reduce the overhead of starting new services, but also ensure that each time a metadata service is activated, it is from a server with a relatively large number of currently available metadata services in the backup state, that is, a server with a relatively light current load. Thus, load balancing between servers can be achieved, avoiding the situation where some servers are overloaded while others are idle. In addition, before activating the metadata services on the target server, an availability check will be performed on it to ensure that the activated metadata services are available, improving the overall reliability of the metadata services.

[0058] Based on any of the above embodiments, the number of metadata services started on any server is determined based on the hardware resources of any server and the hardware resources required by the metadata service.

[0059] Specifically, servers with different hardware resources can start different numbers of MDS services. The number of MDS services started on each server can be determined according to the hardware resources of the server and the hardware resources required by the MDS service. Here, the hardware resources of the server refer to the specifications and performance parameters of various hardware components of the server itself, which determine the overall computing power, storage capacity, and operating efficiency of the server. For example, the hardware resources of the server can include two key hardware indicators: the number of CPU cores and the memory capacity. The hardware resources required by the MDS service refer to the hardware resource requirements necessary for running the MDS service, which can also include two indicators: the CPU core number requirement and the memory capacity requirement.

[0060] Exemplarily, assume that an MDS service requires 4-core CPUs and 64 GB of memory. Then, a server with 32-core CPUs and 256 GB of memory can start 4 MDS services.

[0061] Figure 9 It is the fifth schematic diagram of the start-up and activation of the metadata service provided by the present invention. As Figure 9 shown, assume that two MDS services are started while each server joins the cluster, and there are three servers in the cluster. Then, the MDS services are numbered in the order of start-up, specifically as Figure 9 (a) shows. At this time, if the storage administrator initiates a metadata server activation request by setting max_mds = 3, the management node can obtain the target activation quantity of max_mds = 3 by parsing this request, while the current activation quantity of the MDS services in the cluster is active_mds = 0. Since active_mds is less than max_mds, the activation process of the MDS services will be entered.

[0062] First, the management node can screen out the target server from the server set. Since the number of MDS services in the backup state on each server is the same currently, server 1 can be selected as the target server first. After availability check, it is determined that the MDS0 service on server 1 is available. After activating this service, the current activation quantity is updated to active_mds = 1. Since the updated current activation quantity is still less than the target activation quantity, the second-round activation process of the MDS services will be entered.

[0063] Similarly, it is necessary to first screen out the target server from the server set. At this time, the number of MDS services in the backup state is the largest on server 2 and server 3, and server 2 can be selected as the target server. After availability check, it is determined that the MDS2 service on server 2 is available. This service is activated, and the current activation quantity is updated to active_mds = 2. Since the updated current activation quantity is still less than the target activation quantity, the third-round activation process of the MDS services will be entered.

[0064] In the new-round activation process of the MDS services, since the number of MDS services in the backup state is the largest on server 3 currently, server 3 will be selected as the target server. After availability check, it is determined that the MDS4 service on server 3 is an available MDS service. After activating this service, the current activation quantity is updated to active_mds = 3. Since the updated current activation quantity is equal to the target activation quantity, the activation process of the MDS services ends.

[0065] The distribution of the MDS services after activation is specifically asFigure 9 (as shown in (b), starting from Figure 9 (b), it can be clearly seen that in the embodiments of the present invention, by screening out the target servers from the server set each time and then activating the available MDS services on the target servers, the load balancing between different servers can be effectively achieved. In addition, since the load between the servers is balanced, the operation and maintenance personnel can more easily monitor and manage the running status of the cluster, without manual operation and maintenance management, and at the same time, the troubleshooting and repair work caused by load imbalance is reduced, thereby reducing the operation and maintenance cost.

[0066] Figure 10 is the sixth schematic diagram of the startup and activation of the metadata service provided by the present invention. As Figure 10 (a) shows, assume that the MDS0 service on server 1 is an access hotspot, that is, the access volume of the client to the MDS0 service suddenly increases, resulting in MDS0 occupying a large amount of system resources of server 1. At this time, if the storage administrator increases the max_mds value, for example, adjusts max_mds from 3 to 4, the activation process of the MDS service will be triggered. In the embodiments of the present invention, during the activation process of the MDS service, when the management node checks the availability of MDS3 and MDS6 on server 1, it will find that the current load of server 1 is relatively high. If MDS3 and MDS6 are activated, it will cause the load of server 1 to be too high. Therefore, MDS3 and MDS6 will be considered as unavailable MDS services, and server 2 will be selected as the new target server, and finally the MDS4 service on server 2 will be activated, thereby avoiding causing a greater burden on the highly loaded server 1, and at the same time increasing the number of activated MDS services in the cluster, and being able to provide higher metadata processing capabilities.

[0067] Figure 11 is the seventh schematic diagram of the startup and activation of the metadata service provided by the present invention. As Figure 11 shown, for the scenario of server number expansion, the embodiments of the present invention can also handle it well. As Figure 11 (a) shows, assume that there are originally three servers in the cluster, and there are two MDS services in the active state and one MDS service in the backup state on each server. At this time, a new server (i.e., server 4, on which three MDS services are started) is added to provide higher metadata service capabilities. The administrator sets max_mds to 8. According to the new activation policy provided by the embodiments of the present invention, the management node will first select the server with the largest number of MDS services in the backup state as the target server, that is, select the newly added server 4 as the target server. As Figure 11 (b) shows, finally, it can be seen that the 4 servers evenly bear the load of client access, solving the problem of server number expansion before.

[0068] Figure 12 This is the eighth schematic diagram of the startup and activation of the metadata service provided by the present invention. As Figure 12 shown, in the scenario of heterogeneous servers, the number of MDS services carried by each server will be different. According to the hardware resources of each server and the hardware resources required by the MDS service, the calculated MDS service load distribution is as follows: The hardware resources of server 1 can start 3 MDS services simultaneously, the hardware resources of server 2 can start 4 MDS services simultaneously, the hardware resources of server 3 can start 2 MDS services simultaneously, and the hardware resources of server 4 can start 5 MDS services simultaneously. When the administrator sets max_mds = 9 to trigger the MDS service activation process, in the absence of access hotspots, the management node will first select server 4 as the target server and determine that MDS1 on server 4 is an available service for activation. Subsequently, the corresponding MDS services will continue to be activated according to the activation policy provided by the embodiments of the present invention. Figure 12 Each blue MDS service shown in Figure 12 represents the activated MDS service. For ease of understanding, the numbers of these services are marked in the order of activation. As can be seen from

[0069] Based on any of the above embodiments, in step 820, determining the target server from the server set includes: Step 821, obtaining the current backup number of the metadata service in the cluster; Step 822, when the current backup number is greater than the preset backup number, determining the target server from the server set; Step 823, when the current backup number is less than or equal to the preset backup number, determining that the metadata service activation request fails.

[0070] It should be noted that in order to ensure the high availability of the system, a certain number of MDS services in the backup state (i.e., slave MDS) usually need to be reserved in the cluster. Therefore, before activating the MDS service, it is necessary to first check whether the current number of MDS services in the backup state in the cluster is greater than the minimum redundant backup service number required for the cluster stability (i.e., the preset backup number).

[0071] Specifically, the current number of backup MDS services in a cluster refers to the number of MDS services currently in a backup state in the distributed storage cluster. The method for obtaining this number can be referred to the method for obtaining the current number of active MDS services in the above-mentioned embodiment and will not be further described here. The preset number of backup MDS services refers to a desired number of backup MDS services pre-determined based on factors such as cluster size, business needs, and system performance requirements. This ensures that there are sufficient backup MDS services in the cluster to handle failures or maintenance of the primary MDS service (i.e., the active MDS service), thereby ensuring system reliability and stability.

[0072] If the current number of backups is greater than the preset number of backups, it indicates that there are currently redundant backup MDS services in the cluster, which means that there are sufficient backup MDS services that can be activated to meet the activation requirements of the MDS service. Therefore, the target server can be determined from the server set, and the availability check can be performed on each MDS service in the backup state on the target server to activate the available MDS services. If the current number of backups is less than or equal to the preset number of backups, it indicates that there are currently no redundant backup MDS services in the cluster that can be activated, so the activation operation cannot be performed. At this time, the user can be informed that the activation request for the metadata service has failed.

[0073] Based on any of the above embodiments, each metadata service currently in a backup state on the target server is obtained based on the backup service set of the target server.

[0074] It should be noted that, in order to facilitate the management of the MDS services in the backup state on each server, a backup service set can be maintained for each server. Here, the backup service set refers to the set consisting of all MDS services currently in the backup state on the server.

[0075] Accordingly, the method also includes a metadata service startup process. Before the cluster initially activates the metadata service, the metadata service startup process can be executed to ensure that there are sufficient metadata services in the cluster to meet the activation requirements. The metadata service startup process includes: Start the metadata service on the first server, set the metadata service to backup state, and update the current backup quantity; In the case where the first server is a newly added server, the metadata service information of the first server is added to the server set, and the metadata service is added to the backup service set of the first server; When the first server is not a newly added server, the metadata service is added to the backup service set of the first server.

[0076] Specifically, the first server refers to the server selected to start a new MDS service during the MDS service startup process. It may be an existing server in the cluster or a newly added server to the cluster. When starting the MDS service on the first server, it can be achieved through the following steps: First, use a remote login tool (such as SSH) to log in to the first server. If the MDS software is not installed on the first server, it needs to be installed first. Then, configure the MDS service according to the cluster configuration requirements. Finally, use the corresponding startup command to start the MDS service. After the MDS service is started, it is in the backup state by default.

[0077] After the startup of the MDS service is completed, the number of MDS services in the backup state in the cluster will increase, so the current backup number can be automatically updated. At the same time, the information of the started MDS service needs to be saved to the backup service set of the first server. If the first server is a newly added server, it means that the server is newly added to the distributed storage cluster. In this case, the metadata service information of the first server needs to be saved to the server set first, and then the started MDS service is added to the backup service set of the first server.

[0078] Specifically, when adding the metadata service information of the first server to the server set, the relevant information of the first server, such as the server IP address, hostname, number, hardware resources, etc., can be recorded first. Then, the information of the MDS service started on the first server (such as number, listening port, status, etc.) is associated with the relevant information of the first server and recorded in the server set. It should be understood that by adding the metadata service information of the first server to the server set, the cluster can achieve unified management of all servers, which is convenient for monitoring, scheduling, and maintaining the MDS service.

[0079] If the first server is not a newly added server, it means that the server is already part of the cluster and may have run other services or participated in certain operations of the cluster before. In this case, the relevant information of the server already exists in the server set. Therefore, the newly started metadata service can be directly added to the backup service set of the server.

[0080] Based on any of the above embodiments, in step 820 or step 822, determining the target server from the server set includes: Sorting based on the number of metadata services in the backup state currently owned by each server in the server set, and determining candidate servers from each server according to the sorting result. The candidate server is the server with the largest number of metadata services in the backup state currently owned. When there is one candidate server, the candidate server is used as the target server; When there are multiple candidate servers, based on the number of currently active metadata services on each candidate server, a target server is determined from among the candidate servers.

[0081] It should be noted that when determining the target server from the server set, there may be multiple servers with the same number of MDS services in the backup state. In this case, by comparing the number of currently active MDS services on these servers, the target server can be further screened and determined, so as to ensure load balancing among the servers as much as possible.

[0082] Specifically, for each server in the server set, first, they can be sorted according to the number of MDS services in the backup state they currently have, and candidate servers are determined from the sorted servers. The candidate server is the server with the largest number of metadata services in the backup state currently owned.

[0083] If there is only one candidate server, it indicates that this server has the largest number of MDS services in the backup state and there is no other server competing with it. Therefore, this candidate server can be directly used as the target server.

[0084] When there are multiple candidate servers, a target server can be further determined from these candidate servers based on the number of currently active MDS services on each candidate server. Specifically, a candidate server with a smaller number of active MDS services can be preferentially selected because active MDS services will occupy the resources of the server, and selecting a server with fewer active services can reduce the impact on existing services.

[0085] Exemplarily, as Figure 12 shown, after activating the MDS3 service on server 2, since the current activation quantity has not reached the target activation quantity, a new round of activation process will be entered. In the new round of activation process, since the number of MDS services in the backup state owned by server 1, server 2, and server 4 is all 3, these 3 servers will become candidate servers. Among these candidate servers, since the number of active MDS services on server 1 is 0, the number of active MDS services on server 2 is 1, and the number of active MDS services on server 4 is 2, therefore, server 1 can be selected as the target server, and the available MDS service (i.e., MDS4) on this server can be activated.

[0086] Based on any of the above embodiments, in step 830, perform an availability check on each metadata service currently in the backup state on the target server, including: Step 831, obtain the current load of the target server. If the current load exceeds a preset threshold, it is determined that there is no available metadata service on the target server; Step 832, if the current load does not exceed the preset threshold, perform an availability check on each metadata service based on the startup order of each metadata service currently in the backup state on the target server.

[0087] It should be noted that when performing an availability check on each MDS service currently in the backup state on the target server, first, obtain the current load of the target server to judge the general situation of whether there is an available MDS service. If the load is too high, it is directly determined that there is no available service; if the load is normal, then according to the startup order of the MDS service, for each MDS service in the backup state, determine whether it is an available MDS service according to its specified state and activation settings.

[0088] Specifically, the current load information of the target server, such as indicators like CPU usage rate and memory usage rate, can be obtained through a server monitoring system or relevant commands. These indicators are comprehensively calculated or transformed into a load value according to specific rules, and compared with the preset threshold. Here, the preset threshold can be obtained by averaging the load conditions of each server, or can be preset based on historical experience. The embodiments of the present invention do not make specific limitations on this.

[0089] If the current load of the target server exceeds the preset threshold, it indicates that the current load of the target server is too high. At this time, there is no need to perform a detailed check on each MDS service on the target server, and it can be quickly determined that there is no available metadata service, saving inspection time and resources. This is because when the current load of the target server is too high, if the MDS service on this server continues to be activated, it is easy to cause this server to become the performance bottleneck of the entire cluster. Therefore, the activation of the MDS service on this server is abandoned.

[0090] When the current load of the target server does not exceed the preset threshold, the startup order information of each MDS service currently in the backup state on the target server can be obtained, and in this order, an availability check is performed on each MDS service in turn.

[0091] Further, in step 832, performing an availability check on each metadata service includes: Obtain the specified state and activation setting of any metadata service; In the case where the specified state is unspecified and the activation setting is activatable, determine that any metadata service is an available metadata service.

[0092] Specifically, when performing availability checks on each MDS service on the target server in sequence according to the startup order, the specified status and activation setting information of the MDS service can be obtained through the management interface or configuration file of the MDS service. According to the obtained information, if the specified status is unspecified and the activation setting is activatable, it indicates that the MDS service is an available MDS service. At this time, the availability check for subsequent MDS services is no longer performed, and the available MDS service can be directly activated. If the specified status of a certain MDS service is specified, or the activation setting is non-activatable, then the MDS service is considered an unavailable MDS service and cannot be activated. It should be understood that the specified status refers to whether the MDS service is assigned to other file systems. Among them, unspecified indicates that the MDS service is not assigned to other file systems.

[0093] In the embodiments of the present invention, by clearly defining two key factors, namely the specified status and the activation setting, to determine the availability of the metadata service, it is possible to accurately screen out truly available services and avoid misjudging unavailable or services already occupied by other file systems as available.

[0094] Based on any of the above embodiments, in step 830, after performing availability checks on each metadata service currently in the backup state on the target server, it further includes: If no available metadata service is found among the metadata services, then sort based on the number of metadata services currently in the backup state owned by each server in the server set, and determine the next server of the target server from each server according to the sorting result; Use the next server as the new target server and return to the step of performing availability checks on each metadata service currently in the backup state on the target server.

[0095] Specifically, if there is no available MDS service on the current target server, it is necessary to switch to the next server for inspection. Specifically, the MDS services currently in the backup state owned by each server in the server set can be sorted in descending order, and in the sorted server list, select the server immediately following the current target server as the next server. For example, if the current target server is the server with the largest number of MDS services currently in the backup state, then in the sorted server list, select the server ranked second (this server is the next server of the target server) as the new target server.

[0096] After determining the new target server, you can jump to the step of checking the availability of each MDS service currently in the backup state on the target server and restart the MDS service activation process until the current number of activated MDS services in the cluster reaches the target number of activated MDS services.

[0097] Based on any of the above embodiments, an embodiment of the present invention provides a management method for a single server supporting multiple data services, which mainly includes startup process management and activation process management of MDS services. Figure 13 This is the second flowchart of the metadata service startup provided by the present invention. Figure 13 As shown, a new host_map array (i.e., server set) is added, and each server corresponds to an entry in the array, which is used to record information about all MDS services on the server. At the same time, a host_standy_daemons array (i.e., the backup service set of each server) is maintained for each server in the host_map array to record information about all MDS services in backup status on the server. The standy_daemons array is a global backup service set, which is used to record information about all MDS services in backup status in the cluster. Based on this array, the current number of backups of MDS services in the cluster can be determined. The startup process of the MDS service includes the following steps: Step A1: Start the MDS service on the server and set the MDS service to the backup state by default; Step A2: Determine whether the server where the MDS service is located is a newly added server. If so, add the newly added server to the host_map array and add the MDS service to the host_standy_daemons array corresponding to the newly added server in the host_map array. If not, directly locate the server in the host_map array and add the started MDS service to the host_standy_daemons array corresponding to the server. Step A3: Add the started MDS service to the end of the standy_daemons array to complete the startup of the MDS service.

[0098] Figure 14 This is the second flowchart of the metadata service activation provided by the present invention. Figure 14 As shown, the activation process of the MDS service includes the following steps: Step B1: The administrator initiates an MDS service activation request by adjusting the max_mds parameter; Step B2: Determine the target activation quantity max_mds based on the MDS service activation request, and compare the current activation quantity active_mds of the MDS service in the cluster with the target activation quantity max_mds; Step B3: If active_mds < max_mds, check whether the current backup quantity (i.e., the current backup service quantity) of the MDS service in the cluster is greater than the minimum redundant backup quantity required for cluster stability. If so, continue to execute the subsequent Step B4; otherwise, return a failure; Step B4: The cluster starts trying from the server with the largest current number of backup MDS services, that is, traverse the host_map array, and select the server with the largest number of host_standy_daemons (i.e., the largest current number of backup MDS services) as the target server; Step B5: Take out the host_standy_daemons array of the target server from the host_map array, and traverse this array to check the availability of each MDS service currently in the backup state on the target server; Step B6: If a certain MDS service traversed is available, the MDS service can be deleted from the host_standy_daemons array and the standy_daemons array, and set to the active state; if a certain MDS service traversed is unavailable, continue to traverse and check the availability of the next MDS service; Step B7: After activating the MDS service, update the value of active_mds, and judge the updated active_mds and max_mds. If active_mds is still less than max_mds, repeat the above Steps B3 - B6 until the updated active_mds is equal to max_mds, and end the activation process.

[0099] The method provided by the embodiment of the present invention can enable a single server to carry multiple MDS services, and provide a higher-performance metadata service capability without increasing the operation and maintenance cost and the procurement cost. Moreover, for complex problems such as common load hotspots and heterogeneous environments in the production environment, the present invention can also handle them well.

[0100] Based on any of the above embodiments, Figure 15 is a schematic structural diagram of the metadata service management device provided by the present invention, as Figure 15 shown, the device includes: An obtaining unit 1510, configured to determine the target activation quantity of the metadata service based on the received metadata service activation request, and obtain the current activation quantity of the metadata service in the cluster; A determination unit 1520, configured to determine a target server from a set of servers when the current activation quantity is less than the target activation quantity, where the target server is the server with the largest number of metadata services in the backup state currently owned; An activation unit 1530, configured to perform an availability check on each metadata service in the backup state on the target server. If an available metadata service is detected among the metadata services, activate the available metadata service and update the current activation quantity; A loop unit 1540, configured to, when the updated current activation quantity is less than the target activation quantity, continue to determine a new target server from the set of servers and activate the available metadata services on the new target server until the updated current activation quantity is equal to the target activation quantity.

[0101] The device provided by the embodiment of the present invention can, by receiving a metadata service activation request, determine the target activation quantity of the metadata service, and according to the difference between the current activation quantity and the target activation quantity, the system can dynamically adjust the quantity of the metadata services in the active state in the cluster to achieve on-demand resource allocation. When activating each metadata service, the system will select the server with the largest number of metadata services in the backup state currently owned as the target server, so as to activate the available metadata services on the target server. This way of selecting the target server can not only maximize the utilization of backup resources and reduce the overhead of starting new services, but also ensure that each time a metadata service is activated, it is performed on a server with a relatively large number of metadata services in the backup state currently, that is, a server with a relatively light current load. Thus, load balancing between servers can be achieved, avoiding the situation where some servers are overloaded while other servers are idle. In addition, before activating the metadata service on the target server, an availability check will be performed on it to ensure that the activated metadata service is available, improving the overall reliability of the metadata service.

[0102] Based on any of the above embodiments, the determination unit 1520 is specifically configured to: obtain the current backup quantity of the metadata service in the cluster; determine a target server from the set of servers when the current backup quantity is greater than the preset backup quantity; and determine that the metadata service activation request fails when the current backup quantity is less than or equal to the preset backup quantity.

[0103] Based on any of the above embodiments, each metadata service currently in the backup state on the target server is obtained based on the backup service set of the target server. Correspondingly, the apparatus further includes a startup unit, and the startup unit is configured to: start a metadata service on the first server, set the metadata service to the backup state, and update the current backup count; when the first server is a newly added server, add the metadata service information of the first server to the server set, and add the metadata service to the backup service set of the first server; when the first server is not a newly added server, add the metadata service to the backup service set of the first server.

[0104] Based on any of the above embodiments, the determining unit 1520 is specifically configured to: sort according to the number of metadata services currently in the backup state owned by each server in the server set, and determine a candidate server from each server according to the sorting result, where the candidate server is the server with the largest number of metadata services currently in the backup state; when there is one candidate server, use the candidate server as the target server; when there are multiple candidate servers, determine the target server from each candidate server based on the number of metadata services currently in the active state on each candidate server.

[0105] Based on any of the above embodiments, the activation unit 1530 includes an inspection subunit, and the inspection subunit is configured to: obtain the current load of the target server, if the current load exceeds a preset threshold, determine that there is no available metadata service on the target server; if the current load does not exceed the preset threshold, perform an availability check on each metadata service based on the startup order of each metadata service currently in the backup state on the target server.

[0106] Based on any of the above embodiments, the inspection subunit is specifically configured to: obtain the specified state and activation setting of any metadata service; when the specified state is unspecified and the activation setting is activatable, determine that the any metadata service is an available metadata service.

[0107] Based on any of the above embodiments, the apparatus further includes a server switching unit, and the server switching unit is configured to: if it is detected that there is no available metadata service among all metadata services, sort according to the number of metadata services currently in the backup state owned by each server in the server set, and determine the next server of the target server from each server according to the sorting result; use the next server as the new target server, and return to the step of performing an availability check on each metadata service currently in the backup state on the target server.

[0108] Based on any of the above embodiments, the number of metadata services started on any server is determined based on the hardware resources of any server and the hardware resources required by the metadata services.

[0109] Figure 16 An example of the physical structure diagram of an electronic device is as Figure 16 shown. The electronic device may include: a processor 1610, a communication interface 1620, a memory 1630, and a communication bus 1640. Among them, the processor 1610, the communication interface 1620, and the memory 1630 complete communication with each other through the communication bus 1640. The processor 1610 can call the logical instructions in the memory 1630 to execute the metadata service management method, which includes: based on the received metadata service activation request, determining the target activation number of the metadata service, and obtaining the current activation number of the metadata service in the cluster; in the case where the current activation number is less than the target activation number, determining the target server from the server set, where the target server is the server with the largest number of currently available metadata services in the backup state; performing an availability check on each metadata service currently in the backup state on the target server. If an available metadata service is detected among the metadata services, activating the available metadata service and updating the current activation number; in the case where the updated current activation number is less than the target activation number, continuing to determine a new target server from the server set and activating the available metadata services on the new target server until the updated current activation number is equal to the target activation number.

[0110] In addition, when the logical instructions in the above-mentioned memory 1630 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the related technology, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0111] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the metadata service management method provided by each of the above methods. The method includes: based on the received metadata service activation request, determining the target activation quantity of the metadata service, and obtaining the current activation quantity of the metadata service in the cluster; in the case where the current activation quantity is less than the target activation quantity, determining a target server from the server set, where the target server is the server with the largest number of metadata services in the backup state currently owned; performing an availability check on each metadata service in the backup state on the target server. If an available metadata service is detected among the metadata services, activating the available metadata service and updating the current activation quantity; in the case where the updated current activation quantity is less than the target activation quantity, continuing to determine a new target server from the server set and activating the available metadata service on the new target server until the updated current activation quantity is equal to the target activation quantity.

[0112] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the metadata service management method provided by each of the above methods. The method includes: based on the received metadata service activation request, determining the target activation quantity of the metadata service, and obtaining the current activation quantity of the metadata service in the cluster; in the case where the current activation quantity is less than the target activation quantity, determining a target server from the server set, where the target server is the server with the largest number of metadata services in the backup state currently owned; performing an availability check on each metadata service in the backup state on the target server. If an available metadata service is detected among the metadata services, activating the available metadata service and updating the current activation quantity; in the case where the updated current activation quantity is less than the target activation quantity, continuing to determine a new target server from the server set and activating the available metadata service on the new target server until the updated current activation quantity is equal to the target activation quantity.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative efforts. Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A metadata service management method, characterized in that, Including: Based on the received metadata service activation request, determine the target activation quantity of the metadata service, and obtain the current activation quantity of the metadata service in the cluster; When the current activation quantity is less than the target activation quantity, determine a target server from the server set, where the target server is the server with the largest number of metadata services in the backup state among the currently owned servers; Perform an availability check on each metadata service currently in the backup state on the target server. If an available metadata service is detected among the metadata services, activate the available metadata service and update the current activation quantity; When the updated current activation quantity is less than the target activation quantity, continue to determine a new target server from the server set, and activate the available metadata services on the new target server until the updated current activation quantity is equal to the target activation quantity.

2. The metadata service management method according to claim 1, wherein The determining of the target server from the server set includes: Obtain the current backup quantity of the metadata service in the cluster; When the current backup quantity is greater than the preset backup quantity, determine a target server from the server set; When the current backup quantity is less than or equal to the preset backup quantity, determine that the metadata service activation request fails.

3. The metadata service management method according to claim 2, characterized in that, Each metadata service currently in the backup state on the target server is obtained based on the backup service set of the target server. Correspondingly, the method further includes a metadata service startup process, and the metadata service startup process includes: Start the metadata service on the first server, set the metadata service to the backup state, and update the current backup quantity; When the first server is a newly added server, add the metadata service information of the first server to the server set, and add the metadata service to the backup service set of the first server; When the first server is not a newly added server, add the metadata service to the backup service set of the first server.

4. The metadata service management method according to claim 2, wherein The determining of the target server from the server set includes: Sort based on the number of metadata services currently in the backup state owned by each server in the server set, and according to the sorting result, determine a candidate server from the servers, where the candidate server is the server with the largest number of metadata services in the backup state among the currently owned servers; When there is one candidate server, use the candidate server as the target server; When there are multiple candidate servers, determine the target server from the candidate servers based on the number of metadata services currently in the active state on each candidate server.

5. The metadata service management method according to claim 1, wherein The performing of the availability check on each metadata service currently in the backup state on the target server includes: Obtain the current load of the target server. If the current load exceeds the preset threshold, determine that there is no available metadata service on the target server; If the current load does not exceed the preset threshold, perform an availability check on each metadata service that is currently in the backup state on the target server based on the startup order of the metadata services.

6. The metadata service management method according to claim 5, wherein The performing an availability check on each metadata service includes: Obtain the specified status and activation setting of any metadata service; When the specified status is unspecified and the activation setting is activatable, determine that the any metadata service is an available metadata service.

7. The metadata service management method according to any one of claims 1 to 6, characterized in that After performing the availability check on each metadata service that is currently in the backup state on the target server, it further includes: If no available metadata service is found among the metadata services, sort based on the number of metadata services in the backup state currently owned by each server in the server set, and determine the next server of the target server from the servers according to the sorting result; Take the next server as the new target server and return to the step of performing an availability check on each metadata service that is currently in the backup state on the target server.

8. The metadata service management method according to any one of claims 1 to 6, characterized in that, It further includes: The number of metadata services started on any server is determined based on the hardware resources of the any server and the hardware resources required by the metadata service.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the metadata service management method according to any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the metadata service management method according to any one of claims 1 to 8.

Images