A high-availability hierarchical distributed storage system with strong data consistency

Abstract

The application provides a data strongly consistent high-availability hierarchical distributed storage system and a method for preferentially selecting when a cold backup node is converted into a hot backup node based on the system. The hierarchical distributed storage system is based on a shared storage architecture. Multiple nodes of the hierarchical distributed storage system share a memory to access data. The nodes of the hierarchical distributed storage system include four identities, i.e., a master node, a hot backup node, a cold backup node and an offline node. When the cold backup node is converted into the hot backup node, the cold backup node is converted into the hot backup node based on a preset method for preferentially selecting when a cold backup node is converted into a hot backup node. The system of the present disclosure provides a high-availability and strong-consistency implementation scheme for a distributed storage system based on shared storage. The system makes the consistency of an N-node distributed storage system based on shared storage meet strong consistency, and the availability at most meet N-1 nodes of failure without affecting the system to provide services, while ensuring low latency of read and write requests.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a highly available hierarchical distributed storage system with strong data consistency and a method for selecting the best backup node when converting a cold backup node to a hot backup node based on the system. Background Technology

[0002] Distributed system consistency and availability technology is a relatively mature field, with many different solutions available in the industry. However, due to the limitations of the CAP theorem (Consistency, Availability, and Partition tolerance), all solutions inevitably involve a trade-off between consistency and availability. The CAP theorem states that a distributed system cannot simultaneously satisfy consistency, availability, and partition tolerance, and since partition tolerance is fundamental to distributed systems, a trade-off must ultimately be made between consistency and availability. The more refined PACELC theorem (Consistency, Availability, Partition Tolerance, Eventual Consistency, and Low Latency) takes the CAP theorem further into consideration. When a network partition occurs, as described in the CAP theorem, a trade-off between consistency and availability needs to be made. When there is no network partition, the relationship between consistency and latency also needs to be considered. For example, if every step of a distributed system is synchronized to all nodes in the cluster, strong consistency can certainly be guaranteed, but the latency will be so high that it is unacceptable for practical applications. Conversely, if there is no synchronization, the latency will be low, but consistency cannot be guaranteed, and it will also be difficult to apply to real-world scenarios.

[0003] Currently, industry solutions to the CAP problem are generally divided into AP and CP, which prioritize availability or consistency while ensuring partition tolerance. Products based on CP, such as the well-known distributed consistency system ZooKeeper, are based on Zab, a variant of the classic Paxos algorithm. It guarantees partition tolerance and strong consistency for write operations, but only slightly weaker sequential consistency for read operations. It supports write operations to the leader node and read operations on all nodes, guaranteeing partition tolerance and strong consistency for read and write operations. However, the trade-off is that all read and write operations must go through the leader node; strictly consistent read operations and write operations cannot be performed on other nodes. Some products further relax the consistency requirements, becoming eventual consistency, causal consistency, read-itself consistency, etc. The effectiveness of these consistency methods is strongly related to their respective application scenarios, such as some implementations of Memcached and MySQL master-slave replication. Products based on AP are generally not called distributed consistency systems because their design goals involve sacrificing consistency to some extent. They are often used in scenarios where consistency is not critical, such as comment and like systems on social networking sites. A well-known product is the distributed database Redis, which abandons consistency in favor of ensuring that all nodes are available at all times.

[0004] In the examples above, the stringency of consistency is usually strongly correlated with business logic, and a single standard cannot cover all requirements. Existing solutions based on the Paxos protocol and its variants are difficult to directly apply to distributed storage software that requires strong read / write consistency and low latency. This is because their read / write performance is directly limited by Paxos's majority consensus requirement, meaning that every read / write operation must go through consensus within the distributed cluster. This process involves several network communications and local I / O operations, and such latency is unacceptable for distributed storage software. Other relatively low-latency solutions, however, cannot meet the strong consistency requirement and therefore cannot be used for strictly distributed storage; otherwise, they could potentially cause serious problems such as data loss or inconsistency.

[0005] In the field of distributed storage, a more common solution is to utilize distributed locks or distributed leases implemented in these CP systems. A master node (Active) is elected in the cluster to provide read and write services and synchronize data with the standby node. Since communication only needs to occur between the two nodes, and both the master and standby nodes have internal caches, data persistence is unnecessary after each write. This forms a highly available, strongly consistent distributed storage system with strong consistency between the master and standby nodes and low read / write latency. Such a distributed storage system can tolerate the failure of at most one node without affecting service availability, while ensuring no data loss. Service unavailability and data loss risk occur only when both nodes fail simultaneously. Here, "simultaneously" means the time difference between the failures is less than the time it takes for the master or standby node to detect the failure and persist the cached data.

[0006] If the requirements for high availability continue to increase, simply increasing the number of backups and replicating the synchronization link between primary and backups is not feasible. This would increase the amount of data synchronized in a single write request and also increase the probability of failure. When synchronization fails, data persistence is necessary to ensure strong consistency, further increasing the pressure on the storage medium. This simple solution would significantly increase the network communication volume and write pressure on the storage medium of the distributed system, consuming a large amount of resources and potentially causing flooding. Simultaneously, read and write latency would also increase significantly, severely impacting performance and stability in practical applications. Therefore, it is necessary to propose a design scheme for a distributed storage system with more than two high-availability nodes under strong consistency and low latency requirements. Summary of the Invention

[0007] This application discloses a highly available hierarchical distributed storage system with strong data consistency and a method for optimal selection when converting cold standby nodes to hot standby nodes based on the system.

[0008] In a first aspect, this application discloses a highly available hierarchical distributed storage system with strong data consistency, the system comprising:

[0009] The hierarchical distributed storage system is based on a shared storage architecture;

[0010] In the hierarchical distributed storage system, multiple nodes share a single memory to access data.

[0011] The nodes in the hierarchical distributed storage system include four types of identities: master node, hot standby node, cold standby node, and offline node. When a cold standby node is converted to a hot standby node, a preset optimal selection method is used to convert the cold standby node to a hot standby node.

[0012] In one exemplary embodiment of this disclosure, the system includes:

[0013] When a node is a master node, it is used to provide read and write services.

[0014] When the node is a hot standby node, it is used to synchronize data with the master node, and selects a method to promote it to the master node based on the preset hot standby node conversion method.

[0015] When the node is a cold standby node, there is no data storage. The method for selecting the best option to switch the cold standby node to a hot standby node is used when the node is converted from a cold standby node to a hot standby node.

[0016] The node is in a fault state when it is offline.

[0017] In one exemplary embodiment of this disclosure, the system includes:

[0018] When a node is a master node, it is used to provide read and write services.

[0019] When the node is a hot standby node, it maintains the internal service operation status to keep the data consistent with the master node.

[0020] In one exemplary embodiment of this disclosure, the system includes:

[0021] The master node and the hot standby node provide read and write services when only the master node is used, through components that satisfy distributed mutual exclusion.

[0022] In one exemplary embodiment of this disclosure, the system includes:

[0023] The master node and the hot standby node also include a preset cache, which is used to save the data written by the client in the cache in chronological order.

[0024] In one exemplary embodiment of this disclosure, the system includes:

[0025] The time-series data written by the client stored in the cache is strongly synchronized to ensure strong consistency of the cached data.

[0026] The time-series data written by the client and stored in the cache is asynchronously saved to the shared storage.

[0027] In one exemplary embodiment of this disclosure, the system includes:

[0028] When the master node or the hot standby node fails, the hot standby node is switched using the optimal method based on the preset cold standby node switching method.

[0029] Secondly, this application discloses a method for preferential selection when switching from a cold standby node to a hot standby node in the system, the method comprising:

[0030] Manually configure weights for each operational dimension metric of each server in the distributed cluster, and continuously monitor each operational dimension metric during runtime;

[0031] When a cold standby node needs to be converted to a hot standby node, the coefficient of variation of each of the above-mentioned operational dimension indicators in the cluster is calculated respectively.

[0032] Based on the coefficient of variation, candidate cold standby nodes are determined, and the expected load value of the index when the candidate cold standby nodes are converted into hot standby nodes is calculated.

[0033] Based on the expected load value of the aforementioned indicators, the coefficient of variation of each operational dimension indicator is calculated when the candidate cold standby node is converted into a hot standby node, thus obtaining the coefficient of variation of the expected load value of each operational dimension indicator when the candidate cold standby node is converted into a hot standby node.

[0034] The coefficient of variation of the expected load value is then weighted and averaged.

[0035] The cold standby node corresponding to the minimum value of the weighted average is selected as the preferred choice for converting to a hot standby node.

[0036] Thirdly, this application discloses an electronic device comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to perform the method as described in any of the preceding aspects.

[0037] Fourthly, this application discloses a non-transitory computer-readable storage medium in which, when the instructions in the storage medium are executed by a processor of an electronic device, enable the electronic device to perform the methods described in any of the preceding aspects.

[0038] Fifthly, this application discloses a computer program product in which, when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the method described in any of the preceding aspects.

[0039] This application provides a highly available hierarchical distributed storage system with strong data consistency and a method for optimal selection when converting a cold standby node to a hot standby node. The hierarchical distributed storage system is based on a shared storage architecture; multiple nodes in the hierarchical distributed storage system share a single memory for data access; the nodes in the hierarchical distributed storage system include four identities: master node, hot standby node, cold standby node, and offline node. When a cold standby node is converted to a hot standby node, a preset optimal selection method is used to convert the cold standby node to a hot standby node. The system disclosed in this application provides a high availability and strong consistency implementation scheme for a distributed storage system based on shared storage. The system ensures strong consistency for an N-node distributed storage system based on shared storage, and availability is satisfied even if at most N-1 nodes fail without affecting system service provision, while guaranteeing low latency for read and write requests. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of node transformation in a highly available hierarchical distributed storage system with strong data consistency, as described in this application.

[0041] Figure 2 This is a schematic diagram of node relationships in an embodiment of a ZooKeeper-based highly available hierarchical distributed storage system with strong data consistency, as described in this application.

[0042] Figure 3 This is a flowchart of the steps of a method for selecting the best backup node when converting to a hot backup node in a highly available hierarchical distributed storage system based on strong data consistency, as described in this application.

[0043] Figure 4 This is a block diagram of an electronic device according to this application.

[0044] Figure 5 This is a block diagram of a computer-readable storage medium according to this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] This application provides a highly available hierarchical distributed storage system with strong data consistency and a method for optimal selection when converting a cold standby node to a hot standby node. The hierarchical distributed storage system is based on a shared storage architecture; multiple nodes in the hierarchical distributed storage system share a single memory for data access; the nodes in the hierarchical distributed storage system include four identities: master node, hot standby node, cold standby node, and offline node. When a cold standby node is converted to a hot standby node, a preset optimal selection method is used to convert the cold standby node to a hot standby node. The system disclosed in this application provides a high availability and strong consistency implementation scheme for a distributed storage system based on shared storage. The system ensures strong consistency for an N-node distributed storage system based on shared storage, and availability is satisfied even if at most N-1 nodes fail without affecting system service provision, while guaranteeing low latency for read and write requests.

[0047] Example 1:

[0048] Reference Figure 1 This diagram illustrates a node transformation schematic of a highly available hierarchical distributed storage system with strong data consistency, as described in this application. Specifically, this system may include:

[0049] The hierarchical distributed storage system is based on a shared storage architecture;

[0050] In the hierarchical distributed storage system, multiple nodes share a single memory to access data.

[0051] The nodes in the hierarchical distributed storage system include four types of identities: master node, hot standby node, cold standby node, and offline node. When a cold standby node is converted to a hot standby node, a preset optimal selection method is used to convert the cold standby node to a hot standby node.

[0052] In this example embodiment, the system includes:

[0053] When a node is a master node, it is used to provide read and write services.

[0054] When the node is a hot standby node, it is used to synchronize data with the master node, and selects a method to promote it to the master node based on the preset hot standby node conversion method.

[0055] When the node is a cold standby node, there is no data storage. The method for selecting the best option to switch the cold standby node to a hot standby node is used when the node is converted from a cold standby node to a hot standby node.

[0056] The node is in a fault state when it is offline.

[0057] In this example embodiment, the system includes:

[0058] When a node is a master node, it is used to provide read and write services.

[0059] When the node is a hot standby node, it maintains the internal service operation status to keep the data consistent with the master node.

[0060] In this example embodiment, the system includes:

[0061] The master node and the hot standby node provide read and write services when only the master node is used, through components that satisfy distributed mutual exclusion.

[0062] In this example embodiment, the system includes:

[0063] The master node and the hot standby node also include a preset cache, which is used to save the data written by the client in the cache in chronological order.

[0064] In this example embodiment, the system includes:

[0065] The time-series data written by the client stored in the cache is strongly synchronized to ensure strong consistency of the cached data.

[0066] The time-series data written by the client and stored in the cache is asynchronously saved to the shared storage.

[0067] In this example embodiment, the system includes:

[0068] When the master node or the hot standby node fails, the hot standby node is switched using the optimal method based on the preset cold standby node switching method.

[0069] Example 2:

[0070] In this example embodiment, the system disclosed herein is based on a distributed storage system using a shared storage architecture. In such a distributed storage system, multiple nodes share a single memory to access data, which can guarantee that the data persisted in the memory is strongly consistent.

[0071] The technical solution proposed in this invention has five essential features:

[0072] 1. Node identities include primary, hot standby, and cold standby. The primary node is responsible for providing services to the outside world; the hot standby node does not provide services to the outside world, but maintains the internal service operation status and keeps the data consistent with the primary node. The hot standby node can switch to the primary node when the primary node fails; the cold standby node does not provide services to the outside world, but maintains the internal service operation status and does not keep the data consistent with the primary node. The cold standby node can switch to the hot standby node when the hot standby node fails.

[0073] 2. Under normal operating conditions, there should be at least one primary and one hot standby node. These nodes are connected by a component that satisfies distributed mutual exclusion to ensure that only the primary node can provide read and write services at any given time.

[0074] 3. Both the primary and hot standby servers have caches. Data written by the client is stored in the cache in chronological order, and strong consistency of the cached data is ensured between the primary and hot standby servers through strong synchronization. The data in the cache will eventually be persisted to the shared storage asynchronously.

[0075] 4. Each node in the system has a mechanism to automatically detect faults and automatically switch node identities after a fault is detected.

[0076] 5. When the number of nodes maintaining cached data consistency is insufficient due to a failure of the primary or hot standby, the system will select at least one cold standby to be converted into a hot standby through an optimal selection algorithm to supplement the number.

[0077] The following section describes in detail the node identities and functions in the technical solution, as well as the state transition process of the state machine.

[0078] There are four node identities: Active, Hot Standby, Cold Standby, and Offline. Their functions are as follows:

[0079] Active: Provides read and write services.

[0080] Hot Standby: Synchronizes data with Active and can be promoted to Active.

[0081] Cold Standby: No data, but in good condition, can be promoted to Hot Standby.

[0082] Offline: Faulty / malfunctioning.

[0083] In this example embodiment, the state transition process of the node state machine is as follows:

[0084] 1. Before the state machine starts and after a fault, the state is Offline;

[0085] 2. After the state machine starts, it performs necessary initialization work. If it fails, it remains in the Offline state and keeps retrying until it succeeds. After success, the state machine state changes to Cold Standby.

[0086] 3. When the state machine is in Cold Standby, it remains in that state until it discovers that it has been "upgraded" in some way, at which point the state changes to Hot Standby;

[0087] 4. When the state machine is in Hot Standby, a component for distributed mutual exclusion is used to compete for the state. The state machine that wins the competition changes to Active, while the state machine that does not win the competition remains in Hot Standby. If it loses the ability to compete, it changes to Cold Standby.

[0088] 5. When the state machine is in the Active state, it remains Active as long as it holds the aforementioned mutually exclusive component and that component is valid; otherwise, the state changes to Hot Standby.

[0089] In this example embodiment, the cached data between the primary and hot standby is sorted chronologically to determine the order in which clients write to the same location, preventing overwrite concurrency errors. When the primary or hot standby fails, unpersisted data in the cache is at risk of loss. After a new primary is established, the cached data needs to be persisted before processing client I / O; otherwise, data loss will occur. Then, the caches of the new primary and hot standby are cleared to restore strong consistency. This method is similar to write-ahead logging, both aiming to reduce disk write operations and lower client I / O latency. The difference is that write-ahead logging typically requires persisting the log content, still requiring disk I / O, while this method only stores it in the cache. The advantage is further reduced latency, but the cost is increased risk of data loss, which can be mitigated by increasing the number of hot standby devices and increasing the frequency of cache persistence.

[0090] A cold standby node is a standby node that does not communicate with the primary or hot standby nodes for data synchronization. Therefore, adding any number of cold standby nodes will not significantly increase the network load between clusters. Cold standby differs from the "observer" commonly found in distributed systems. The latter may provide read services but can never provide write services, and its identity is immutable; a cold standby, however, cannot provide read or write services, but has the ability to provide them after a state transition.

[0091] If the number of hot backups is insufficient, multiple cold backups can be converted into hot backups. The priority of the selected cold backups will be determined by the optimization algorithm. The optimization algorithm considers dimensions including but not limited to system load, custom weights, and user preference configurations.

[0092] In this example embodiment, the availability, consistency, and latency levels of the technical solution proposed by the present invention include:

[0093] Strong consistency: Strong synchronization of caches between primary and hot standby ensures strong consistency of non-persistent data, while the shared storage architecture ensures strong consistency of persisted data.

[0094] High Availability: For a distributed storage system with one primary, N hot standby, and M cold standby nodes, the system can tolerate up to N+M node failures simultaneously without affecting system availability. Data written to the cache by clients may be lost only if the primary and N hot standby nodes fail simultaneously. Here, "simultaneously" means that the time interval between failures is less than the time it takes for the primary or hot standby nodes to detect the failure of the other node and persist the cached data.

[0095] Low latency: Client write requests only need to write to the master's cache and complete strong synchronization across a limited number of links to be considered successful, without requiring disk I / O. The number of synchronization links depends on the number of hot spares. Furthermore, due to the existence of an optimal selection algorithm, hot spares are usually the nodes with the lowest network communication load in the system, further reducing the impact of data synchronization on latency.

[0096] Example 3:

[0097] In the embodiments of this example, as Figure 2 As shown, the distributed consistency open-source software ZooKeeper is used as the basis for this embodiment. ZooKeeper has several features that can effectively reduce the implementation difficulty of the proposed solution. First, there are mature industry solutions that can use ZooKeeper to implement distributed leases, and distributed leases can serve as a tool to achieve mutual exclusion between the primary and hot standby nodes. Second, ZooKeeper provides a temporary node, which requires the client session to remain active to exist. Once the client session loses activity for various reasons, the temporary node will be deleted immediately. This feature can serve as a way for nodes in the cluster to detect faults. Third, ZooKeeper itself can also be used as a lightweight distributed storage system. The data it maintains satisfies sequential consistency, and some distributed data read / write requirements in this embodiment can be directly implemented using ZooKeeper.

[0098] In this example embodiment, the technical solution of the invention is specifically described in conjunction with the state transition process of the state machine.

[0099] 1. Before the state machine starts and after a failure, it loses connection with ZooKeeper, the session expires and the created temporary nodes are automatically deleted. At this time, the external appearance is Offline.

[0100] 2. After the state machine starts, it creates a session with ZooKeeper. Upon successful session creation, a temporary node is created, called the alive node. The main function of the alive node is to indicate the liveness of its own state machine. Then, a permanent node is created, called the authority node. The main function of the authority node is to indicate whether its own state machine is eligible to compete to become the active state machine. After completing the above steps, the state changes to Cold Standby.

[0101] 3. When the state machine is in Cold Standby, upon discovering that authority has been marked with a specific value, it believes it has the right to participate in the competition for Active, and the state changes to Hot Standby;

[0102] 4. When the state machine is in Hot Standby, it uses distributed leases to compete for Active status. The state machine that wins the competition becomes Active, while the state machine that does not win the competition remains in Hot Standby. If the lease becomes active during this period (usually due to losing connection with ZooKeeper), the state becomes Cold Standby.

[0103] 5. When the state machine is in the Active state, it remains Active as long as it holds a lease and the lease is valid; otherwise, the state changes to Hot Standby.

[0104] Both the live node and the distributed lease have similar "volatile" characteristics: when the state machine loses connection with the ZooKeeper cluster, the live node is deleted and the lease expires, which can be considered as a failure of the state machine.

[0105] In summary, this embodiment achieves distributed mutual exclusion between the primary and hot standby through distributed leases based on ZooKeeper, uses Socket Channel to achieve strong synchronization of cached data, implements a mechanism for discovering node failures through the "volatility" of the live node and the distributed lease, and enables the cold standby to switch between a state of not providing service and a state of providing service through the content information of the authority node. The specific solutions are described below.

[0106] In this example embodiment, three questions will be explained in more detail: when and how the content of the authority node is set; how to recover from a possible error state to a normal state when a concurrent failure occurs; and how to design the optimal algorithm for switching from cold standby to hot standby.

[0107] When an authority node is created, its default value is the boolean value `false`, meaning it is not currently eligible to participate in Active state transitions. When all state machines are ineligible, the high-availability cluster's state transitions will stall, and neither primary nor hot standby will occur. There are two possible scenarios where this happens: first, all state machines are starting up for the first time, and all authority nodes contain the default value `false`; second, all state machines with `true` as their authority node value fail simultaneously, leaving only a few state machines with `false` as their authority node value. In either case, third-party intervention is required to authorize at least one state machine to compete for the authority node, thus restarting the high-availability cluster's automatic state transition process.

[0108] The third party is a service independent of the high-availability system, and it is itself highly available. In this embodiment, it can be a primary-secondary high-availability system based on a distributed lease. The third party monitors changes to the live nodes and authority nodes of these state machines via ZooKeeper listener callbacks. When any live node change is detected, the third party performs an availability check on the high-availability cluster. By checking the existence of all live nodes and the content of the authority nodes, it can determine whether the high-availability system still has the ability to transition states. When the authority node content of all state machines with live nodes is false, meaning that all non-faulty state machines are ineligible to participate in Active competition, the high-availability system no longer has the ability to transition states. When the third party detects this scenario, it needs to intervene to restore the system. Specifically, it sets the authority node content of one of the state machines with live nodes to true, allowing that state machine to automatically switch from cold standby to hot standby, and then become the primary, thus restoring the service.

[0109] When neither the primary nor the hot standby fails simultaneously, error recovery can be performed without the involvement of a third party.

[0110] Within the primary state machine process, there exists a communication thread that synchronizes data with the hot standby. This thread also maintains a heartbeat between the primary and hot standby. When this thread detects a heartbeat timeout, the primary considers the hot standby to have failed and must first persist the data. If persistence is successful, the synchronization thread selects a cold standby with an live node, sets its authority node content to true, and the cold standby's state machine automatically converts to a hot standby, restoring the system to a stable state of one primary and one hot standby. If persistence or other steps fail, it indicates a primary failure. The primary state machine releases the distributed lease and converts to a hot standby, potentially transitioning to a cold standby or going offline depending on the impact of the failure. At this point, the high-availability system has two possibilities: one, both the primary and hot standby fail simultaneously (the recovery method for this scenario has been described above); two, the hot standby is not failed or has recovered from the failure, in which case it wins the lease competition and becomes the primary. In this case, the system has at least one primary, similar to the scenario where the primary recovers through the synchronization thread described above.

[0111] In this example embodiment, the heartbeat maintained between the master synchronization thread and the hot standby is also a way to implement a fault detection mechanism. Compared to the method based on ZooKeeper's "volatile" nodes for judgment, the difference lies in whether the fault detection thread and the data synchronization thread are synchronous or asynchronous. The synchronous or asynchronous implementation affects data consistency performance, and distributed storage systems with different consistency requirements can choose different implementation methods.

[0112] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by this application.

[0113] Reference Figure 3 This paper illustrates a flowchart of the steps in a method for optimal selection when converting a cold standby node to a hot standby node in a highly available hierarchical distributed storage system based on strong data consistency, according to this application. The method includes:

[0114] Step S110: Manually configure weights for each operational dimension indicator of each server in the distributed cluster, and continuously monitor each operational dimension indicator during runtime.

[0115] Step S120: When a cold standby node needs to be converted to a hot standby node, calculate the coefficient of variation of each operational dimension indicator in the cluster.

[0116] Step S130: Based on the coefficient of variation, determine the candidate cold standby nodes and calculate the expected value of the index load when converting the candidate cold standby nodes into hot standby nodes.

[0117] Step S140: Calculate the coefficient of variation of each operational dimension indicator when converting the candidate cold standby node to a hot standby node based on the expected load value of the indicator, and obtain the coefficient of variation of the expected load value of each operational dimension indicator when converting the candidate cold standby node to a hot standby node.

[0118] Step S150: Perform a weighted average on the coefficient of variation of the expected load value;

[0119] Step S160: Select the cold standby node corresponding to the minimum value of the weighted average as the preferred choice for converting to a hot standby node.

[0120] In this example embodiment, several operational metrics for each server in the distributed cluster are considered, including but not limited to inbound bandwidth utilization, outbound bandwidth utilization, remaining memory, and CPU utilization. Their weights are manually configured, and these metrics are continuously monitored during runtime. When a cold standby to hot standby scenario occurs, the coefficient of variation (COP) of these metrics in the cluster is calculated. The COP is a dimensionless statistical value, mathematically representing the level of dispersion of a metric's distribution in the cluster. A larger COP indicates a higher degree of dispersion, meaning the current load on that metric in the cluster is more uneven. Then, a candidate cold standby is simulated to be converted to a hot standby, and the expected load value of the metric under this condition is calculated. This yields the expected load values ​​after the conversion of all candidate cold standby simulations. The COP is then calculated again for these expected load values. Mathematically, this represents the impact of different cold standby conversions on the dispersion of that metric's distribution in the cluster. A larger COP indicates that the dispersion of this statistical value is more significantly affected by the cold standby distribution, i.e., how much the choice of cold standby affects the load balancing of this metric. This calculation yields the coefficient of variation for the expected load values ​​of all indicators. Since the coefficient of variation is dimensionless, a weighted average can be calculated to obtain the overall load balance of the cluster after converting a cold standby to a hot standby. Selecting the cold standby with the smallest weighted average, i.e., selecting this cold standby for conversion to a hot standby, will result in the most balanced load distribution among the indicators in the cluster.

[0121] Optionally, this application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0122] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0123] Figure 4 This is a block diagram illustrating an electronic device 800. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0124] Reference Figure 4 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0125] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0126] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, images, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0127] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0128] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0129] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0130] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0131] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0132] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast operation information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0133] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0134] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0135] Figure 5This is a block diagram illustrating a computer-readable storage medium 1900. For example, the computer-readable storage medium 1900 can be provided as a server.

[0136] Reference Figure 5 The computer-readable storage medium 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by memory 1932 for storing instructions executable by the processing component 1922, such as an application program. The application program stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0137] The computer-readable storage medium 1900 may also include a power supply component 1926 configured to perform power management of the computer-readable storage medium 1900, a wired or wireless network interface 1950 configured to connect the computer-readable storage medium 1900 to a network, and an input / output (I / O) interface 1958. The computer-readable storage medium 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0140] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0146] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0147] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A highly available, hierarchical distributed storage system with strong data consistency, characterized in that, The system includes: The hierarchical distributed storage system is based on a shared storage architecture; In the hierarchical distributed storage system, multiple nodes share a single memory to access data. The nodes in the hierarchical distributed storage system include four types of identities: master node, hot standby node, cold standby node, and offline node. When a cold standby node is converted to a hot standby node, a preset optimal selection method is used to convert the cold standby node to a hot standby node.

2. The system as described in claim 1, characterized in that, The system includes: When a node is a master node, it is used to provide read and write services. When the node is a hot standby node, it is used to synchronize data with the master node, and selects a method to promote it to the master node based on the preset hot standby node conversion method. When the node is a cold standby node, there is no data storage. The method for selecting the best option to switch the cold standby node to a hot standby node is used when the node is converted from a cold standby node to a hot standby node. The node is in a fault state when it is offline.

3. The system as described in claim 2, characterized in that, The system includes: When a node is a master node, it is used to provide read and write services. When the node is a hot standby node, it maintains the internal service operation status to keep the data consistent with the master node.

4. The system as described in claim 2, characterized in that, The system includes: The master node and the hot standby node provide read and write services when only the master node is used, through components that satisfy distributed mutual exclusion.

5. The system as described in claim 2, characterized in that, The system includes: The master node and the hot standby node also include a preset cache, which is used to save the data written by the client in the cache in chronological order.

6. The system as described in claim 5, characterized in that, The system includes: The time-series data written by the client stored in the cache is strongly synchronized to ensure strong consistency of the cached data. The time-series data written by the client and stored in the cache is asynchronously saved to the shared storage.

7. The system as described in claim 1, characterized in that, The system includes: When the master node or the hot standby node fails, the hot standby node is switched using the optimal method based on the preset cold standby node switching method.

8. A method for optimal selection when converting a cold standby node to a hot standby node in a highly available hierarchical distributed storage system based on strong data consistency, characterized in that... The method includes: Manually configure weights for each operational dimension metric of each server in the distributed cluster, and continuously monitor each operational dimension metric during runtime; When a cold standby node needs to be converted to a hot standby node, the coefficient of variation of each of the above-mentioned operational dimension indicators in the cluster is calculated respectively. Based on the coefficient of variation, candidate cold standby nodes are determined, and the expected load value of the index when the candidate cold standby nodes are converted into hot standby nodes is calculated. Based on the expected load value of the aforementioned indicators, the coefficient of variation of each operational dimension indicator is calculated when the candidate cold standby node is converted into a hot standby node, thus obtaining the coefficient of variation of the expected load value of each operational dimension indicator when the candidate cold standby node is converted into a hot standby node. The coefficient of variation of the expected load value is then weighted and averaged. The cold standby node corresponding to the minimum value of the weighted average is selected as the preferred choice for converting to a hot standby node.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method of claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in claim 8.

Images