Fusion method and device of hyper-converged system and active-active storage service
By configuring hybrid communication mode and extending service time in a hyperconverged system, the problem that management network failure affects the host access to storage and abnormal nodes cannot be isolated, achieving more stable system operation and highly available storage services.
Patent Information
- Application Number
- CN202510125750.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-12-31
- Filing Date
- 2025-01-26
- Publication Date
- 2025-05-27
AI Technical Summary
In the convergence of the hyperconverged system and the dual-active storage service, the management network failure affects the host's access to the storage, and due to the short pull-up period, the abnormal node cannot be isolated, resulting in the cluster being stuck.
Configure the network communication mode of the cloud file storage domain in the hyperconverged system to be a hybrid communication mode, switch to disk communication mode. When the management network is abnormal, and configure the service pull time of the monitoring component according to the heartbeat time of the system node to ensure that the abnormal node is completely isolated before the service pull is started.
By automatically switching to disk communication mode, the host can access storage normally, and by extending the service pull-up time, ensuring that abnormal nodes are correctly isolated, avoiding cluster jamming, and improving system stability and reliability.
Smart Images

Figure CN120050163A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the computer field, and more specifically, to a method and device for integrating a hyper-converged system with an active-active storage service. Background Art
[0002] With the advent of the cloud computing and big data era, hyper-convergence technology is gradually becoming an important support for the digital transformation of enterprises. It can not only simplify IT operations, improve resource utilization efficiency and management efficiency, but also bring higher flexibility and scalability to enterprises to meet ever-changing business needs.
[0003] Hyper-convergence technology is already quite mature and widely used in the market. At present, the stability and reliability are continuously improved and optimized on the basis of ensuring the basic hyper-convergence functions. Among them, the reliability of hyper-convergence technology is mainly reflected in its architecture design, data protection, high availability and fault tolerance mechanism. Hyper-convergence technology achieves high availability and fault tolerance mechanism by deploying redundant backup of multiple nodes and components. When a node or component fails, the system can automatically switch the business to other normal nodes or components to ensure the continuity and stability of the business.
[0004] However, when the hyper-converged system is integrated with the active-active storage service, the host management network and the virtual machine business network use the same Gigabit network port, which will compete with each other for network traffic bandwidth. During the peak period of cloud platform business, business access may be delayed and stuck, which will affect the customer's business experience. The "CFS network communication mode" relies solely on the stability of the management network. Once the management network becomes unstable, the host will not be able to access the backend storage normally, and the business virtual machine may crash. The heartbeat disk timeout logic of the cfs storage domain (cloud file storage domain) of the hyper-converged system is handled according to the default scenario of 120s, and the timeout configuration is changed to 300S. In the "CFS network communication mode", when a node in the cluster is disconnected from the network, other nodes will perform fence isolation (i.e. kick out of the cluster) on the abnormal node at the 300th second of the disconnection. However, the storagemonitor monitoring service will pull the service again in 90 seconds, affecting the cluster lock. The abnormal node cannot be fenced, and eventually the cluster is stuck. Summary of the invention
[0005] The embodiments of the present application provide a method and device for integrating a hyper-converged system with an active-active storage service, so as to at least solve the problem in the related art that a management network failure affects the host's access to storage, and that the cluster lock is affected due to a too short startup cycle, and abnormal nodes cannot be isolated, ultimately causing the cluster to be stuck.
[0006] According to an embodiment of the present application, a method for integrating a hyper-converged system and an active-active storage service is provided, including:
[0007] Configure the network communication mode of the cloud file storage domain in the hyper-converged system to the hybrid communication mode. In the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode. Configure the service startup time of the monitoring component in the hyper-converged system according to the heartbeat time of the system node. Based on the network communication mode of the cloud file storage domain and the service startup time of the monitoring component, the hyper-converged system and the active-active storage service are integrated to obtain a converged system.
[0008] In an exemplary embodiment, the converged system includes: a storage domain, a computing domain and a network domain, wherein the storage domain includes at least a CFC storage domain and an active-active storage service, the computing domain includes multiple computing nodes, and the network domain includes a management network and a business network.
[0009] In an exemplary embodiment, the method further comprises:
[0010] Configure physical network interfaces for the management network and the business network respectively. The management network corresponds to the management network interface, and the business network corresponds to the business network interface. The management network is isolated from the business network.
[0011] In an exemplary embodiment, when the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode, including: determining the fault state of the management network according to a preset threshold and the heartbeat signal of the management network, wherein the preset threshold is determined based on the network environment in which the converged system operates and the average delay of communication between system nodes, and when the update time of the heartbeat signal of the management network is greater than the preset threshold, the fault state of the management network is judged to be faulty; in response to the fault state of the management network indicating that the management network is faulty, the communication mode of the cloud file storage domain is switched from the network communication mode to the disk communication mode, wherein the disk communication mode refers to the cloud file storage domain communicating data between nodes by directly accessing the shared storage disk; after switching to the disk communication mode, real-time monitoring of the fault state of the management network is performed; in response to the fault state of the management network changing from faulty to normal, the communication mode of the cloud file storage domain is changed from the disk communication mode to the network communication mode, wherein when the update time of the heartbeat signal of the management network is less than or equal to the preset threshold, the fault state of the management network is judged to be changed from faulty to normal.
[0012] In an exemplary embodiment, the service startup time of the monitoring component is set to 1.5 times the heartbeat time of the system node.
[0013] In an exemplary embodiment, the method further includes: setting the priority of the disk communication mode in the mixed communication mode to be lower than the priority of the network communication mode.
[0014] According to another embodiment of the present application, a fusion device of a hyper-converged system and an active-active storage service is provided, including:
[0015] The first configuration module is used to configure the network communication mode of the cloud file storage domain in the hyper-converged system to a hybrid communication mode, wherein in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode; the second configuration module is used to configure the service startup time of the monitoring component in the hyper-converged system according to the heartbeat time of the system node; the fusion module is used to fuse the hyper-converged system and the active-active storage service to obtain a fusion system based on the network communication mode of the cloud file storage domain and the service startup time of the monitoring component.
[0016] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0017] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0018] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0019] By configuring the network communication mode of the cloud file storage domain in the hyper-converged system as a hybrid communication mode in this application, wherein, in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode; according to the heartbeat time of the system node, the service pull-up time of the monitoring component in the hyper-converged system is configured; based on the network communication mode of the cloud file storage domain and the service pull-up time of the monitoring component, the hyper-converged system and the active-active storage service are integrated to obtain a fusion system. By configuring the network communication mode of the cloud file storage domain as a hybrid communication mode, when the management network is unstable, the CFS storage domain will automatically switch to the disk communication mode. At this time, the entire cluster will rely on the FC storage to ensure normal communication, thereby ensuring that the host can still access the storage normally. At the same time, through the configuration of the service pull-up time, it is ensured that the service is pulled only after the abnormal node is completely isolated from the cluster. This can solve the problem that the management network failure affects the host's access to storage and the cluster lock is affected due to the short pull-up cycle. The abnormal node cannot be isolated, and the cluster is finally in a stuck state, thereby achieving a more stable operation of the fused system. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a hardware structure block diagram of a server device of a method for integrating a hyper-converged system and active-active storage services according to an embodiment of the present application;
[0021] Figure 2 is a flow chart of a method for integrating a hyper-converged system and an active-active storage service according to an embodiment of the present application;
[0022] Figure 3 It is a structural block diagram of a fusion device of a hyper-converged system and an active-active storage service according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0025] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a method for integrating a hyper-converged system and active-active storage services in an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for integrating the hyper-converged system and the active-active storage service in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0028] It should be noted that the normal operation of OCFS2 (shared file system) depends on the communication between nodes. When the management network is abnormal, the communication between nodes is blocked, which leads to the freezing of virtual machines. For abnormal node failures, InCloud Sphere's OCFS2 supports the quorum mechanism, which can remove abnormal nodes from the cluster to ensure that the remaining nodes in the cluster can still work normally.
[0029] The working principle of OCFS2's quorum mechanism: When the communication between nodes is abnormal for more than 30 seconds, the quorum process will be triggered, and all nodes will begin to monitor the heartbeat updates of other nodes in the cluster. When it is found that the heartbeat of a node has stopped updating for O2CB_HEARTBEAT_THRESHOLD (heartbeat timeout), the node will evaluate whether to perform a fence operation. Specifically, if a node finds that the number of nodes connected to it is less than half, the node will actively stop the heartbeat and remove itself from the cluster; other normal nodes will also begin to find that the heartbeat of the abnormal node has stopped updating, and will continue to wait for O2CB_HEARTBEAT_THRESHOLD before kicking the abnormal node out of the cluster.
[0030] When "management network + business network integration" and "CFS network communication mode" are used, the corresponding CFS storage domain configuration heartbeat timeout is 300s. In the current OCFS2 abnormal node fence umount policy implementation, when a node is removed from the cluster, it will periodically try to re-start the storage domain service. The period is greater than 2 minutes but less than 5 minutes. If the management network is abnormal during this period, it will cause a stuck problem.
[0031] The problem process is as follows:
[0032] 1. Starting from the time the network cable is unplugged, at 300+30s, the node (A) that unplugs the network cable starts to execute fence umount to remove itself from the cluster. At the same time, the normal nodes start counting 300s, and A is removed only after the timer expires.
[0033] 2. At 420s, that is, about 90s after the fence umount of node A is successful, storagemonitor starts to pull up the o2cb service. This operation will restart the heartbeat of A, causing equal to return to 0, which will interfere with B's counting of A's heartbeat stop:
[0034] 3. It is expected that the logically normal nodes will remove node A from the cluster in about 630 seconds. However, the storagemonitor service will continue to initiate the o2cb service action with a period of about 90 seconds: A can neither join the cluster because other nodes are still waiting for A's heartbeat to time out and remove it, nor can other nodes remove A because A's continuous heartbeat action interferes and affects the cluster lock, causing the cluster to be stuck.
[0035] In this embodiment, a method for integrating a hyper-converged system and an active-active storage service is provided. Figure 2 is a flow chart of a method for integrating a hyper-converged system and an active-active storage service according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:
[0036] Step S201, configuring the network communication mode of the cloud file storage domain in the hyper-converged system to be a hybrid communication mode, wherein, in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode;
[0037] Step S202, configuring the service startup time of the monitoring component hyper-converged into the system according to the heartbeat time of the system node;
[0038] Step S203: Based on the network communication mode of the cloud file storage domain and the service startup time of the monitoring component, the hyper-converged system and the active-active storage service are merged to obtain a converged system.
[0039] Specifically, in a hyper-converged system, the communication mode of the cloud file storage domain (CFS storage domain) can usually be configured as a "network communication mode" that relies only on the management network, or a more flexible "hybrid communication mode". In the hybrid communication mode, the CFS storage domain supports the use of both the management network and other communication paths (such as FC storage network, business network, or direct disk access) for data communication between nodes. When an abnormality occurs in the management network, the CFS storage domain should automatically detect the abnormality and switch the network communication mode to the "disk communication mode", that is, maintain communication between nodes by directly accessing the shared disk. This switching process needs to be seamless and fast to avoid affecting business continuity and data access.
[0040] To achieve this function, the system needs to preset the logic of heartbeat detection and fault switching. The heartbeat detection mechanism is used to monitor the heartbeat signal of the management network. When it is detected that the heartbeat signal stops updating and exceeds the preset threshold, the communication mode switch is triggered. The setting of the preset threshold should take into account the network delay and the maximum time of abnormal communication between nodes to ensure timely response without misjudgment when the management network fails.
[0041] The monitoring component is responsible for monitoring the status of storage services in the hyper-converged system. The configuration of its service startup time should be coordinated with the heartbeat time of the system node to avoid misoperation under abnormal circumstances. Under the default configuration, StorageMonitor will try to restart the service before the heartbeat timeout. This will cause the cluster lock resources to be disturbed when the management network is unstable, so that the abnormal node cannot be properly isolated, which in turn affects the stability and reliability of the entire system.
[0042] To optimize this problem, the service startup time of StorageMonitor should be configured to 1.5 times the heartbeat timeout. This means that when the heartbeat of a node stops updating for more than the preset heartbeat timeout, StorageMonitor will not immediately try to restart the service, but wait for an additional period of time (i.e. 0.5 times the heartbeat timeout) to ensure that the abnormal node has been completely fenced. This time setting should ensure that the status of the abnormal node is accurately judged and processed after the heartbeat timeout caused by the management network failure, while not affecting the operation of other normal nodes in the cluster.
[0043] The key to integrating hyperconverged systems and active-active storage services is to ensure that the communication modes and monitoring mechanisms of the two work together to improve the overall stability and reliability of the system. In the "hybrid communication mode", the CFS storage domain can automatically switch to the disk communication mode when the management network fails to maintain data access and business continuity. At the same time, by optimizing the service startup time of StorageMonitor, it is ensured that no misoperation occurs during the fence isolation process of abnormal nodes.
[0044] The fusion process involves the following key steps:
[0045] Communication mode synchronization: Ensures the compatibility of the communication mode of the CFS storage domain with the active-active storage service so that when the management network fails, the active-active storage service can seamlessly communicate with the CFS storage domain through the disk communication mode.
[0046] Monitoring component configuration: Configure StorageMonitor to adapt to the new network communication mode, including but not limited to the adjustment of service startup time and heartbeat detection threshold, and coordination with the CFS storage domain and active-active storage service status synchronization mechanism.
[0047] Fault recovery strategy: Develop a detailed fault recovery strategy, including fence isolation of abnormal nodes, protection of cluster lock resources, and smooth switching of communication modes after the management network is restored. The fault recovery strategy should ensure business continuity and data consistency while minimizing the impact of fault handling on system performance.
[0048] After completing the above configuration and integration, the fusion system should be strictly tested, including simulated management network failures, network stress testing during business peaks, and real-time monitoring and fault recovery drills to verify the effectiveness and stability of the optimization plan. Through this integration, the hyper-converged system can more effectively respond to network failures, improve the high availability and fault tolerance of storage services, and provide users with more stable and reliable data storage and access services.
[0049] By configuring the network communication mode of the cloud file storage domain in the hyper-converged system as a hybrid communication mode in this application, wherein, in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode; according to the heartbeat time of the system node, the service pull-up time of the monitoring component in the hyper-converged system is configured; based on the network communication mode of the cloud file storage domain and the service pull-up time of the monitoring component, the hyper-converged system and the active-active storage service are integrated to obtain a fusion system. By configuring the network communication mode of the cloud file storage domain as a hybrid communication mode, when the management network is unstable, the CFS storage domain will automatically switch to the disk communication mode. At this time, the entire cluster will rely on the FC storage to ensure normal communication, thereby ensuring that the host can still access the storage normally. At the same time, through the configuration of the service pull-up time, it is ensured that the service is pulled only after the abnormal node is completely isolated from the cluster. This can solve the problem that the management network failure affects the host's access to storage and the cluster lock is affected due to the short pull-up cycle. The abnormal node cannot be isolated, and the cluster is finally in a stuck state, thereby achieving a more stable operation of the fused system.
[0050] The execution subject of the above steps may be a server, a terminal, etc., but is not limited thereto.
[0051] The execution order of step S201 and step S202 is interchangeable, that is, step S202 may be executed first, and then step S2021.
[0052] In an exemplary embodiment, the converged system includes: a storage domain, a computing domain and a network domain, wherein the storage domain includes at least a CFC storage domain and an active-active storage service, the computing domain includes multiple computing nodes, and the network domain includes a management network and a business network.
[0053] Specifically, the storage domain is one of the core components of the converged system and is responsible for data storage and management. Among them, the CFS (Cloud File Storage) domain and the active-active storage service are key services in the storage domain:
[0054] CFS storage domain: Based on distributed file storage technology, it provides scalable shared file storage services. In the optimization scheme of the present invention, the communication mode of the CFS storage domain is configured as "hybrid communication mode", which means that when the management network fails, the CFS storage domain can automatically switch to the disk communication mode and directly communicate data between nodes through the storage disk to maintain business continuity and data access stability.
[0055] Active-active storage service: refers to two or more storage systems providing services at the same time. When one of the systems fails, the other system can seamlessly take over the business, ensuring high data availability and continuity of storage services. The combination of active-active storage service and CFS storage domain further improves the redundancy and fault tolerance of the hyper-converged system at the storage level.
[0056] The computing domain consists of multiple computing nodes, each of which usually contains a processing unit (such as a CPU), memory, and local storage resources for running business loads such as virtual machines and containers. The computing domain is the part of the hyper-converged system responsible for computing-intensive tasks. It communicates with the storage domain through the network domain, dynamically allocates and schedules resources to meet the flexibility and scalability of business needs.
[0057] The network domain is the part responsible for communication in the converged system. It is divided into two independent network planes: the management network and the business network:
[0058] Management network: A network used for system management, configuration, monitoring, and maintenance operations to ensure the management and control signal transmission of the hyper-converged system. The stability and security of the management network are critical to the overall operation of the system. In the optimization solution, the management network and the business network are physically or logically separated to improve the redundancy and reliability of the network.
[0059] Business network: A network that carries business data and application communications, such as communications between virtual machines and user access services. The design of the business network must take into account high bandwidth, low latency, and high reliability to ensure that the performance and stability of business data transmission are not affected by management operations.
[0060] The architecture design of the converged system aims to provide a unified, highly available IT resource pool through close integration and collaboration between the computing domain, storage domain, and network domain. Among them, the CFS storage domain and active-active storage service of the storage domain ensure the efficiency and security of data storage; the computing domain provides dynamic allocation and load balancing of computing resources; and the network domain is responsible for ensuring high efficiency and trouble-free communication within the system and with external networks. By separating the management network and the business network and optimizing the communication mode of the CFS storage domain, the converged system not only improves business continuity in the event of a management network failure, but also enhances the stability and reliability of the overall architecture, providing users with a safer and more efficient service experience.
[0061] In an exemplary embodiment, the method further includes: configuring physical network interfaces for the management network and the service network respectively, wherein the management network corresponds to the management network interface, the service network corresponds to the service network interface, and the management network is isolated from the service network.
[0062] Specifically, in the network architecture of the hyper-converged system, in order to enhance the stability and security of the system, the management network and the business network are designed as two independent network planes, and this separation is achieved by configuring physical network interfaces separately. The specific explanation is as follows:
[0063] The management network interface is a physical network interface on each computing node that is dedicated to management network communications. It is connected to the management network switch or router and only allows management traffic, such as configuration updates, status monitoring, system maintenance signals, etc. to pass through. The configuration of the management network interface usually includes setting an independent IP address, subnet mask, and default gateway to ensure that management operations can be performed in a closed network environment without being affected by the business network.
[0064] The business network interface is another physical network interface on the computing node used to carry business data and application communication. It is connected to the business network switch or router and is responsible for communication between virtual machines, user access services, and transmission of business data. The configuration of the business network interface also includes an independent IP address, subnet mask, and default gateway to support high-bandwidth and low-latency data communication requirements and ensure high performance and continuity of business.
[0065] It can be understood that the isolation of the management network and the business network means that the two network planes communicate independently through different physical network interfaces and network devices, and there is no direct physical connection or data channel between them, thereby preventing management operations or management network failures from affecting the business network and business data transmission.
[0066] In an exemplary embodiment, when the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode, including: determining the fault state of the management network according to a preset threshold and the heartbeat signal of the management network, wherein the preset threshold is determined based on the network environment in which the converged system operates and the average delay of communication between system nodes, and when the update time of the heartbeat signal of the management network is greater than the preset threshold, the fault state of the management network is judged to be faulty; in response to the fault state of the management network indicating that the management network is faulty, the communication mode of the cloud file storage domain is switched from the network communication mode to the disk communication mode, wherein the disk communication mode refers to the cloud file storage domain communicating data between nodes by directly accessing the shared storage disk; after switching to the disk communication mode, real-time monitoring of the fault state of the management network is performed; in response to the fault state of the management network changing from faulty to normal, the communication mode of the cloud file storage domain is changed from the disk communication mode to the network communication mode, wherein when the update time of the heartbeat signal of the management network is less than or equal to the preset threshold, the fault state of the management network is judged to be changed from faulty to normal.
[0067] Specifically, in a hyper-converged system, the health status of the management network is monitored through a heartbeat signal, which is a signal sent periodically to confirm the connection status and activity between nodes. The system presets a threshold, which is determined based on the network environment in which the converged system operates and the average delay in communication between system nodes. It is usually optimized through a large number of tests and data analysis. When the update time of the heartbeat signal of the management network exceeds this preset threshold, the system will determine that the management network has entered a fault state. The setting of the preset threshold is a balance point. It needs to be long enough to avoid normal network fluctuations being mistakenly judged as faults, but also short enough to ensure timely response when a real network failure occurs.
[0068] Once the system detects the fault status of the management network and confirms that the fault status is a fault, the communication mode of the cloud file storage domain (CFS storage domain) will automatically switch from "network communication mode" to "disk communication mode". In disk communication mode, the CFS storage domain no longer relies on the management network for communication between nodes, but instead directly accesses the shared storage disk to complete data transmission and synchronization operations. This switching mechanism ensures that even in the event of a management network failure, data access and business continuity will not be affected, thereby improving the overall stability of the system.
[0069] Even after switching to disk communication mode, the hyper-converged system still needs to monitor the fault status of the management network in real time so that it can switch back to network communication mode in time when the management network is restored. This monitoring mechanism includes but is not limited to continuous heartbeat signal detection, health check of network connection, and regular update of configuration and status information related to the management network. The purpose of monitoring is to ensure that the system can respond to changes in network status in a timely manner, whether it is a fault or recovery.
[0070] When the fault status of the management network changes from faulty to normal, that is, when the update time of the heartbeat signal of the management network returns to less than or equal to the preset threshold, the system will automatically determine that the fault status of the management network has changed from faulty to normal. At this time, the communication mode of the cloud file storage domain will change from disk communication mode to network communication mode, and the management network will be restored as the main communication path. Before switching back to the network communication mode, the system will perform a series of verification operations, including another heartbeat signal detection, network connection test, and data consistency check to ensure that the switching process will not cause any adverse effects on the data, and to ensure that all nodes can communicate normally through the management network. This switching process is smooth and efficient, which can ensure business continuity while restoring the full functionality of the management network.
[0071] Through this mechanism, the hyper-converged system can automatically adjust the communication mode of the storage domain when the management network is abnormal, thereby ensuring the stable operation of the business and high availability of data. When the management network is restored, the system can quickly switch back to the network communication mode, restore normal management operations and optimize network performance, realize automatic isolation and recovery of network failures, and enhance the overall robustness of the system and user experience.
[0072] In an exemplary embodiment, the service startup time of the monitoring component is set to 1.5 times the heartbeat time of the system node.
[0073] Specifically, the heartbeat time of a system node refers to the time interval at which a node regularly sends a heartbeat signal to the cluster. The heartbeat signal is used to confirm the active status of the node and the stability of the network connection. The preset heartbeat time is usually determined based on the system design, network environment, and reliability requirements. For example, in the case in the technical briefing, the heartbeat timeout is set to 300 seconds.
[0074] Service startup time refers to the waiting time for the monitoring component to restart or restore the service after detecting a service anomaly or a node being offline. In the optimization plan, the service startup time of the monitoring component is set to 1.5 times the heartbeat time of the system node, which is about 450 seconds (1.5 times 300 seconds). This setting is based on the following considerations:
[0075] Avoid resource conflicts: If the monitoring component attempts to restart the service before the heartbeat timeout, it may conflict with the ongoing fence isolation operation, resulting in the abnormal node being unable to be properly isolated, thus affecting the stability and data consistency of the entire cluster.
[0076] Ensure fence operation is complete: Setting the service startup time to 1.5 times the heartbeat time can ensure that the monitoring component will not immediately try to restart the service after the heartbeat timeout caused by the management network failure. In this way, the abnormal node has enough time to be completely fenced by the cluster, ensuring accurate judgment and processing of the node status.
[0077] Improve system stability: By extending the service startup time, the monitoring component can more accurately identify and respond to network failure recovery situations, avoiding misoperation when the management network is unstable, thereby improving the overall stability and reliability of the system.
[0078] Through this optimization, the monitoring component's response strategy when dealing with network failures is more reasonable, which not only ensures the timely processing of abnormal nodes, but also avoids unnecessary resource conflicts and misoperations. It plays an important role in improving the fault tolerance, data security and business continuity of the hyper-converged system.
[0079] In an exemplary embodiment, the method further includes: setting the priority of the disk communication mode in the mixed communication mode to be lower than the priority of the network communication mode.
[0080] Specifically, in a hyper-converged system, the hybrid communication mode is used to solve the problem that when the management network fails, the cloud file storage domain (CFS storage domain) can still maintain communication between nodes to ensure business continuity and data access stability. The hybrid communication mode includes two communication modes: network communication mode and disk communication mode. Among them, the network communication mode is usually used as the main communication mode, while the disk communication mode is used as a backup or fault recovery mechanism.
[0081] The priority of disk communication mode is set lower than that of network communication mode. The purpose of this mechanism is to give priority to network communication mode for data communication between nodes under normal operation, and use disk communication mode as a backup communication mode when the management network fails. The specific explanation is as follows:
[0082] The network communication mode has a high priority. The network communication mode uses the management network or business network for data transmission. It has the advantages of high speed, low latency and large bandwidth, and is suitable for daily business operations and data access. When the system is operating normally, communication between nodes should be carried out through the network communication mode to maximize communication efficiency and performance. This is the basis for the hyper-converged system to achieve efficient and fast resource scheduling and data access.
[0083] The disk communication mode has a low priority. The disk communication mode directly accesses the shared storage disk for data communication. The communication efficiency of this mode is usually lower than that of the network communication mode, especially in a large-scale cluster environment, which may be limited by the disk I / O performance and the concurrency of disk access. However, the disk communication mode provides communication redundancy in the event of a management network failure, ensuring that even if the management network or network communication mode is unavailable, the nodes can still maintain necessary data synchronization and communication through the shared storage disk, thereby ensuring business continuity.
[0084] In hybrid communication mode, the system will automatically select the communication mode according to the network status. When the management network is stable, the system will give priority to the network communication mode; once the management network fails, the system will automatically detect the fault status and switch the communication mode to the disk communication mode to maintain communication between nodes. After the fault is recovered, the system will re-evaluate the network status and automatically switch the communication mode back to the network communication mode to restore efficient communication.
[0085] By setting the disk communication mode to have a lower priority than the network communication mode, the hyper-converged system can maintain high-performance communication under normal circumstances, while providing necessary communication redundancy when the management network fails, thereby ensuring system stability and business continuity.
[0086] In addition to separating the management network and the business network to improve the reliability of the network communication mode, the present invention can be further expanded to cope with more complex and multi-layered network failure scenarios. For example, additional network planes can be introduced, such as a backup network, to provide an emergency communication path when both the main management network and the business network fail. The details are as follows:
[0087] Backup network configuration: Pre-configure a low-bandwidth, high-latency but high-reliability network, such as a satellite network or a backup fiber optic network, as a last resort for emergency communications.
[0088] Automatic switching logic: When it is detected that both the management network and the business network are unavailable, the system automatically triggers the activation of the backup network to maintain minimum system communication and ensure the operation of key businesses.
[0089] In order to solve the traffic bandwidth problem caused by the competition between the management network and the business network during the peak business period, intelligent network traffic control mechanisms such as DSCP (Differentiated Services Code Point) marking and priority queues can be introduced to optimize the allocation of network resources. The details are as follows:
[0090] DSCP marking and priority: Different DSCP markings and priorities are assigned to management network traffic and business network traffic based on the packet type and importance, ensuring that management network traffic can still maintain priority under high load and avoiding the impact of business network traffic on management operations.
[0091] Dynamically adjust thresholds: Based on real-time network monitoring data, dynamically adjust preset thresholds to adapt to changes in the network environment and ensure that network fault status can be accurately determined under different business loads.
[0092] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0093] In this embodiment, a fusion device of a hyper-converged system and an active-active storage service is also provided, which is used to implement the above embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0094] Figure 3 is a structural block diagram of a fusion device of a hyper-converged system and active-active storage service according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:
[0095] The first configuration module 22 is used to configure the network communication mode of the cloud file storage domain in the hyper-converged system to a hybrid communication mode, wherein in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode; the second configuration module 24 is used to configure the service startup time of the monitoring component in the hyper-converged system according to the heartbeat time of the system node; the fusion module 26 is used to fuse the hyper-converged system and the active-active storage service to obtain a fusion system based on the network communication mode of the cloud file storage domain and the service startup time of the monitoring component.
[0096] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0097] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.
[0098] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0099] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0100] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0101] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0102] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0103] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.
[0104] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0105] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0106] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for integrating a hyper-converged system with an active-active storage service, characterized in that: include: Configuring the network communication mode of the cloud file storage domain in the hyper-converged system to be a hybrid communication mode, wherein in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode; According to the heartbeat time of the system node, configure the service startup time of the monitoring component in the hyper-converged system; Based on the network communication mode of the cloud file storage domain and the service startup time of the monitoring component, the hyper-converged system and the active-active storage service are merged to obtain a fusion system.
2. The method according to claim 1, characterized in that The fusion system includes: a storage domain, a computing domain and a network domain, wherein the storage domain includes at least a CFC storage domain and an active-active storage service, the computing domain includes multiple computing nodes, and the network domain includes a management network and a service network.
3. The method according to claim 2, characterized in that The method further comprises: Physical network interfaces are configured for the management network and the service network respectively, wherein the management network corresponds to a management network interface, the service network corresponds to a service network interface, and the management network is isolated from the service network.
4. The method according to claim 1, characterized in that When the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode, including: Determine the fault state of the management network according to a preset threshold and the heartbeat signal of the management network, wherein the preset threshold is determined based on the network environment in which the fusion system operates and the average delay of communication between system nodes, and when the update time of the heartbeat signal of the management network is greater than the preset threshold, determine that the fault state of the management network is a fault; In response to the failure state of the management network indicating that the management network is faulty, switching the communication mode of the cloud file storage domain from a network communication mode to a disk communication mode, wherein the disk communication mode refers to the cloud file storage domain performing data communication between nodes by directly accessing a shared storage disk; After switching to the disk communication mode, real-time monitoring of the fault status of the management network; In response to the fault state of the management network changing from faulty to normal, the communication mode of the cloud file storage domain is changed from a disk communication mode to a network communication mode, wherein when the update time of the heartbeat signal of the management network is less than or equal to the preset threshold, it is determined that the fault state of the management network has changed from faulty to normal.
5. The method according to claim 1, characterized in that The service startup time of the monitoring component is set to 1.5 times the heartbeat time of the system node.
6. The method according to claim 1, characterized in that The method further includes: setting the priority of the disk communication mode in the mixed communication mode to be lower than the priority of the network communication mode.
7. A fusion device of a hyper-converged system and active-active storage service, characterized in that: include: A first configuration module is used to configure the network communication mode of the cloud file storage domain in the hyper-converged system to be a hybrid communication mode, wherein in the hybrid communication mode, if the management network corresponding to the hyper-converged system is abnormal, the network communication mode of the cloud file storage domain is switched to the disk communication mode; A second configuration module is used to configure the service startup time of the monitoring component in the hyper-converged system according to the heartbeat time of the system node; A fusion module is used to fuse the hyper-converged system and the active-active storage service to obtain a fusion system based on the network communication mode of the cloud file storage domain and the service startup time of the monitoring component.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 6 when executed by a processor.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.