A method and system for network element high availability based on shared storage and heartbeat detection

Abstract

The application relates to a network element high-availability method and system based on shared storage and heartbeat detection. A master network element node obtains a distributed lock in a shared storage module through an atomic operation, the TTL of the lock is 200 milliseconds, and the lock is regularly renewed at a period less than the TTL to maintain the validity of the lock. A backup network element node continuously monitors heartbeat messages. When heartbeat messages are continuously lost for a preset number of times or a service health state exception is detected, it is determined that the master network element node fails and a fault detection is triggered. Based on the fault detection, the backup network element node initiates an atomic preemption operation to the shared storage module at a frequency higher than the TTL of the lock, attempts to obtain the distributed lock. If the distributed lock preemption succeeds, the backup network element node immediately triggers a VIP management module to drift a virtual IP to the backup network element node through a VRRP protocol. After the virtual IP drift is completed, the backup network element node takes over service traffic and becomes a new master node, and performs the master node responsibilities of steps S101 to S102.

Description

Technical Field

[0001] This application relates to the field of information and data technology, specifically to a method and system for high availability of network elements based on shared storage and heartbeat detection. Background Technology

[0002] The 4G and 5G core networks carry critical communication services, and the network elements (SMF - Session Management Function, AMF - Access and Mobility Management Function, UDM - Unified Data Management) require extremely high availability (HA). Any single point of failure may lead to widespread service interruption, affecting user experience and operator reputation.

[0003] In existing technologies, traditional HA solutions mainly employ two approaches:

[0004] Primary and backup heartbeat detection and failover: The primary and backup network element instances detect each other's status through periodic heartbeat signals. When the primary node fails (heartbeat timeout), the backup node takes over the service. The advantage of this method is that fault detection is relatively fast (heartbeat interval is usually in the second or sub-second range) and it is relatively simple to implement.

[0005] Shared storage method: The master and backup nodes compete for a "lock" in the shared storage (database, distributed storage system) to determine who has control. The node that obtains the lock becomes the master node. The advantage of this method is that it can effectively prevent "split-brain" (i.e., both nodes think they are the master node) and has high reliability.

[0006] The core objective of this invention is to significantly shorten the primary / standby switchover time while effectively preventing the "split-brain" phenomenon. In the primary / standby state switchover decision, it not only considers the node's liveness status but also comprehensively considers the actual health status and load information of the business instance, optimizes the lock contention mechanism, reduces unnecessary lock operation overhead during normal system operation, and avoids unnecessary jitter in the primary / standby state due to network jitter or lock renewal issues. Summary of the Invention

[0007] To address the problems existing in the prior art, this application aims to provide a method and system for high availability of network elements based on shared storage and heartbeat detection.

[0008] The network element high availability method based on shared storage and heartbeat detection described in this application includes:

[0009] S101. The primary network element node sends a compressed heartbeat message to the backup network element node via the UDP protocol at intervals not exceeding 500 milliseconds. The heartbeat message contains service health status and load data.

[0010] S102. The main network element node acquires a distributed lock in the shared storage module through atomic operations. The lock has a TTL of 200 milliseconds and is periodically renewed at a period less than the TTL to maintain the lock's validity.

[0011] S103. The backup network element node continuously monitors the heartbeat messages. When a preset number of heartbeat messages are lost consecutively or an abnormal service health status is detected, the primary network element node is determined to be faulty and fault detection is triggered.

[0012] S104. Based on the fault detection, the backup network element node initiates an atomic preemption operation to the shared storage module at a frequency higher than the lock TTL, attempting to acquire the distributed lock.

[0013] S105. If the distributed lock preemption is successful, the backup network element node immediately triggers the VIP management module to migrate the virtual IP to its own node through the VRRP protocol;

[0014] S106. After the virtual IP migration is completed, the backup network element node takes over the service traffic and becomes the new master node, performing the master node responsibilities of steps S101 to S102.

[0015] Preferably, in step S101, the primary network element sends a compressed heartbeat message via the UDP protocol. After decoding, the backup node obtains the service health status and load data. If the health status is lower than the threshold, a switching command is immediately sent to the backup node to trigger the primary-backup switch. During the switch, the backup node parses the new heartbeat data and calculates the load change trend by comparing the new and old load values.

[0016] When the trend value exceeds the preset threshold, the allocation algorithm is invoked to dynamically adjust the resource allocation of the primary and backup nodes, generate a new resource configuration, and based on this configuration, the system updates the compression encoding parameters (Gzip compression ratio) of the heartbeat message in real time, generates an optimized heartbeat packet, and sends it back to the backup node via UDP.

[0017] Preferably, in step S102, the main network element acquires the distributed lock through atomic operations. If successful, it starts the renewal mechanism (executes a Lua script to renew every 150ms) to ensure that the lock remains valid. Based on the lock validity, it extracts the associated task scheduling data, generates task allocation instructions, obtains the queue to be executed from the shared storage, sorts the tasks according to priority, and dynamically adjusts the computing resources according to the execution order by calling the allocation algorithm to form a resource allocation configuration.

[0018] Update the task scheduling parameters of the distributed lock, generate optimized scheduling instructions, and distribute them to the main network element nodes through shared storage.

[0019] Preferably, in step S103, the standby node receives a 37-byte data packet through a UDP port to directly identify the health status of the master node;

[0020] Connecting to the window detection workflow: Based on a 300ms time window cyclic queue, when there are no new packets in 100ms and the number of packets in the window is less than 3, it is determined that the heartbeats are lost continuously. This result triggers a two-condition response in the fault determination workflow: if the heartbeats are lost ≥3 times or the flag bit is non-zero, fault handling is started.

[0021] Subsequently, in the storage confirmation workflow, a fault flag with a 500ms TTL is written to Redis. This operation is associated with the anti-false positive workflow: a 150ms secondary verification mechanism reduces the false positive rate to below 0.1%.

[0022] Ultimately, the workflow is optimized through caching: the local heartbeat status table reduces Redis interactions to once per failure, and direct resolution of flags reduces CPU load by 10%.

[0023] Preferably, in step S104, the backup network element monitors the distributed lock status in the shared storage in real time. If the lock is idle, it initiates an atomic preemption operation. After successful preemption, it extracts the master node's running data, analyzes the heartbeat sequence to confirm the fault, generates a fault signal, and uses a consistent hashing algorithm to parse the service configuration and determine the takeover task sequence.

[0024] The backup network element allocates computing resources to the backup nodes, optimizes the scheduling configuration through a load balancing algorithm, generates resource allocation instructions, updates the business status through task takeover instructions, and then re-verifies the lock holding status. If the status is correct, the backup network element atomically releases the distributed lock and updates the lock management log.

[0025] Preferably, in step S105, after the virtual IP successfully migrates to the backup node, the IP binding status and timestamp in the shared storage are immediately verified to confirm the binding consistency. Based on the result, the network interface status of the master node is analyzed. If it is confirmed that there is no active connection, a traffic switching instruction is generated through a consistent hash algorithm. Then, the bandwidth resources of the backup node are allocated, and the traffic scheduling is optimized using the least connection number algorithm to form a bandwidth allocation instruction. After executing the traffic takeover instruction to update the network service status, the distributed lock holding status is verified again. If it is confirmed that the backup node still holds the lock, the lock is released atomically.

[0026] Preferably, in step S106, the backup node continuously verifies the binding status and timestamp of the virtual IP in the shared storage to confirm the responsibility of the master node. Based on the responsibility status, it monitors the interface bandwidth utilization through the VRRP protocol, analyzes the load status, then calls the network configuration data, uses the consistent hashing algorithm to dynamically optimize the traffic strategy, generates traffic instructions, verifies the connection stability based on the optimization results, and if the threshold is met, sends a traffic takeover confirmation instruction and updates the network service status.

[0027] Next, the continuity of timestamps in the responsibility switching logs is analyzed. After confirming the stability of the switching, the responsibility allocation records in the shared storage are atomically updated and the takeover timestamp is marked.

[0028] A high-availability network element system based on shared storage and heartbeat detection includes:

[0029] Heartbeat monitoring module, distributed lock management module, fault detection and switching module, atomic lock preemption module, VIP drifting module, traffic takeover and optimization module;

[0030] The heartbeat monitoring module is used by the master node to send a Gzip compressed heartbeat (including CPU, memory, and latency data) every 450ms. The receiver verifies the continuity of the packet header. If the heartbeat is lost ≥3 times consecutively, or if the CPU usage is >80% or the latency is >50ms, an exception is triggered, and the load trend is output to the traffic takeover and optimization module.

[0031] The distributed lock management module is used to atomically acquire locks via Redis SETNX (TTL=200ms), renew locks every 150ms using Lua scripts, cache lock status locally (90% hit rate), exponentially back off and retry when acquisition fails, and trigger failover in case of lock exceptions.

[0032] The fault detection and switching module is used to detect heartbeat loss in a sliding window (300ms), and parse the health status bit in the heartbeat packet. The health status bit includes: a 1-byte flag bit, 0 / 1 indicating normal / abnormal. When ≥3 consecutive heartbeats are lost or the status bit is abnormal, the atomic lock preemption module is triggered and the abnormal signal of heartbeat monitoring is received.

[0033] The atomic lock preemption module is used to compete for Redis locks at a frequency of 50ms after a failure. The Lua script ensures the atomicity of SETNX+EXPIRE, and the random delay (0-10ms) reduces conflicts. After successful preemption, the status is broadcast through Pub and Sub, which drives the VIP drift module to perform IP switching.

[0034] The VIP drift module is used to elevate the priority to 200 via the VRRP protocol, multicast to 224.0.0.18, send gratuitous ARP to update the MAC mapping, ensure reliability with an 80-120ms heartbeat broadcast, execute drift after receiving the lock preemption signal, and link with the traffic takeover and optimization module;

[0035] The traffic takeover and optimization module is used to bind virtual IPs to new master nodes, allocate traffic based on consistent hashing (1024 virtual nodes), perform ICMP health checks (50ms interval), achieve a local session cache hit rate of 99.8%, receive VIP migration completion signals, and trigger heartbeat monitoring as a new master node.

[0036] The network element high availability method and system based on shared storage and heartbeat detection described in this application has the advantage that it achieves rapid fault detection (detection time can be controlled within hundreds of milliseconds) through high-frequency heartbeat and intelligent status judgment. By setting a short lock TTL and high-frequency preemption of the backup node, it ensures that the primary and backup permissions can be switched quickly and reliably after the fault is detected, avoiding the switching delay problem caused by long TTL in pure shared storage schemes.

[0037] The exclusivity of the shared lock is the core guarantee to prevent "split-brain". It ensures that at most one master node exists at any time. The design of the backup node only preempts the lock after confirming the failure of the master node. Combined with the stable renewal mechanism of the master node, it effectively eliminates the jitter of the master and backup status caused by instantaneous network jitter and unnecessary switching, and overcomes the shortcomings of pure heartbeat schemes that are prone to split-brain and false switching.

[0038] The design of the standby node not participating in lock contention or queries under normal circumstances significantly reduces the access pressure on the shared storage (Redis) and reduces the overall system overhead. This solves the load problem caused by continuous lock contention or queries in the pure shared storage scheme.

[0039] The solution is designed to be applicable to local disaster recovery scenarios for most 5G core network elements (SMF, AMF, UDM). Attached Figure Description

[0040] Figure 1 This is a flowchart of a network element high availability method based on shared storage and heartbeat detection as described in this application;

[0041] Figure 2 This is a structural diagram of a network element high availability system based on shared storage and heartbeat detection as described in this application. Detailed Implementation

[0042] like Figures 1-2 As shown in the figure, this application describes a method for high availability of network elements based on shared storage and heartbeat detection.

[0043] like Figure 1 As shown in S101, the primary network element node sends compressed heartbeat messages to the backup network element node via the UDP protocol at intervals not exceeding 500 milliseconds. The heartbeat messages contain service health status and load data.

[0044] Further, in step S101, compressed and encoded heartbeat messages are obtained from the main network element node via the UDP protocol, and the service health status and load data are decoded.

[0045] If the health status of the decoded service is lower than the preset threshold, a switching instruction is sent to the backup network element node through network communication to determine the primary / backup switching status.

[0046] According to the switching instruction, obtain the heartbeat message to be processed from the backup network element node, and decode it to obtain the new business health status and load data;

[0047] By comparing old and new load data, the load change trend is calculated to obtain the load balancing demand. If the load balancing demand exceeds the preset threshold, the resource allocation of the primary network element node and the backup network element node is adjusted through the pre-established allocation algorithm to determine the new resource configuration.

[0048] Based on the new resource configuration, update the compression encoding parameters of the heartbeat message to generate an optimized heartbeat message;

[0049] The optimized heartbeat message is sent to the backup network element node via the UDP protocol to complete the status monitoring and load balancing update.

[0050] Specifically, in step S101, the primary network element node sends compressed heartbeat messages to the backup network element node via UDP protocol at intervals not exceeding 500 milliseconds. In the specific implementation, the primary network element node first collects business health status and load data. For example, it samples CPU utilization, memory usage, and business request processing latency every 100 milliseconds through an internal monitoring module. Assuming the current sampled values ​​are CPU utilization of 70%, memory usage of 60%, and request latency of 20 milliseconds, these data are serialized into strings in JSON format, with a total size of approximately 200 bytes.

[0051] To reduce network bandwidth consumption, the Gzip algorithm is used to compress the data. The compressed size is about 80 bytes. The compression process is implemented through the standard library and takes about 2 milliseconds. Analysis shows that Gzip can achieve a compression rate of up to 60% with small data volumes, making it suitable for real-time heartbeat scenarios.

[0052] The compressed data is encapsulated into UDP packets, with a 16-byte identification field in the header, including a 4-byte main network element ID, an 8-byte timestamp, and a 4-byte sequence number, to ensure that the backup network element node can verify the message source and order;

[0053] The UDP sending interval is controlled by a high-precision timer, set to 450 milliseconds, which is lower than the upper limit of 500 milliseconds. The timer is based on the system clock and has an error of less than 1 millisecond. Analysis shows that this interval can balance real-time performance and network load.

[0054] After receiving the data packet, the backup network element node first verifies the packet header identifier and confirms the sequence number continuity. If the loss rate exceeds 5%, an alarm is triggered, indicating network instability.

[0055] Next, the heartbeat message is decompressed, the JSON data is restored, and the business health status is parsed. If the CPU utilization exceeds 80% or the latency exceeds 50 milliseconds, the primary network element is determined to be abnormal, and the backup network element enters the pre-switch state.

[0056] Load data is used for dynamic adjustment. For example, if the load of the primary network element is higher than 70%, the backup network element will allocate resources in advance. The switching time is predicted based on historical data. The prediction algorithm uses a weighted moving average method with weights of 0.5, 0.3, and 0.2 of the most recent three heartbeats. The calculation results show that the switching preparation time is about 300 milliseconds.

[0057] The entire process is implemented through automated scripts, and the logs record the sending and receiving time of each heartbeat. The average network latency is approximately 10 milliseconds, which meets the real-time requirements.

[0058] This mechanism ensures efficient communication between primary and backup nodes, has rigorous logic, and numerical analysis supports stable system operation.

[0059] In one embodiment, the definition of terms includes:

[0060] HA (High-Availability): High availability is an architectural design principle that ensures that a software system can continuously provide services under various conditions and can perform failover and service switching when a failure occurs.

[0061] TTL (Time-To-Live): Time to Live, specifically referring in this scheme to the automatic expiration time of the distributed lock key set in shared storage (Redis);

[0062] Split-brain: In a high-availability system, due to network partitioning or other reasons, two or more nodes may each believe themselves to be the master node and provide services to the outside world at the same time, resulting in data conflicts and service chaos.

[0063] VIP (Virtual IP): A virtual IP address. Clients access services by accessing this VIP. When the physical node that actually provides the service fails and switches over, the VIP will migrate to the new active node, which is transparent to the client.

[0064] VRRP (Virtual Router Redundancy Protocol): A protocol for sharing a virtual IP address among multiple routers (or hosts) to provide redundancy for the default gateway;

[0065] Redis: An open-source, high-performance in-memory key-value database, commonly used in scenarios such as caching, message queues, and distributed locks. In this solution, it is used as shared storage to implement a distributed lock.

[0066] SMF (Session Management Function): 5G core network session management function;

[0067] AMF (Access and Mobility Management Function): 5G core network access and mobility management function;

[0068] UDM (Unified Data Management): Unified data management function for 5G core network.

[0069] like Figure 1 As shown in Figure S102, the main network element node acquires the distributed lock in the shared storage module through atomic operations. The TTL of the lock is 200 milliseconds, and the lock is periodically renewed at a period less than the TTL to maintain the lock's validity.

[0070] Further, in step S102, a distributed lock is acquired from the shared storage module through atomic operations, and it is determined whether the lock has been successfully acquired to obtain the lock acquisition status;

[0071] If the lock acquisition status is successful, the renewal operation is triggered at an interval less than the lifespan through the preset renewal mechanism to determine the continued validity of the lock;

[0072] Based on the continued validity of the lock, the task scheduling data associated with the lock is extracted from the main network element node to obtain the task allocation instruction;

[0073] The task allocation instruction is used to obtain the queue of tasks to be executed from the shared storage module, determine the task priority, and determine the task execution order.

[0074] Based on the task execution order, the computing resources of the main network element nodes are adjusted using a preset allocation algorithm to obtain the resource allocation configuration;

[0075] By configuring resource allocation, updating the task scheduling parameters of the distributed lock, and generating optimized task scheduling instructions;

[0076] The optimized task scheduling instructions are distributed to the main network element nodes through the shared storage module to complete the task scheduling update.

[0077] Specifically, in step S102, the master network element node acquires a distributed lock in the shared storage module through atomic operations to ensure exclusive access to the shared resources in the distributed system. In the specific implementation, the master network element node uses a Redis-based distributed lock mechanism and attempts to acquire the lock through the SETNX command. The key name of the lock is resource_lock, and the master network element node ID is a 4-byte integer, such as ID 1001.

[0078] The SETNX operation takes approximately 0.5 milliseconds. If the lock is successfully acquired, the lock's TTL is immediately set to 200 milliseconds to prevent deadlock. This is achieved using the EXPIRE command, which takes approximately 0.3 milliseconds.

[0079] Analysis shows that a TTL of 200 milliseconds can cover the typical business operation time (average 150 milliseconds) while avoiding locks being held for a long time;

[0080] To maintain lock validity, the main network element node renews the lock every 150 milliseconds via a timed task. The renewal operation is executed via a Lua script, which checks whether the lock holder is the current node ID. If a match is found, EXPIRE is called to extend the TTL to 200 milliseconds. The script execution takes about 0.4 milliseconds. Analysis shows that the 150-millisecond renewal interval can ensure that the renewal is completed before the TTL expires, with an error of less than 1 millisecond. This is achieved using a high-precision timer based on the system clock.

[0081] If renewal fails, for example, if the Redis response delay exceeds 50 milliseconds, the system logs the event and triggers a retry. The retry interval uses an exponential backoff algorithm, with an initial interval of 10 milliseconds and a maximum interval of 100 milliseconds. Analysis shows that the success rate reaches 99.8% after three retries.

[0082] If lock acquisition fails, the main network element node enters the waiting queue and listens for lock release through the Redis BLPOP command. The maximum waiting time is 500 milliseconds. An alarm is triggered after the timeout. The average waiting time recorded in the log is 20 milliseconds.

[0083] After the lock is released, the main network element node writes the operation result to the shared storage, such as updating the resource status to "allocated". The data size is about 100 bytes and the writing takes about 1 millisecond.

[0084] To optimize performance, the system maintains a local cache that records the key name and timestamp of the 10 most recent lock operations, achieving a cache hit rate of 90% and reducing the frequency of Redis access.

[0085] The entire process is managed by automated scripts, and the logs record the start time, end time and result of each lock operation. Analysis shows that the average lock holding time is 160 milliseconds, which meets the real-time requirements of the business.

[0086] This mechanism ensures the efficiency and reliability of distributed locks through atomic operations and timed renewal.

[0087] like Figure 1 As shown in S103, the backup network element node continuously monitors the heartbeat messages. When a preset number of heartbeat messages are lost consecutively or an abnormal service health status is detected, the primary network element node is determined to be faulty and fault detection is triggered.

[0088] Furthermore, in step S103, during the heartbeat resolution workflow, the standby node receives a 37-byte data packet via a UDP port to directly identify the health status of the master node;

[0089] Connect to the window detection workflow: Based on a 300ms time window circular queue, if there are no new packets within 100ms and the number of packets in the window is less than 3, it is determined that the continuous heartbeat has been lost;

[0090] This result triggers a two-condition response in the fault determination workflow: fault handling is initiated if the heartbeat is lost ≥3 times or the flag bit is non-zero.

[0091] Subsequently, in the storage confirmation workflow, a fault flag with a 500ms TTL is written to Redis. This operation is associated with the anti-false positive workflow: a 150ms secondary verification mechanism reduces the false positive rate to below 0.1%.

[0092] Ultimately, the workflow is optimized through caching: the local heartbeat status table reduces Redis interactions to once per failure, and direct resolution of flags reduces CPU load.

[0093] Specifically, in step S103, the standby metanode continuously listens for heartbeat information sent by the master node through the UDP communication port and executes the fault detection process:

[0094] In heartbeat message parsing, a fixed-format UDP data packet is received, with a total length of 37 bytes, including a 16-byte header, a 1-byte status flag, and 20 bytes of compressed payload data. The header consists of three parts: a 4-byte master node ID (unique identifier), an 8-byte high-precision timestamp (millisecond level), and a 4-byte incrementing sequence number (used for continuity verification). The status flag defines the health status bit by bit: bit 0 being 1 indicates that the CPU utilization exceeds the 80% threshold, bit 1 being 1 indicates that the service latency exceeds the 50 millisecond threshold, and bit 2 being 1 indicates that the memory utilization exceeds the 90% threshold.

[0095] For example, receiving a status value of 5 (binary 00000101) indicates that the master node is experiencing both CPU overload and memory overrun anomalies.

[0096] In the sliding window detection mechanism, the backup node maintains a circular queue with a time window size of 300 milliseconds to record the arrival timestamps of the three most recent heartbeat packets. Each time a valid heartbeat packet is received, the current timestamp is added to the end of the queue. If no new packet is received within 100 milliseconds, the difference between the timestamps at the beginning and end of the queue is checked. If the time span exceeds 300 milliseconds and the number in the queue is less than 3, it is determined to be a continuous heartbeat loss.

[0097] The fault procedure is initiated if either of the two fault triggering conditions is met:

[0098] Condition 1: The number of valid heartbeat packets in the time window is less than 3, that is, ≥3 heartbeats are lost within 300 milliseconds;

[0099] Condition 2: The parsed status flag is non-zero, meaning any health indicator exceeds the threshold.

[0100] In fault confirmation and storage, when the fault conditions are met, the standby node immediately writes a fault flag using the Redis SETEX command: Key name format: node_fault: master node ID, for example, node_fault:1001, key value content: a combination of fault occurrence timestamp and reason code, with an automatic expiration time of 500 milliseconds, covering the entire switchover cycle. The reason code is defined as: 1: continuous heartbeat loss, 2: CPU abnormality, 3: latency abnormality, 4: composite abnormality (e.g., CPU + memory abnormality).

[0101] In the dual protection against false positives, the first is a secondary verification mechanism. After the first anomaly is detected, it waits for 150 milliseconds. If a normal heartbeat is received during this period, the fault mark is cleared. Secondly, in terms of false positive rate control, based on historical operation data statistics, the false positive rate probability is consistently below 0.1%.

[0102] In the local cache table, a heartbeat status record table with a capacity of 100 records is maintained in the memory of the standby node. Three key fields are updated in real time: latest sequence number (to prevent replay), last health flag (for quick prediction), and consecutive loss count (for auxiliary window detection). In the resource consumption optimization, the status flag is directly parsed to avoid decompressing 20-byte load data, thereby reducing CPU overhead and reducing the Redis access frequency to only 1 interaction per failure event.

[0103] like Figure 1 As shown in S104, based on the fault detection, the backup network element node initiates an atomic preemption operation to the shared storage module at a frequency higher than the lock TTL, attempting to acquire the distributed lock.

[0104] Further, in step S104, the state information of the distributed lock is obtained from the shared storage module, the current holder of the lock and the timestamp are recorded, and it is determined whether the lock is in an idle state;

[0105] If the lock is in an idle state, the backup network element node attempts to acquire the distributed lock through an atomic preemption operation and obtains the lock acquisition result;

[0106] Based on the lock acquisition result, extract the running status data of the main network element node from the shared storage module, analyze the heartbeat message sequence of the node, and determine whether the main network element node is in a fault state.

[0107] If the main network element node is in a fault state, a fault confirmation signal is generated to determine the fault detection result;

[0108] Based on the fault detection results, the pre-stored service configuration data is obtained from the shared storage module. The consistent hashing algorithm is used to analyze the service allocation strategy, determine the takeover task sequence of the backup network element node, and obtain the task allocation result.

[0109] Based on the task allocation results, obtain the computing resource status of the backup network element nodes, optimize the resource scheduling configuration using a load balancing algorithm, generate resource allocation instructions, and determine the resource scheduling results;

[0110] Based on the resource scheduling results, a task takeover instruction is sent to the shared storage module to update the service operation status of the backup network element node and obtain the service takeover result.

[0111] Based on the business takeover results, obtain the current state of the distributed lock and determine whether the lock is still held by the backup network element node;

[0112] If the lock is still held by the backup network element node, the distributed lock is released through an atomic operation to determine the lock release result;

[0113] Based on the lock release result, update the lock management log in the shared storage module, record the timestamp of the takeover operation of the backup network element node, and obtain the lock management update result.

[0114] Specifically, in step S104, after detecting a failure in the primary network element node, the backup network element node initiates an atomic preemption operation to the shared storage module at a frequency of once every 50 milliseconds to attempt to acquire a distributed lock in order to ensure high availability switching.

[0115] The shared storage module is implemented based on Redis. The distributed lock uses the setnx command, the lock key name is lock_node_1002, and the lock value contains the backup network element node ID (4-byte integer, such as ID is 1003) and the preemption timestamp (64-bit long integer).

[0116] The lock's TTL is set to 500 milliseconds, and the preemption frequency is higher than the reciprocal of the TTL to ensure that the lock will not be accidentally released due to timeout.

[0117] Before the preemption operation, the backup network element node checks the lock status through the local memory cache. The cache key is lock_status_1002, the cache hit rate is 98%, and the query time is about 0.1 milliseconds.

[0118] If the cache misses, the node sends a get command to Redis, which takes about 0.5 milliseconds.

[0119] During preemption, the node uses Lua scripts to perform atomic operations, ensuring that the setnx and expire commands are completed in a single call, with the script execution taking approximately 0.3 milliseconds;

[0120] To avoid conflicts caused by multiple nodes simultaneously competing for resources, the system introduces a random delay algorithm with a delay range of 0 to 10 milliseconds, evenly distributed to reduce the probability of contention. Analysis shows that the conflict rate is less than 0.05%.

[0121] Nodes that fail to preempt enter the exponential backoff retry mechanism, with an initial retry interval of 20 milliseconds, a maximum interval of 100 milliseconds, a maximum of 3 retries, and a total time not exceeding 320 milliseconds.

[0122] If the lock is successfully acquired, the node writes the master node status to Redis with the key master_node and the value node ID and timestamp. The write takes about 0.4 milliseconds.

[0123] To ensure consistency, the node broadcasts its status update to other backup nodes, using Redis's pub and sub mechanism. The topic is node_status, the message size is about 100 bytes, and the broadcast takes about 1 millisecond.

[0124] All operations are logged with the log key log_node_1003, which includes timestamp, operation type and result, and the write time is approximately 0.6 milliseconds;

[0125] Logically, after a successful preemption, the node immediately initiates the service takeover process to ensure business continuity. The takeover takes approximately 5 milliseconds, meeting the real-time requirements of distributed systems.

[0126] like Figure 1 As shown in S105, if the distributed lock preemption is successful, the backup network element node immediately triggers the VIP management module to migrate the virtual IP to its own node through the VRRP protocol.

[0127] Further, in step S105, if the drift operation result indicates that the virtual IP address has been successfully drifted to the backup network element node, the current binding status of the virtual IP is obtained from the shared storage module, the binding node and timestamp are recorded, it is determined whether the binding status is consistent, and the virtual IP binding result is determined.

[0128] Based on the virtual IP binding result, the network configuration data of the main network element node is extracted from the shared storage module, the network interface status is analyzed, and it is determined whether the main network element node still maintains an active network connection to obtain the network connection status.

[0129] If the network connection status shows that there are no active connections to the main network element node, the traffic allocation strategy of the virtual IP is analyzed by the consistent hashing algorithm, a traffic switching instruction is generated, and the traffic allocation result is determined.

[0130] Based on the traffic allocation results, obtain the network bandwidth status of the backup network element node, optimize the traffic scheduling configuration using the minimum connection number algorithm, generate bandwidth allocation instructions, and determine the bandwidth scheduling results;

[0131] Based on the bandwidth scheduling results, a traffic takeover command is sent to the shared storage module to update the network service status of the backup network element node and obtain the network service takeover result.

[0132] If the network service takeover result indicates that the backup network element node has taken over the traffic, then obtain the current state of the distributed lock, determine whether the lock is still held by the backup network element node, and determine the lock holding status.

[0133] If the lock holding status shows that the distributed lock is still held by the backup network element node, then the distributed lock is released through an atomic operation, the lock management log in the shared storage module is updated, the takeover operation timestamp is recorded, and the lock management update result is obtained.

[0134] Specifically, in step S105, after the backup network element node successfully acquires the distributed lock, the VIP management module is immediately triggered to migrate the virtual IP to its own node through the VRRP protocol;

[0135] The backup network element node first sends a priority adjustment instruction to the VRRP protocol stack, raising its own priority from the default value of 100 to 200. The priority data is encapsulated in a 32-bit integer and sent to the multicast address 224.0.0.18. The multicast message size is about 50 bytes and the sending time is about 0.2 milliseconds.

[0136] The VRRP protocol stack updates based on a local priority table, with the table key being vrrp_instance_1 and the value containing the node ID (e.g., 1003) and priority. The update takes approximately 0.1 milliseconds.

[0137] To ensure drift reliability, nodes broadcast a VRRP advertisement message every 100 milliseconds. The message includes the virtual IP (e.g., 192.168.1.100) and the current priority. The broadcast frequency is set based on a heartbeat algorithm, with the heartbeat interval evenly distributed between 80 and 120 milliseconds. Analysis shows that the advertisement loss rate is less than 0.01%.

[0138] If a priority conflict is detected with other nodes, the node resolves the conflict by comparing node IDs (lower IDs take precedence). The comparison algorithm takes approximately 0.05 milliseconds.

[0139] When a virtual IP migrates, the node calls the kernel ARP module to send a gratuitous ARP broadcast, announcing the mapping between the virtual IP and its own MAC address (e.g., 00:1A:2B:3C:4D:5E). The broadcast packet is about 60 bytes in size and takes about 0.3 milliseconds to send.

[0140] To ensure consistency during drift, nodes subscribe to the vrrp_status topic through Redis to receive status updates from other nodes. The message size is approximately 80 bytes, and the reception time is approximately 0.7 milliseconds.

[0141] If the drift fails, the node adopts a linear retry mechanism with a fixed retry interval of 50 milliseconds, a maximum of 2 retries, and a total time not exceeding 150 milliseconds.

[0142] After the migration is complete, the node updates its local routing table with the key route_vip_192.168.1.100 and the value the interface index (e.g., index 3 for eth0). The update takes about 0.4 milliseconds.

[0143] At the same time, the node writes drift records to the log module with the log key vrrp_log_1003, which includes the timestamp, virtual IP and operation result. The write takes about 0.5 milliseconds.

[0144] All operations are executed through automated scripts, ensuring no human intervention. Logically, they are linked with a distributed lock mechanism to form a high-availability switching closed loop.

[0145] like Figure 1 As shown in step S106, after the virtual IP migration is completed, the backup network element node takes over the service traffic and becomes the new master node, performing the master node responsibilities of steps S101 to S102.

[0146] Further, in step S106, the binding status of the virtual IP address is obtained from the shared storage module, the node identifier and timestamp in the binding status are analyzed, and it is determined whether the backup network element node continues to hold the primary node responsibility, thus obtaining the responsibility holding status.

[0147] If the responsibility holding status shows that the backup network element node continues to hold the responsibility of the primary node, then the interface status of the backup network element node is extracted through the VRRP protocol, the bandwidth utilization of the network interface is analyzed, and the interface load status is determined.

[0148] Based on the interface load status, network configuration data is obtained from the shared storage module, the traffic allocation strategy is adjusted using the consistent hashing algorithm, traffic optimization instructions are generated, and the traffic optimization result is determined.

[0149] By analyzing the connection stability of backup network element nodes based on traffic optimization results, it is determined whether the network interface meets the preset stability threshold, and the stability verification results are obtained.

[0150] If the stability verification result shows that the network interface meets the stability threshold, a traffic takeover confirmation command is sent to the shared storage module to update the network service status of the backup network element node and obtain the service status update result.

[0151] Based on the service status update results, extract the log records of the master node's responsibilities from the shared storage module, analyze the timestamp sequence in the logs, determine the continuity of responsibility switching, and determine the stability of responsibility switching;

[0152] By ensuring the stability of responsibility switching, atomic operations are used to update the responsibility allocation record in the shared storage module, recording the takeover timestamp, and obtaining the responsibility allocation update result.

[0153] Specifically, in step S106, after the backup network element node completes the virtual IP migration, it immediately takes over the service traffic, becomes the new master node, and performs the duties of the master node.

[0154] The node first initializes the service interface through the traffic scheduling module, binding the virtual IP (e.g., 192.168.1.200) to the main interface eth1, with an interface index of 4. The binding process is completed through kernel ioctl calls, taking approximately 0.6 milliseconds.

[0155] After binding, the node starts the traffic distribution algorithm, which processes client requests based on the consistent hashing algorithm. The hash ring contains 1024 virtual nodes. The client IP is mapped to the ring using the SHA-1 algorithm, and the mapping takes about 0.08 milliseconds. Analysis shows that the hash collision rate is less than 0.005%.

[0156] To ensure a smooth traffic transition, the node publishes a traffic_takeover message to Redis, notifying other nodes that it has taken over the traffic. The message size is approximately 100 bytes, and the publication time is approximately 0.9 milliseconds.

[0157] The node also initiates a health check mechanism, sending probe packets (32 bytes each) to the associated service (e.g., 192.168.1.10) every 50 milliseconds via the ICMP protocol. The round-trip time is approximately 0.4 milliseconds, and the probe loss rate is less than 0.02%.

[0158] If a service anomaly is detected, the node adjusts the detection interval using the exponential backoff algorithm, with an initial interval of 50 milliseconds and a maximum of 100 milliseconds. The adjustment takes approximately 0.05 milliseconds.

[0159] To optimize traffic processing, the node updates its local cache table with the key client_ip_192.168.1.x and the value session ID (e.g., session_7890). The cache hit rate reaches 99.8%, and the update takes approximately 0.3 milliseconds.

[0160] The node also intercepts incoming traffic through the kernel NFQUEUE module, matches packets with virtual IPs, processes queues up to 1000 packets, and takes about 0.2 milliseconds to process each packet.

[0161] To record the takeover status, the node writes a record to the distributed log system with the key traffic_log_1004, which includes a timestamp, virtual IP, and traffic statistics. The write takes about 0.7 milliseconds.

[0162] All operations are coordinated through automated scripts to ensure a logical closed loop and seamless integration with high availability mechanisms.

[0163] like Figure 2 As shown, a high-availability network element system based on shared storage and heartbeat detection includes:

[0164] Heartbeat monitoring module, distributed lock management module, fault detection and switching module, atomic lock preemption module, VIP drifting module, traffic takeover and optimization module;

[0165] The heartbeat monitoring module is used by the master node to send a Gzip compressed heartbeat (including CPU, memory, and latency data) every 450ms. The receiver verifies the continuity of the packet header. If the heartbeat is lost ≥3 times consecutively, or if the CPU usage is >80% or the latency is >50ms, an exception is triggered, and the load trend is output to the traffic takeover and optimization module.

[0166] The distributed lock management module is used to atomically acquire locks via Redis SETNX (TTL=200ms), renew locks every 150ms using Lua scripts, cache lock status locally (90% hit rate), exponentially back off and retry when acquisition fails, and trigger failover in case of lock exceptions.

[0167] The fault detection and switching module is used to detect heartbeat loss in a sliding window (300ms), and parse the health status bit in the heartbeat packet. The health status bit includes: a 1-byte flag bit, 0 / 1 indicating normal / abnormal. When ≥3 consecutive heartbeats are lost or the status bit is abnormal, the atomic lock preemption module is triggered and the abnormal signal of heartbeat monitoring is received.

[0168] The atomic lock preemption module is used to compete for Redis locks at a frequency of 50ms after a failure. The Lua script ensures the atomicity of SETNX+EXPIRE, and the random delay (0-10ms) reduces conflicts. After successful preemption, the status is broadcast through Pub and Sub, which drives the VIP drift module to perform IP switching.

[0169] The VIP drift module is used to elevate the priority to 200 via the VRRP protocol, multicast to 224.0.0.18, send gratuitous ARP to update the MAC mapping, ensure reliability with an 80-120ms heartbeat broadcast, execute drift after receiving the lock preemption signal, and link with the traffic takeover and optimization module;

[0170] The traffic takeover and optimization module is used to bind virtual IPs to new master nodes, allocate traffic based on consistent hashing (1024 virtual nodes), perform ICMP health checks (50ms interval), achieve a local session cache hit rate of 99.8%, receive VIP migration completion signals, and trigger heartbeat monitoring as a new master node.

[0171] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this application.

Claims

1. A method for high availability of network elements based on shared storage and heartbeat detection, characterized in that, include: S101. The primary network element node sends a compressed heartbeat message to the backup network element node via the UDP protocol at intervals not exceeding 500 milliseconds. The heartbeat message contains service health status and load data. S102. The main network element node acquires a distributed lock in the shared storage module through atomic operations. The lock has a TTL of 200 milliseconds and is periodically renewed at a period of 150 milliseconds to maintain the lock's validity. The atomic operations use the RedisSETNX command, and the renewal operations are executed through Lua scripts. S103. The backup network element node continuously monitors the heartbeat messages. When a preset number of heartbeat messages are lost consecutively or an abnormal service health status is detected, the primary network element node is determined to be faulty and fault detection is triggered. S104. Based on the fault detection, the backup network element node initiates an atomic preemption operation to the shared storage module at a frequency of 50 milliseconds to attempt to acquire the distributed lock, wherein the atomic preemption operation uses a Redis Lua script to execute the SETNX and EXPIRE commands. S105. If the distributed lock preemption is successful, the backup network element node immediately triggers the VIP management module to migrate the virtual IP to its own node through the VRRP protocol; S106. After the virtual IP migration is completed, the backup network element node takes over the service traffic and becomes the new master node, performing the master node responsibilities of steps S101 to S102.

2. The method for high availability of network elements based on shared storage and heartbeat detection according to claim 1, characterized in that, Step S101 includes: Heartbeat messages are compressed using Gzip, and the header contains a 16-byte identification field, which includes a 4-byte ID, an 8-byte timestamp, and a 4-byte sequence number. The backup node verifies the continuity of the sequence number. If the loss rate is >5%, an alarm is triggered. When the CPU usage is >80% or the latency is >50ms, the backup node enters the pre-switch state.

3. The method for high availability of network elements based on shared storage and heartbeat detection according to claim 1, characterized in that, Step S103 includes: Heartbeat loss is detected using a sliding window algorithm with a window size of 300ms. A fault is determined when ≥3 consecutive heartbeats are lost or the business health status exceeds a threshold, triggering a backup node to preempt the Redis lock.

4. A system for implementing the method of any one of claims 1-3, characterized in that, include: Heartbeat monitoring module, distributed lock management module, fault detection and switching module, atomic lock preemption module, VIP drifting module, traffic takeover and optimization module; The heartbeat monitoring module is configured to send and verify compressed heartbeat messages; The distributed lock management module is configured to manage Redis locks through atomic operations; The fault detection and switching module is configured to trigger switching based on a sliding window and abnormal heartbeat or status bit judgment. The atomic lock preemption module is configured to compete for the distributed lock at a frequency of 50ms. The VIP drift module is configured to execute the VRRP protocol and ARP broadcast; The traffic takeover and optimization module is configured to bind virtual IPs and allocate traffic.

Images