Method and apparatus for multi-stage orderly shutdown of SDN controller

Abstract

The application provides an SDN controller multi-stage orderly closing method and device, including: setting the state of a node to be closed to a DRAINING state, activating a folding priority queue and low priority task preemption module; in the DRAINING state, switching the folding priority queue from a first comparator based on task priority value ordering to a second comparator through an atomic replacement comparator, and only scheduling the highest priority task; when a working thread executing a low priority task executes to a safe breakpoint, migrating the low priority task to other nodes and releasing the working thread for executing the highest priority task; after all the highest priority tasks are executed, performing a closing operation on the node to be closed. The application can guarantee task migration data consistency and reduce end-to-end scheduling delay of high priority tasks in the DRAINING state.

Description

Technical Field

[0001] This application belongs to the field of SDN controllers, and particularly relates to a multi-stage ordered shutdown method and apparatus for an SDN controller. Background Technology

[0002] In carrier-grade SDN (Software-Defined Networking) controller clusters, when a controller node needs to be upgraded or repaired, a rolling upgrade approach is typically used. Services on the node to be shut down are gradually migrated to other healthy nodes before the shutdown operation. Existing solutions treat all incomplete tasks as having the same priority and queue them for migration during shutdown. If a high-priority task such as an emergency cutover or fault recovery is submitted at this time, it must wait for a large backlog of low-priority tasks in the queue to complete their migration or scheduling before it can be processed, resulting in end-to-end scheduling delays exceeding seconds. Some solutions temporarily improve the scheduling order of high-priority tasks by modifying the queue structure or traversing and rebuilding the heap. However, traversing and rebuilding involves moving queue elements, and performing this operation during shutdown can cause scheduling bottlenecks. Other solutions directly interrupt currently executing low-priority tasks to release threads, but lack safety checks on the task execution state. The interruption point may be in an intermediate state of holding a distributed lock or an uncommitted transaction, leading to lock leaks and data inconsistencies. Therefore, how to reduce the end-to-end scheduling latency of high-priority tasks in the DRAINING state while ensuring smooth switching of scheduling logic and no data consistency risks during task migration is an urgent problem to be solved in the current SDN controller rolling shutdown scenario. Summary of the Invention

[0003] This application aims to provide a multi-stage ordered shutdown method and apparatus for SDN controllers, so as to achieve low-latency scheduling of high-priority tasks while ensuring that the switching of scheduling rules is smooth during the shutdown process and that there is no risk of data inconsistency during the migration of low-priority tasks.

[0004] This application discloses a multi-stage ordered shutdown method for an SDN controller, including: Set the state of the node to be closed to DRAINING and activate the folding priority queue and low-priority task preemption module. In the DRAINING state, the folded priority queue is switched from a first comparator based on task priority numerical sorting to a second comparator by an atomic replacement comparator. Tasks with a priority lower than the current highest priority in the folded priority queue are considered to be at the same level, and only the highest priority task is scheduled. The low-priority task preemption module monitors worker threads executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, and the worker thread is released to execute the highest-priority task. The safe interruption point is the state where the worker thread has finished processing a network packet or committed a sub-transaction, does not hold a distributed lock, and has no uncommitted transactions. After all the highest priority tasks in the folded priority queue have been executed, a shutdown operation is performed on the node to be shut down.

[0005] Optionally, the folded priority queue is switched from a first comparator based on task priority numerical sorting to a second comparator via an atomic replacement comparator, including: Obtain a comparator reference to the folded priority queue, the comparator reference being held by the VarHandle atom; The comparator reference is atomically replaced by the VarHandle, replacing the first comparator with the second comparator, without adjusting the binary heap storage structure of the folded priority queue during the replacement process.

[0006] Optionally, the second comparator dynamically updates the folding threshold based on the priority of the submitted task. When it detects that the priority of the submitted task is higher than the current highest priority, it sets the priority of the submitted task to the new folding threshold.

[0007] Optionally, the method further includes: When exiting the DRAINING state, the comparator reference is replaced by the VarHandle atom, the second comparator is switched back to the first comparator, and the scheduling logic based on task priority numerical sorting is restored.

[0008] Optionally, the low-priority task preemption module monitors worker threads executing low-priority tasks, and when a worker thread reaches a safe interruption point, migrates the low-priority task to another node, including: The number of instructions executed by the worker thread is sampled using a hardware performance counter; When the number of instructions executed by the worker thread reaches a preset threshold, a preemption trigger flag is set for the worker thread; When the marked worker thread enters the safe breakpoint, the low-priority tasks being executed on the worker thread are migrated to other nodes.

[0009] Optionally, when the marked worker thread enters the safe breakpoint, the low-priority tasks being executed on the worker thread are migrated to other nodes, including: At the safety interruption point, scan the thread stack and local variables of the worker thread; After confirming from the scan results that the worker thread does not hold a distributed lock, the low-priority task is migrated to another node.

[0010] Optionally, activate the low-priority task preemption module, including: When entering the DRAINING state, the sampling event of the hardware performance counter is registered, and preemption judgment logic is injected into the callback function corresponding to the safe interrupt point; When exiting the DRAINING state, the sampling event is cancelled and the injected preemption judgment logic is removed.

[0011] Optionally, the thread stack and local variables of the worker thread are scanned, including: Traverse the distributed lock instances in the current heap using JVMTI heap scan; Determine whether the worker thread's thread stack and local variables hold references to the distributed lock instance.

[0012] Optionally, the method further includes: When an exception occurs during the JVMTI heap scan, the low-priority task preemption module is shut down, while the folded priority queue continues to run.

[0013] Another aspect of this application discloses a multi-stage ordered shutdown device for an SDN controller, comprising: The folded priority queue module is activated when the state of a node to be closed is set to DRAINING. It switches the folded priority queue from a first comparator based on task priority numerical sorting to a second comparator through an atomic replacement comparator. Tasks with a priority lower than the current highest priority in the folded priority queue are treated as the same level, and only the highest priority task is scheduled. The low-priority task preemption module is activated in the DRAINING state to monitor worker threads that are executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, and the worker thread is released to execute the highest-priority task. The safe interruption point is the state where the worker thread has finished processing a network packet or committed a sub-transaction, does not hold a distributed lock, and has no uncommitted transactions. The shutdown execution module is used to perform a shutdown operation on the node to be shut down after all the highest priority tasks in the folded priority queue have been executed.

[0014] As can be seen from the above technical solution, when the node to be closed enters the DRAINING state, this application activates the folded priority queue and the low-priority task preemption module. By switching the comparator of the priority queue to the second comparator, tasks with a priority lower than the current highest priority are treated as the same level, and only the highest priority task is scheduled. This avoids a large number of low-priority tasks in the queue blocking the scheduling of high-priority tasks, effectively reducing the queuing delay of high-priority tasks. At the same time, the low-priority task preemption module monitors the worker threads that are executing low-priority tasks, and when the thread reaches the safe interruption point (i.e., after processing a network packet or committing a sub-transaction, without holding a distributed lock and without any uncommitted transactions), the low-priority task is migrated to other nodes, releasing the worker thread to execute the highest-priority task. This achieves both safe interruption of low-priority tasks and ensures that high-priority tasks can obtain thread resources in a timely manner. After all the highest-priority tasks have been executed, the node is closed. The above mechanisms work together to significantly reduce the end-to-end scheduling delay of high-priority tasks in the DRAINING state while ensuring that there is no data consistency risk during task migration. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart of an embodiment of the multi-stage ordered shutdown method for SDN controllers provided in this application; Figure 2 A flowchart illustrating how the folded priority queue is switched from a first comparator based on task priority numerical sorting to a second comparator via an atomic substitution comparator, as provided in this application; Figure 3 The flowchart provided in this application shows how the low-priority task preemption module monitors worker threads executing low-priority tasks and migrates the low-priority tasks to other nodes when the worker threads reach a safe interruption point. Figure 4 A flowchart for scanning the thread stack and local variables of the worker thread provided in this application; Figure 5 This is a structural block diagram of an embodiment of the SDN controller multi-stage ordered shutdown device provided in this application. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not limiting, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without such specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0018] Please refer to Figure 1 , Figure 1 This is a flowchart of an embodiment of the multi-stage ordered shutdown method for SDN controllers provided in this application. The method is applied in a carrier-grade SDN controller cluster environment and is suitable for scenarios that require the ordered shutdown of nodes, such as rolling upgrades, fault maintenance, and resource scaling down of controller nodes.

[0019] An SDN controller cluster typically consists of multiple controller nodes that collectively handle network control plane tasks. When one or more nodes need to be shut down, a rolling shutdown approach is usually used to avoid service interruption. This involves shutting down nodes one by one, migrating the service tasks on each node to other healthy nodes before shutdown. In a traditional rolling shutdown process, the node to be shut down stops accepting new tasks and waits for all received tasks to complete before performing the shutdown operation. While this method ensures service continuity, the shutdown process can be lengthy if there is a backlog of low-priority tasks on the nodes. Furthermore, if urgent high-priority tasks need to be processed during this period, they must wait for all low-priority tasks in the queue to complete before being scheduled, significantly increasing the processing latency of high-priority tasks.

[0020] To address the aforementioned technical issues, this application provides a multi-stage ordered shutdown method for SDN controllers, as follows: Figure 1 As shown, it includes the following steps: S100 sets the state of the node to be closed to DRAINING and activates the folding priority queue and low-priority task preemption module.

[0021] The DRAINING state is an intermediate state for a node to be shut down, falling between the normal operation state and the shutdown state. When a node enters the DRAINING state, it stops accepting new ordinary service tasks, but can still accept urgent high-priority tasks. Simultaneously, the node activates the folded priority queue and low-priority task preemption module provided in this embodiment to ensure that high-priority tasks are processed promptly during the shutdown process. In the DRAINING state, the node gradually migrates existing low-priority tasks to other healthy nodes until all high-priority tasks are completed, at which point the final shutdown operation is performed.

[0022] For example, in an SDN controller cluster consisting of multiple controller nodes, when the cluster management node detects that a controller node needs to be upgraded, it sends a state switching command to the node to be upgraded, changing the node's state from normal operation to DRAINING. Upon receiving the state switching command, the node to be upgraded will stop receiving new low-priority tasks such as ordinary tenant configuration tasks and network traffic statistics tasks, but can still receive high-priority tasks such as network cutover tasks and fault recovery tasks. Simultaneously, the node to be upgraded will activate the folded priority queue and the low-priority task preemption module, and begin executing the orderly shutdown process.

[0023] S200, in the DRAINING state, the folded priority queue is switched from the first comparator based on the task priority value to the second comparator through the atomic replacement comparator. Tasks with a priority lower than the current highest priority in the folded priority queue are regarded as the same level, and only the highest priority task is scheduled.

[0024] In alternative implementations, such as Figure 2 As shown, switching the folded priority queue from a first comparator based on task priority numerical sorting to a second comparator via an atomic replacement comparator includes: S210, Obtain the comparator reference of the folded priority queue, the comparator reference being held by the VarHandle atom.

[0025] S220, perform atomic substitution on the comparator reference through the VarHandle, replacing the first comparator with the second comparator, without adjusting the binary heap storage structure of the folded priority queue during the substitution process.

[0026] The folded priority queue is an improvement on the traditional priority queue in this application. This queue is implemented based on a binary heap storage structure. Its main difference from the traditional priority queue is that its comparator can be atomically replaced and it supports two different comparison logics. In a traditional priority queue, the comparator is usually determined during queue initialization and cannot be modified throughout the queue's lifetime. If the queue's scheduling rules need to be changed, a new queue usually needs to be created, and all elements in the original queue moved to the new queue. This process has a time complexity of O(n), which can cause scheduling delays and affect the normal operation of the business.

[0027] Specifically, this embodiment uses VarHandle to atomically hold the comparator reference of the queue. VarHandle is a tool for manipulating object fields, providing a series of atomic operation methods that can atomically update object fields without using locks. Compared with traditional locking mechanisms, VarHandle's atomic update operation has lower overhead, completing the update operation in nanoseconds without blocking the enqueue and dequeue processes of tasks. During the initialization of the folded priority queue, a VarHandle instance is created, which points to the comparator field of the queue. When it is necessary to switch comparators, it is only necessary to replace the comparator reference from the first comparator to the second comparator using the atomic update method of VarHandle. The entire replacement process is atomic and there are no intermediate states.

[0028] For example, during the initialization of the folded priority queue, the comparator is initialized to the first comparator, which implements the logic of sorting based on task priority values. When two tasks are compared, the first comparator first compares their priority values, with the task with the higher priority value being prioritized. If the two tasks have the same priority value, their submission times are compared, with the task submitted earlier being prioritized. This comparison logic ensures scheduling fairness in general scenarios, allowing high-priority tasks to be processed first, while tasks with the same priority are executed sequentially according to their submission order. When a node enters the DRAINING state, the comparator reference is atomically replaced with the second comparator via VarHandle, which implements the folded comparison logic. During the replacement process, the binary heap storage structure of the queue remains unchanged, and all elements in the queue are not moved, thus preventing scheduling delays.

[0029] In this embodiment, the second comparator dynamically updates the folding threshold according to the priority of the submitted task. When it detects that the priority of the submitted task is higher than the current highest priority, it sets the priority of the submitted task to the new folding threshold.

[0030] The folding threshold is a crucial parameter in the second comparator, determining which tasks are folded into the same level. When comparing two tasks, the second comparator first checks if both tasks have priorities lower than the current folding threshold. If both tasks have priorities lower than the threshold, the second comparator treats them as being in the same level, and their relative order does not affect the position of the top of the heap. If one task has a priority higher than or equal to the folding threshold, the second comparator compares them according to their normal priority values.

[0031] In practice, the second comparator maintains a current highest priority variable, which records the priority value of the highest-priority task currently existing in the queue. When a new task is submitted to the queue, the second comparator compares the new task's priority with the current highest priority. If the new task's priority is higher than the current highest priority, the second comparator updates the current highest priority to the new task's priority and sets the folding threshold to the new current highest priority. If the new task's priority is lower than or equal to the current highest priority, the current highest priority and the folding threshold remain unchanged. This dynamic folding threshold update method can adapt to scenarios involving dynamic adjustments to task priorities and temporary increases in priority for emergency tasks, automatically adapting to scheduling needs without requiring any manual rule configuration.

[0032] For example, when a node first enters the DRAINING state, there are multiple tasks in the queue with priorities of 60, 70, 80, and 85. At this point, the second comparator sets the highest priority to 85 and the folding threshold to 85. When a task with a priority of 90 is submitted to the queue, the second comparator detects that this task's priority is higher than the highest priority of 85, and therefore updates the highest priority to 90 and the folding threshold to 90. At this point, all tasks with priorities lower than 90 in the queue are folded to the same level, and only the task with a priority of 90 participates in the competition for the top of the heap. After the task with a priority of 90 completes its execution, the second comparator rescans the tasks in the queue, updating the highest priority to the highest priority of the remaining tasks in the queue, 85, and the folding threshold is updated accordingly to 85.

[0033] S300: The low-priority task preemption module monitors the worker thread executing a low-priority task. When the worker thread reaches a safe interruption point, the low-priority task is migrated to another node, and the worker thread is released to execute the highest-priority task. The safe interruption point is the state where the worker thread has finished processing a network packet or committed a sub-transaction, does not hold a distributed lock, and has no uncommitted transactions.

[0034] In alternative implementations, such asFigure 3 As shown, the low-priority task preemption module monitors worker threads executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, including: S310 uses a hardware performance counter to sample the number of instructions executed by the worker thread.

[0035] One of the main improvements in this application's embodiments is the low-priority task preemption module, which addresses the problem of low-priority tasks already running on the processor occupying threads, causing high-priority tasks to wait for time slices. In traditional scheduling systems, once a thread starts executing a task, it will continue executing until completion unless the task voluntarily relinquishes the CPU or is preempted by the operating system. In the DRAINING state, if all worker threads are executing low-priority tasks, even if there are high-priority tasks waiting to be executed in the queue, these high-priority tasks must wait for the currently executing low-priority tasks to complete before they can be scheduled, which leads to a significant increase in the processing latency of high-priority tasks.

[0036] This application embodiment uses a hardware performance counter to sample the number of instructions executed by the worker thread. A hardware performance counter is a hardware component built into the processor that can count various events executed by the processor, such as instruction retirements, cache hits, and branch prediction errors. Compared to software sampling, hardware performance counter sampling has lower overhead and does not significantly affect the execution of business logic.

[0037] S320, when the number of instructions executed by the worker thread reaches a preset threshold, mark the worker thread with a preemption pending trigger flag.

[0038] In this embodiment, a hardware performance counter sampling event is registered for each worker thread executing a low-priority task. This event is triggered every certain number of instructions executed by the worker thread. When the sampling event is triggered, it is checked whether the worker thread needs to be preempted. If preemption is required, a preemption pending flag is set for the worker thread.

[0039] For example, the preset threshold can be set to 1 million instructions. When a worker thread starts executing a low-priority task, the low-priority task preemption module registers a hardware performance counter sampling event for that thread. This event is triggered every 1 million instructions executed by the thread. When the sampling event is triggered, the low-priority task preemption module checks if there are any high-priority tasks waiting to be executed in the queue. If there are high-priority tasks waiting to be executed in the queue, the worker thread is marked with a preemption trigger flag. If there are no high-priority tasks waiting to be executed in the queue, no action is taken, and the thread continues to wait for the next sampling event to be triggered.

[0040] S330, when the marked worker thread enters the safe interrupt point, the low-priority task being executed on the worker thread is migrated to another node.

[0041] In an optional implementation, when the marked worker thread enters the safety breakpoint, the low-priority tasks being executed on the worker thread are migrated to other nodes. This includes: at the safety breakpoint, scanning the worker thread's thread stack and local variables. After confirming based on the scan results that the worker thread does not hold a distributed lock, the low-priority tasks are migrated to other nodes.

[0042] A safe interrupt point refers to a specific location during task execution where interrupting the task will not cause issues such as data inconsistency or lock leakage. This application reuses the native probe callback point of the controller kernel's sharded task framework as a safe interrupt point. This point is an empty callback triggered after each sharded task processes a network packet and commits a sub-transaction. At this time, the task does not hold any distributed locks, there are no uncommitted transactions, and the context has been persisted to the sharded storage. Interrupting the task at this point and migrating it to other nodes will not affect the consistency of the business logic.

[0043] Specifically, when a worker thread marked with a preemption trigger flag reaches a safe breakpoint, the injected preemption judgment logic is triggered. This logic first scans the worker thread's thread stack and local variables to check if the thread holds any distributed lock instances. If the scan shows the thread does not hold any distributed lock instances, it's safe to interrupt the thread at this point, and low-priority tasks currently executing on that thread can be migrated to other nodes. If the scan shows the thread holds a distributed lock instance, interrupting the thread at this point might lead to lock leakage; therefore, no preemption operation is performed. Instead, the preemption trigger flag is cleared, and the thread waits for the next sampling event to trigger.

[0044] For example, a worker thread is executing a baseline verification task that processes multiple network packets. Each time the worker thread finishes processing a network packet, it enters a probe callback point. If the worker thread has been marked for preemption, preemption logic will be triggered when entering the probe callback point. This preemption logic scans the worker thread's thread stack and local variables to check for references to a distributed lock instance. If no reference to a distributed lock instance is found, the controller's native shard migration interface is called to migrate the baseline verification task to another healthy node. After migration, the worker thread is released and used to execute higher-priority tasks in the queue.

[0045] S400, after all the highest priority tasks in the folded priority queue have been executed, a closing operation is performed on the node to be closed.

[0046] Compared to related technologies, this application activates a folded priority queue and a low-priority task preemption module when a node to be shut down enters the DRAINING state. By switching the comparator of the priority queue to a second comparator, tasks with priorities lower than the current highest priority are treated as being at the same level, and only the highest priority task is scheduled. This avoids a large number of low-priority tasks blocking the scheduling of high-priority tasks, effectively reducing the queuing latency of high-priority tasks. Simultaneously, the low-priority task preemption module monitors worker threads executing low-priority tasks and migrates them to other nodes when a thread reaches a safe interruption point (i.e., after processing a network packet or committing a sub-transaction, without holding a distributed lock and without any uncommitted transactions). This releases the worker thread to execute the highest-priority task, achieving both safe interruption of low-priority tasks and ensuring that high-priority tasks can obtain thread resources in a timely manner. After all the highest-priority tasks have been executed, the node is shut down. The above mechanisms work together to significantly reduce the end-to-end scheduling latency of high-priority tasks in the DRAINING state, while ensuring that there is no data consistency risk during task migration.

[0047] In an optional implementation, the method further includes: when the DRAINING state is exited, replacing the comparator reference with the VarHandle atom to switch the second comparator back to the first comparator and restore the scheduling logic based on task priority numerical sorting.

[0048] In this embodiment, after the node to be shut down has completed the execution of all high-priority tasks and all low-priority tasks have been migrated to other healthy nodes, the cluster management node will send a state switching instruction to the node, switching its state from DRAINING to the shut-down state. During the process of the node exiting the DRAINING state, the comparator reference is atomically replaced back to the first comparator using VarHandle, restoring normal scheduling logic. Similar to the process of switching to the second comparator, the process of switching back to the first comparator is also atomic, without adjusting the binary heap storage structure or moving any queue elements, thus preventing scheduling delays.

[0049] Specifically, when a node exits the DRAINING state, it first checks if there are any unexecuted tasks in the queue. If there are, these tasks are reordered according to the logic of the first comparator. Since the first comparator sorts tasks based on their priority values, the tasks in the queue are rearranged in descending order of priority, and tasks of the same priority are arranged in ascending order of their submission time. If there are no unexecuted tasks in the queue, the node directly performs the shutdown operation.

[0050] For example, once the node to be upgraded has completed all network cutover tasks and all low-priority tasks such as baseline verification and tenant query tasks have been migrated to other healthy nodes, the cluster management node will send a shutdown command to that node. Upon receiving the shutdown command, the node to be upgraded will use VarHandle to atomically replace the comparator reference from the second comparator back to the first comparator, restoring normal scheduling logic. At this point, if there are any remaining tasks in the queue, these tasks will be reordered according to the logic of the first comparator. If there are no remaining tasks in the queue, the node will perform the final shutdown operation, completing the entire rolling upgrade process.

[0051] In an optional implementation, activating the low-priority task preemption module includes: When entering the DRAINING state, the sampling event of the hardware performance counter is registered, and preemption judgment logic is injected into the callback function corresponding to the safe interrupt point.

[0052] When exiting the DRAINING state, the sampling event is cancelled and the injected preemption judgment logic is removed.

[0053] The activation and deactivation of the low-priority task preemption module are fully linked to the state of the folded priority queue. It is only activated in the DRAINING state and does not generate any additional overhead in normal operation. This design can prevent the preemption logic from affecting the fairness of daily scheduling and also reduce the overall overhead of the system.

[0054] Specifically, when a node enters the DRAINING state, the low-priority task preemption module first registers sampling events for hardware performance counters. These sampling events are registered to all worker threads executing low-priority tasks to monitor the number of instructions executed by these threads. Simultaneously, the low-priority task preemption module uses bytecode injection technology to inject preemption judgment logic into the callback function corresponding to the safe breakpoint. Bytecode injection technology can add new logic to existing classes without modifying the kernel source code, adapting to the access requirements of operators' live networks that do not allow modification of kernel code. When a node exits the DRAINING state, the low-priority task preemption module first unregisters all registered hardware performance counter sampling events, ceasing monitoring of the number of instructions executed by worker threads. At the same time, it removes the preemption judgment logic previously injected into the safe breakpoint callback function, restoring the callback function to its original state.

[0055] For example, when a node enters the DRAINING state, the low-priority task preemption module iterates through all worker threads, identifies threads executing low-priority tasks, and registers hardware performance counter sampling events for these threads. Simultaneously, it injects preemption judgment logic at the beginning of the probe callback function using bytecode injection. This logic first checks if the current thread is marked with a preemption pending trigger flag. If marked, it performs subsequent lock scanning and task migration operations. If not marked, it returns directly and continues executing subsequent business logic. When the node exits the DRAINING state, the low-priority task preemption module iterates through all worker threads, unregisters the previously registered hardware performance counter sampling events, and removes the preemption judgment logic previously injected into the probe callback function using bytecode injection, restoring the callback function to its original state.

[0056] In alternative implementations, such as Figure 4 As shown, the thread stack and local variables of the worker thread are scanned, including: S510 uses JVMTI heap scan to traverse the distributed lock instances in the current heap.

[0057] JVMTI is a tool interface for monitoring and controlling the Java Virtual Machine. It provides a series of methods for obtaining the virtual machine's running status, operating objects, debugging programs, etc. This embodiment utilizes JVMTI's lightweight heap scanning capability to quickly traverse the current thread's stack and local variables to confirm whether it holds a distributed lock instance. Compared to traditional heap scanning methods, JVMTI's lightweight heap scanning has lower overhead, completing the scan operation in microseconds without significantly impacting normal business operations.

[0058] S520, determine whether the worker thread's thread stack and local variables hold a reference to the distributed lock instance.

[0059] Specifically, during lock scanning, the system first retrieves all distributed lock instances in the current heap using JVMTI. Then, it iterates through the current worker thread's stack and local variables, checking if any variables hold references to these distributed lock instances. If a variable is found to hold a reference to a distributed lock instance, it means the current thread holds that distributed lock, and the thread cannot be interrupted. If no variable is found to hold a reference to a distributed lock instance, it means the current thread does not hold any distributed lock, and the thread can be safely interrupted.

[0060] For example, when a worker thread enters a safe breakpoint and triggers the preemption judgment logic, the JVMTI heap scan method is called to obtain all object instances of type distributed lock in the current heap. Then, all stack frames of the worker thread are traversed, checking whether local variables in each stack frame hold references to these distributed lock instances. Simultaneously, all member variables of the worker thread are checked to see if they hold references to these distributed lock instances. If no references to any distributed lock instances are found during the entire scan, it means that the current thread does not hold any distributed locks, and the task migration operation can be performed.

[0061] In an optional implementation, the method further includes: when an exception occurs during the JVMTI heap scan, shutting down the low-priority task preemption module while retaining the folded priority queue to continue running.

[0062] To ensure system availability, this application embodiment incorporates degradation logic. Specifically, when the low-priority task preemption module malfunctions, it is automatically shut down, leaving only the folded priority queue logic running. This does not affect normal business scheduling, and the degradation logic can prevent business interruptions in extreme failure scenarios, thus improving system reliability.

[0063] Specifically, the low-priority task preemption module may encounter various anomalies, such as JVMTI interface call failure, hardware performance counter registration failure, and bytecode injection failure. When these anomalies are detected, the low-priority task preemption module will automatically perform a degradation operation. The degradation operation includes unregistering all registered hardware performance counter sampling events, removing all injected bytecode logic, and stopping all preemption-related processing flows. After degradation, the system will only retain the folded priority queue logic to run. Although preemption of low-priority tasks cannot be achieved at this time, the folded priority queue can still guarantee low-latency scheduling of high-priority tasks on the queue side, which can meet the processing needs of high-priority tasks to a certain extent.

[0064] For example, when a node enters the DRAINING state and attempts to activate the low-priority task preemption module, if the JVMTI heap scan interface call fails due to virtual machine version incompatibility, the low-priority task preemption module will detect this exception. In this case, the low-priority task preemption module will automatically perform a degradation operation, shutting down the entire module. The system will continue to use the folded priority queue for task scheduling, and high-priority tasks will still receive priority processing on the queue side, but they will not be able to preempt threads of currently executing low-priority tasks. In this situation, the processing latency of high-priority tasks may increase, but the normal operation of the business will still be guaranteed, and there will be no business interruption.

[0065] Compared with related technologies, the embodiments of this disclosure, when a node to be closed enters the DRAINING state, use VarHandle atomic replacement of the queue comparator to switch the queue from the native scheduling logic based on task priority numerical sorting to folded scheduling logic. The replacement process does not adjust the binary heap storage structure or move any queue elements, thus the scheduling rule switching is smooth. The folded comparator treats tasks with priorities lower than the current highest priority in the queue as being at the same level, with only the highest priority task participating in the heap top competition. When a high-priority task is enqueued, it only needs to be compared with a small number of nodes in the heap to reach the top, reducing the enqueue and dequeue times to near constant levels, thus solving the problem of excessively high queuing latency for high-priority tasks in queue backlog scenarios. Simultaneously, the activated low-priority task preemption module performs instruction-level sampling on worker threads executing low-priority tasks through hardware performance counters. Once the number of executed instructions reaches a preset threshold, a preemption flag is set for that thread. After processing the current network packet or committing a sub-transaction, the thread enters a safe interruption point. At this point, the thread does not hold a distributed lock and there are no uncommitted transactions. After scanning the thread stack at this point to confirm no lock, the low-priority task is migrated to another node, releasing the thread to immediately execute the high-priority task at the top of the heap. This solves the problem of high-priority tasks waiting for time slices due to all threads being occupied by low-priority tasks. Both the queue folding and task preemption mechanisms mentioned above are activated in conjunction with the DRAINING state signal. When exiting the DRAINING state, the original comparator is atomically switched back and the preemption sampling is deregistered, restoring the normal scheduling logic. Throughout the shutdown process, the queue side avoids stuttering caused by traversal reconstruction by atomically replacing the comparator, while the execution side avoids lock leakage and transaction inconsistency caused by strong task interruptions through safe interruption point verification. The two ends work together to achieve a microsecond-level end-to-end delay from enqueuing to execution of high-priority tasks, and the scheduling rules are restored without any trace after shutdown.

[0066] Based on the same inventive concept, this disclosure also provides a multi-stage ordered shutdown device for an SDN controller, such as... Figure 5 As shown, the device includes: The folded priority queue module 10 is activated when the state of the node to be closed is set to DRAINING. It switches the folded priority queue from a first comparator based on task priority numerical sorting to a second comparator through an atomic replacement comparator. Tasks with a priority lower than the current highest priority in the folded priority queue are regarded as the same level, and only the highest priority task is scheduled.

[0067] The low-priority task preemption module 20 is activated in the DRAINING state to monitor worker threads executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, and the worker thread is released to execute the highest-priority task. The safe interruption point is the state where the worker thread has finished processing a network packet or committed a sub-transaction, does not hold a distributed lock, and has no uncommitted transactions.

[0068] The shutdown execution module 30 is used to perform a shutdown operation on the node to be shut down after all the highest priority tasks in the folded priority queue have been executed.

[0069] The apparatus provided in this application embodiment can be implemented through software, hardware, or a combination of both. In a typical implementation, the apparatus can be integrated into the kernel of the SDN controller as a built-in module. The folded priority queue module 10 replaces the native priority queue of the controller kernel thread pool and is responsible for task queuing and scheduling. The low-priority task preemption module 20 runs as an independent module, responsible for monitoring the execution status of worker threads and performing task preemption operations at appropriate times. The shutdown execution module 30 is responsible for coordinating the entire ordered shutdown process and performing the final shutdown operation after all high-priority tasks have been executed.

[0070] In an optional implementation, the folded priority queue module 10 internally maintains a task queue based on a binary heap and a comparator reference held by a VarHandle atom. In normal operation, the comparator reference points to the first comparator, and the queue is sorted according to task priority values ​​and submission times. In the DRAINING state, the comparator reference is atomically replaced with the second comparator, and the queue is sorted according to the folded comparison logic. The low-priority task preemption module 20 internally maintains a hardware performance counter manager and a bytecode injection manager. The hardware performance counter manager is responsible for registering and deregistering hardware performance counter sampling events and monitoring the number of instructions executed by worker threads. The bytecode injection manager is responsible for injecting and removing preemption judgment logic in the safe interrupt point callback function. The shutdown execution module 30 internally maintains a state machine to track the state changes of nodes to be shut down and performs corresponding operations based on the state changes.

[0071] For example, when the cluster management node sends a DRAINING state switching instruction to the node to be shut down, the shutdown execution module 30 receives the instruction and switches the node's state to DRAINING. Simultaneously, the shutdown execution module 30 sends an activation instruction to the folded priority queue module 10 and the low-priority task preemption module 20. Upon receiving the activation instruction, the folded priority queue module 10 atomically replaces the comparator reference from the first comparator to the second comparator using VarHandle. Upon receiving the activation instruction, the low-priority task preemption module 20 registers sampling events for all worker threads executing low-priority tasks through the hardware performance counter manager and injects preemption judgment logic into the safe breakpoint callback function through the bytecode injection manager. When the folded priority queue module 10 detects that all highest-priority tasks have been completed, it sends a notification to the shutdown execution module 30. Upon receiving the notification, the shutdown execution module 30 sends a deregistration instruction to both the folded priority queue module 10 and the low-priority task preemption module 20. Upon receiving the deregistration command, the priority queue module 10 uses VarHandle to atomically replace the comparator reference from the second comparator back to the first comparator. Upon receiving the deregistration command, the low-priority task preemption module 20 deregisters all sampled events through the hardware performance counter manager and removes the injected preemption judgment logic through the bytecode injection manager. Finally, the shutdown execution module 30 performs the final shutdown operation, completing the entire ordered shutdown process.

[0072] For example, in a real-world application scenario, a carrier's SDN controller cluster needs to perform a rolling upgrade operation. The cluster consists of five controller nodes, each running the same version of the controller software. The cluster management node first selects the first node as the node to be upgraded and sends a DRAINING state switching command to that node. Upon receiving the command, the node to be upgraded sets its own state to DRAINING and stops receiving new ordinary service tasks. Simultaneously, the node to be upgraded activates the folded priority queue module 10 and the low-priority task preemption module 20.

[0073] After activation, the priority folding queue module 10 atomically replaces the comparator from the first comparator to the second comparator using VarHandle. At this point, the queue contains multiple tasks, including baseline verification tasks, tenant query tasks, and traffic statistics tasks, with priorities of 60, 70, 80, and 85 respectively. The second comparator automatically identifies the highest priority in the current queue as 85 and sets the folding threshold to 85. All tasks with priorities lower than 85 are folded to the same level, and their relative order does not affect the position of the top of the stack.

[0074] During the upgrade process, an emergency cutover operation was required, so a network cutover task was submitted to the SDN controller cluster. This task had a priority of 95, higher than the current folding threshold of 85. When this task was submitted to the folding priority queue of the node to be upgraded, the second comparator detected that the task's priority was higher than the current highest priority, so it updated the current highest priority to 95, and the folding threshold was updated accordingly to 95. When enqueuing this task, it only needed to be compared with the task at the top of the queue; since its priority was higher than the task at the top of the queue, it was directly placed at the top of the queue.

[0075] At this point, all worker threads on the node to be upgraded are executing low-priority baseline verification tasks. After activation, the low-priority task preemption module 20 has registered hardware performance counter sampling events for all these worker threads, with a sampling threshold set to 1 million instructions. When one of the worker threads has accumulated 1 million instructions, the hardware performance counter will trigger a sampling event. Upon receiving the sampling event, the low-priority task preemption module 20 checks for high-priority tasks waiting to be executed in the queue and therefore marks the worker thread with a preemption trigger flag.

[0076] The worker thread continues executing its current baseline verification task. After processing a network packet, it enters a safe breakpoint. At the safe breakpoint, the injected preemption logic is triggered. This logic first checks if the current thread is marked with a preemption pending flag. Upon confirming the flag, the logic uses JVMTI's lightweight heap scanning capability to traverse the current thread's stack and local variables to check if it holds a distributed lock instance. The scan results show that the thread does not hold any distributed lock instances, therefore the task migration operation can be safely performed.

[0077] The preemption judgment logic calls the controller's native shard migration interface to migrate the baseline verification task to another healthy node in the cluster. After the migration is complete, the worker thread is released. The folded priority queue module 10 immediately retrieves the network cutover task from the top of the heap and assigns it to the worker thread for execution. The entire process from submission to execution of the network cutover task is short, which can meet the time requirements of emergency cutover operations.

[0078] After the network cutover task is completed, the folding priority queue module 10 rescans the tasks in the queue, updating the current highest priority to 85, and the folding threshold is also updated accordingly to 85. At this time, the remaining tasks in the queue are all low-priority tasks with a priority of 85 or lower. The low-priority task preemption module 20 continues to monitor the execution status of worker threads and gradually migrates these low-priority tasks to other healthy nodes.

[0079] Once all low-priority tasks have been migrated to other healthy nodes and there are no unexecuted tasks in the queue, the shutdown module 30 will perform the final shutdown operation, shutting down the node to be upgraded. The cluster management node will then perform a version upgrade on this node after it is shut down. After the upgrade is complete, the cluster management node will rejoin the node to the cluster and begin upgrading the next node.

[0080] Throughout the entire rolling upgrade process, there was no service interruption. High-priority network cutover tasks were processed promptly, and low-priority baseline verification tasks were safely migrated to other nodes for continued execution. After the upgrade was completed, all nodes returned to normal operation, and the scheduling logic was restored to its original state without leaving any trace.

[0081] Those skilled in the art will understand that traditional SDN controller rolling shutdown schemes typically handle high-priority tasks in two ways: one is to stop receiving all new tasks, including high-priority tasks, during the shutdown process, which obviously cannot meet the needs of urgent services. The other approach is to allow the reception of high-priority tasks, but high-priority tasks must wait for all low-priority tasks in the queue to complete before they can be scheduled. This approach leads to a significant increase in the processing latency of high-priority tasks, failing to meet the low-latency requirements of carrier-grade services.

[0082] To address the scheduling latency issue of high-priority tasks, some existing technologies attempt to temporarily improve their scheduling order by modifying the queue structure. For example, some solutions iterate through the entire queue and move the high-priority task to the head when it is detected. This approach has a time complexity of O(n), which can cause scheduling bottlenecks and disrupt normal business operations when the queue contains a large number of tasks. Other solutions create a new high-priority queue, place high-priority tasks into this queue, and schedule tasks in this queue first. This approach requires modifying the scheduler logic, increasing system complexity, and even with multiple high-priority tasks, priority sorting is still necessary, failing to fundamentally solve the scheduling latency problem.

[0083] Other existing technologies attempt to release threads by interrupting low-priority tasks in progress, allowing high-priority tasks to be processed more promptly. For example, some solutions use the operating system's thread preemption mechanism to directly interrupt low-priority task threads. While this approach can quickly release threads, it lacks safety checks on the task's execution state. If the task holds a distributed lock or is in an uncommitted transaction state when interrupted, interrupting the task can lead to lock leaks and data inconsistencies, severely impacting the correctness of business logic. Still other solutions require inserting numerous interruption checkpoints into the business logic code, determining at each checkpoint whether task interruption is necessary. This approach requires significant modifications to the business logic code, violating the operator's access control requirement that kernel code modification is not permitted in live networks, and also increases the complexity and maintenance costs of the business logic code.

[0084] This application is not limited to this. The embodiments of this application combine the folded priority queue module 10 and the low-priority task preemption module 20 to form a complete solution. The folded priority queue module 10 achieves fast switching of scheduling rules without adjusting the binary heap storage structure or moving any queue elements by atomically replacing the comparator. The folded comparison logic treats all tasks with priorities lower than the current highest priority as the same level, allowing high-priority tasks to reach the top of the heap simply by comparing with the task at the top of the heap when enqueuing. This reduces the time complexity of enqueueing and dequeueing to near constant levels, solving the scheduling latency problem on the queue side.

[0085] The low-priority task preemption module 20 achieves secure preemption of low-priority tasks by reusing the controller's native security interrupt points, combined with hardware performance counter sampling and lightweight lock verification, without any business intrusion or data consistency risks. Hardware performance counter sampling requires no data entry points in the business logic, minimizing its impact. Lightweight lock verification ensures that lock leaks and data inconsistencies do not occur when tasks are interrupted. Task migration reuses the controller's native fragmentation migration capabilities, eliminating the need for additional task serialization and state synchronization logic, thus guaranteeing data consistency.

[0086] Without going into detail here, the two modules in this embodiment are interconnected and are only activated in the DRAINING state. In normal operation, they adopt the native scheduling logic. This design can meet the low-latency scheduling requirements in extreme scenarios without affecting the scheduling fairness in general scenarios. All technical features reuse the native capabilities of existing systems, hardware, and operating environments without introducing additional heavy components. The system overhead is extremely low and can be directly deployed in the existing network environment.

[0087] This application's embodiments change the queue's scheduling logic by atomically replacing the comparator, rather than modifying the queue structure or rebuilding the heap. The advantage of this approach is that the comparator replacement is atomic and does not involve moving queue elements, thus avoiding scheduling delays. Traditional priority queues typically tightly couple the comparison logic with the queue structure; changing the scheduling logic requires modifying the queue structure or rebuilding the heap. This application's embodiments decouple the comparison logic from the queue structure, achieving the switching of scheduling logic through atomic replacement of comparator references—a design approach uncommon in existing technologies.

[0088] Furthermore, this application employs a folded comparison logic, treating all tasks with priorities lower than the current highest priority as being at the same level. This comparison logic differs significantly from traditional priority comparison logic. Traditional priority comparison logic strictly sorts tasks according to their priority values, with each task's priority affecting its position in the queue. The folded comparison logic, however, only considers whether a task's priority is higher than the current highest priority; for tasks with lower priorities, their relative order is irrelevant. This comparison logic greatly simplifies the enqueue and dequeue operations for high-priority tasks, enabling them to reach the top of the heap and be scheduled for execution as quickly as possible.

[0089] In this embodiment, the low-priority task preemption module 20 reuses the controller's native secure interrupt points as preemption trigger points, instead of inserting additional interrupt checkpoints into the service code. The advantage of this approach is that it requires no modification to any service code, complying with the operator's current network access requirements. Furthermore, secure interrupt points are verified safe locations where task interruption will not cause data inconsistency issues. Traditional task preemption schemes either require modifying service code to insert checkpoints or directly using the operating system's thread preemption mechanism, both of which have significant drawbacks. This embodiment, by reusing existing secure interrupt points, avoids modifying service code while ensuring the security of task interruption.

[0090] This application embodiment uses a hardware performance counter to sample the number of instructions executed by worker threads to determine the timing of preemption. Compared with the traditional software-timed sampling method, this approach has higher accuracy and lower overhead. Software-timed sampling typically requires setting a timer to trigger sampling at regular intervals. This method has lower accuracy and incurs certain system overhead. Hardware performance counter sampling, on the other hand, is directly performed by the processor hardware, resulting in higher accuracy and minimal impact on the system. This application embodiment sets the sampling threshold to a certain number of instructions, rather than a certain amount of time, which better balances the preemption trigger frequency and business operation efficiency. Because different tasks execute at different speeds, the number of instructions executed within the same time period also varies. Using the number of instructions as the sampling threshold ensures that each task can execute a certain amount of work before being preempted, avoiding the additional overhead caused by frequent preemption.

[0091] This application's embodiment uses lightweight heap scanning to check whether a thread holds a distributed lock, instead of maintaining a global lock holding table. The advantage of this approach is that it eliminates the need to modify the distributed lock implementation logic or maintain additional global data structures. Lightweight heap scanning can directly obtain thread-held lock information from the virtual machine level, making it more accurate and reliable. Traditional lock holding checks typically require adding logic to the distributed lock's locking and unlocking methods to record the lock information held by each thread. This approach requires modifying the distributed lock implementation code and increases the overhead of locking and unlocking operations. This application's embodiment, through JVMTI lightweight heap scanning, can quickly and accurately check whether a thread holds a distributed lock without modifying any code, ensuring the safety of task preemption.

[0092] The two modules in this embodiment are linked by the DRAINING state signal, activating only when needed and automatically deactivating when no longer required. This design minimizes system overhead and avoids impacting daily operations. Traditional solutions typically run high-priority task scheduling logic as a resident module, incurring system overhead even when no high-priority tasks are present. In contrast, the two modules in this embodiment are only activated when a node enters the DRAINING state and automatically deactivate upon exiting, reverting to their original state. This on-demand activation design ensures that system performance remains unaffected during daily operation.

[0093] This application embodiment incorporates robust degradation logic. When the low-priority task preemption module 20 malfunctions, it automatically shuts down, leaving only the folded priority queue logic running. This design ensures system availability under extreme failure scenarios, preventing the entire system from failing due to the failure of a single module. Traditional solutions typically lack robust degradation logic; a module failure could lead to system crashes or inability to schedule tasks properly. The degradation logic in this application embodiment ensures that even when the low-priority task preemption module 20 malfunctions, the system can still maintain the priority scheduling of high-priority tasks through the folded priority queue, sustaining basic business operations.

[0094] Those skilled in the art will understand that in practical system design, there is often a trade-off between performance and complexity. To improve system performance, it is often necessary to increase system complexity. Conversely, to reduce system complexity, it is often necessary to sacrifice some performance. In the design process of this application, the embodiments fully consider the balance between performance and complexity, and by reusing the native capabilities of existing systems, achieve low-latency scheduling of high-priority tasks without significantly increasing system complexity.

[0095] For example, the folded priority queue module 10 only modifies the update logic of the queue comparator without changing the underlying storage structure of the queue. This design makes the implementation of this module very simple and ensures good compatibility with existing scheduling systems. Simultaneously, the atomic replacement of the comparator enables rapid switching of scheduling rules without causing scheduling lag, thus meeting performance requirements. The low-priority task preemption module 20 reuses the controller's native safe interrupt points and fragmentation migration capabilities, without implementing additional task interruption and migration logic. This design not only reduces system complexity but also ensures the safety and reliability of task interruption and migration.

[0096] Furthermore, all the technical features in this application's embodiments are interconnected and mutually supportive, forming an organic whole. The folded priority queue module 10 solves the scheduling delay problem on the queue side, while the low-priority task preemption module 20 solves the scheduling delay problem on the execution side. The two modules are linked through the DRAINING status signal, jointly achieving low-latency scheduling of high-priority tasks from enqueueing to execution. The absence of either module would prevent the achievement of the desired effect. For example, if only the folded priority queue module 10 is present, then when all worker threads are occupied by low-priority tasks, high-priority tasks, even if they reach the top of the heap, cannot be scheduled for timely execution. If only the low-priority task preemption module 20 is present, then when there are a large number of low-priority tasks in the queue, high-priority tasks will have to wait a long time to reach the top of the heap, also failing to achieve low-latency scheduling.

[0097] It is understood that the technical solutions of this application are not only applicable to the rolling shutdown scenario of SDN controllers, but also to other scenarios that require ensuring low-latency scheduling of high-priority tasks under extreme conditions. For example, in the fault recovery scenario of a distributed system, when a node fails, an emergency fault recovery task needs to be executed to quickly migrate the services on that node to other nodes. The technical solutions of this application can be applied to such scenarios to ensure that fault recovery tasks can be processed in a timely manner, shortening service interruption time. In large-scale data processing scenarios, when there are urgent data analysis tasks that need to be processed, the technical solutions of this application can ensure that the urgent task can be executed with priority over other ordinary data analysis tasks, improving the efficiency of data analysis.

[0098] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multi-stage ordered shutdown method for an SDN controller, characterized in that, include: Set the state of the node to be closed to DRAINING and activate the folding priority queue and low-priority task preemption module. In the DRAINING state, the folded priority queue is switched from a first comparator based on task priority numerical sorting to a second comparator by an atomic replacement comparator. Tasks with a priority lower than the current highest priority in the folded priority queue are considered to be at the same level, and only the highest priority task is scheduled. The low-priority task preemption module monitors worker threads executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, and the worker thread is released to execute the highest-priority task. The safe interruption point is the state where the worker thread has finished processing a network packet or committed a sub-transaction, does not hold a distributed lock, and has no uncommitted transactions. After all the highest priority tasks in the folded priority queue have been executed, a shutdown operation is performed on the node to be shut down.

2. The method according to claim 1, characterized in that, Switching the folded priority queue from a first comparator based on task priority numerical sorting to a second comparator via an atomic substitution comparator includes: Obtain a comparator reference to the folded priority queue, the comparator reference being held by the VarHandle atom; The comparator reference is atomically replaced by the VarHandle, replacing the first comparator with the second comparator, without adjusting the binary heap storage structure of the folded priority queue during the replacement process.

3. The method according to claim 2, characterized in that, The second comparator dynamically updates the folding threshold based on the priority of the submitted task. When it detects that the priority of the submitted task is higher than the current highest priority, it sets the priority of the submitted task to the new folding threshold.

4. The method according to claim 2, characterized in that, The method further includes: When exiting the DRAINING state, the comparator reference is replaced by the VarHandle atom, the second comparator is switched back to the first comparator, and the scheduling logic based on task priority numerical sorting is restored.

5. The method according to claim 1, characterized in that, The low-priority task preemption module monitors worker threads executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, including: The number of instructions executed by the worker thread is sampled using a hardware performance counter; When the number of instructions executed by the worker thread reaches a preset threshold, a preemption trigger flag is set for the worker thread; When the marked worker thread enters the safe breakpoint, the low-priority tasks being executed on the worker thread are migrated to other nodes.

6. The method according to claim 5, characterized in that, When the marked worker thread enters the safe breakpoint, the low-priority tasks being executed on the worker thread are migrated to other nodes, including: At the safety interruption point, scan the thread stack and local variables of the worker thread; After confirming from the scan results that the worker thread does not hold a distributed lock, the low-priority task is migrated to another node.

7. The method according to claim 5, characterized in that, Activate the low-priority task preemption module, including: When entering the DRAINING state, the sampling event of the hardware performance counter is registered, and preemption judgment logic is injected into the callback function corresponding to the safe interrupt point; When exiting the DRAINING state, the sampling event is cancelled and the injected preemption judgment logic is removed.

8. The method according to claim 6, characterized in that, Scan the thread stack and local variables of the worker thread, including: Traverse the distributed lock instances in the current heap using JVMTI heap scan; Determine whether the worker thread's thread stack and local variables hold references to the distributed lock instance.

9. The method according to claim 8, characterized in that, The method further includes: When an exception occurs during the JVMTI heap scan, the low-priority task preemption module is shut down, while the folded priority queue continues to run.

10. A multi-stage ordered shutdown device for an SDN controller, characterized in that, include: The folded priority queue module is activated when the state of a node to be closed is set to DRAINING. It switches the folded priority queue from a first comparator based on task priority numerical sorting to a second comparator through an atomic replacement comparator. Tasks with a priority lower than the current highest priority in the folded priority queue are treated as the same level, and only the highest priority task is scheduled. The low-priority task preemption module is activated in the DRAINING state to monitor worker threads that are executing low-priority tasks. When a worker thread reaches a safe interruption point, the low-priority task is migrated to another node, and the worker thread is released to execute the highest-priority task. The safe interruption point is the state where the worker thread has finished processing a network packet or committed a sub-transaction, does not hold a distributed lock, and has no uncommitted transactions. The shutdown execution module is used to perform a shutdown operation on the node to be shut down after all the highest priority tasks in the folded priority queue have been executed.

Images