A Smart Scheduling Method and System for Edge Nodes Based on the Internet of Things

Abstract

This invention relates to the field of electronic digital data processing, and discloses an intelligent scheduling method and system for edge nodes based on the Internet of Things (IoT). The method includes: distinguishing between lightly loaded and heavily loaded tasks through a task profile lookup table; the edge gateway bypassing the intelligent scheduling algorithm to perform bypass distribution for lightly loaded tasks and transferring heavily loaded tasks to the intelligent scheduling algorithm; the method also establishes a feedback loop based on execution time: the execution node monitors the execution time of bypass tasks, and if a timeout occurs, a profile degradation request is triggered; the edge gateway dynamically updates the lookup table, downgrading the task type to intelligent scheduling. This invention removes the decision-making overhead of lightly loaded tasks from the intelligent scheduling module, freeing up scheduler resources to handle heavily loaded tasks and avoiding head-end blocking.

Description

Technical Field

[0001] This invention relates to an intelligent scheduling method and system for edge nodes based on the Internet of Things, belonging to the field of electronic digital data processing technology. Background Technology

[0002] Currently, especially in large-scale distributed computing platforms, using a logically centralized intelligent scheduler to uniformly receive and allocate computing tasks to execution nodes is a common technique for achieving global load balancing and optimized allocation of processing resources. This approach relies on the scheduler's execution decision algorithm to match computing tasks with execution nodes, demonstrating its effectiveness in resource management when handling logically complex and high-value tasks. However, with the rapid growth of IoT applications, edge computing scenarios exhibit task heterogeneity, namely, the coexistence of massive amounts of logically fixed telemetry-type light-load tasks and a few logically complex management-type heavy-load tasks. The existing single-entry, unified-schedule data processing method begins to show its inherent design limitations when facing such high-concurrency mixed tasks. That is, the scheduler is required to perform a complete decision action for each arriving light-load task, causing the scheduler's own computing resources as a computing unit to be exhausted by such high-frequency, repetitive decision overhead.

[0003] In response to this situation, some direct improvement approaches, such as upgrading the scheduler hardware specifications or designing more complex task priority queues within the scheduler, do not fundamentally solve the problem. This is because lightly loaded task flows still need to enter the intelligent scheduler's processing flow, and task reception and classification themselves constitute new performance issues under high concurrency. Meanwhile, some technical solutions in this field attempt to optimize at the scheduling algorithm level. For example, Chinese invention patent CN111245950B discloses a deep learning-based intelligent scheduling system and method for industrial IoT edge resources. This solution attempts to use deep learning models and task age... The AoT (Average Time Tolerance) metric is used for task scheduling and computation offloading. However, the inherent flaw of this approach is that it still requires the scheduler to perform a complex evaluation on the vast majority of arriving tasks, including a large number of lightly loaded tasks. For example, it needs to calculate the AoT change or the optimal energy consumption solution before making a decision. This design fails to isolate the decision-making overhead of high-frequency lightly loaded tasks from that of low-frequency heavyly loaded tasks in the data processing flow. As a result, the scheduler's own computing resources are still easily exhausted in mixed task high-concurrency scenarios. This reveals an operational contradiction: in order to achieve fine-grained resource management, the system has instead led to the exhaustion of the management unit's own computing resources.

[0004] Therefore, how to optimize the data processing flow and design a system that can bypass most lightly loaded tasks from the intelligent scheduling decision process when a large number of heterogeneous tasks arrive, thereby reserving the scheduler's computing resources for a few heavily loaded tasks, has become the technical problem to be solved by this invention. Summary of the Invention

[0005] This invention provides an intelligent scheduling method and system for edge nodes based on the Internet of Things. Its main purpose is to solve the problem of how to bypass lightly loaded tasks in the data processing flow when a large number of heterogeneous computing tasks arrive concurrently, so as to avoid overloading the scheduler's own decision-making overhead and ensure the scheduling quality of heavy-load tasks.

[0006] To achieve the above objectives, this invention provides an intelligent scheduling method for edge nodes based on the Internet of Things (IoT). This method is executed in an electronic digital data processing system and includes:

[0007] Step a: Predefine a task profile lookup table. The task profile lookup table will at least distinguish between light-load profiles and heavy-load profiles, and associate a bypass scheduling strategy flag and an execution time benchmark threshold with the light-load profile, and associate an intelligent scheduling strategy flag with the heavy-load profile.

[0008] Step b: The edge gateway receives the computing task and uses the task type ID of the computing task to query the task profile lookup table to obtain its associated scheduling policy flag.

[0009] Step c: Based on the acquired scheduling policy flag, the edge gateway performs a logical judgment: if the scheduling policy flag is a bypass scheduling policy flag, the fast lane process is triggered: the edge gateway bypasses the local intelligent scheduling algorithm and uses a preset stateless distribution algorithm to deliver the computing task to the target node in the lightly loaded node pool; if the scheduling policy flag is an intelligent scheduling policy flag, the slow lane process is triggered: the edge gateway transfers the computing task to the intelligent scheduling algorithm, which selects a target node for the computing task within the heavy-load node pool.

[0010] Step d: When the target node in the lightly loaded node pool executes the computation task delivered by the fast lane process, the execution time of the computation task is monitored.

[0011] Step e: When the execution time exceeds the execution time baseline threshold associated with the lightly loaded profile, the target node sends a profile downgrade request containing the task type ID to the edge gateway.

[0012] In step f, the edge gateway responds to the profile downgrade request by updating the task profile lookup table and changing the scheduling policy flag corresponding to the task type ID from the bypass scheduling policy flag to the intelligent scheduling policy flag.

[0013] Preferably, in step a, the data structure of the light-load profile also includes an affinity key, and the target nodes in the light-load node pool are also marked with an affinity key; and the stateless distribution algorithm used in the fast lane process in step c is specifically a consistent hashing algorithm based on the affinity key for routing decisions.

[0014] Preferably, in step c, the intelligent scheduling algorithm used in the slow lane process includes: the intelligent scheduling algorithm performs load balancing calculation based on the query results of the real-time load of each node in the heavy-load node pool.

[0015] Preferably, the method further includes: the edge gateway performing health status monitoring on the task delivery of the fast lane process; when the delivery failure rate of the light-load node pool exceeds the preset circuit breaker threshold, the circuit breaker mechanism is triggered; the circuit breaker mechanism temporarily suspends the execution of the bypass scheduling strategy flag and temporarily switches all computation tasks marked with the bypass scheduling strategy flag back to the slow lane process for processing by the intelligent scheduling algorithm.

[0016] Preferably, the method further includes: edge nodes in the heavy-load node pool monitoring their own load status; when the load status is lower than a preset idle threshold, the edge node registers with the edge gateway and requests temporary inclusion in the target pool of the stateless distribution algorithm of the fast lane process; the edge gateway also delivers the computation task of the light-load profile to the registered edge nodes in the heavy-load node pool; and when the registered edge nodes in the heavy-load node pool receive the computation task of the heavy-load profile, the edge node preempts or terminates any light-load profile computation task being executed on it at the process scheduling level.

[0017] Preferably, health status monitoring specifically involves the edge gateway counting the total number of deliveries and the number of delivery failures in the fast lane process within a preset time window; and calculating the delivery failure rate. ,in , This represents the number of delivery failures. Total number of deliveries; the circuit breaker mechanism in It is triggered when the value exceeds the preset circuit breaker threshold.

[0018] Preferably, in step c, the intelligent scheduling algorithm used in the slow lane process includes: the intelligent scheduling algorithm asynchronously receiving and maintaining the predicted state vector broadcast by each node in the heavy-load node pool, the predicted state vector representing the future load of the corresponding node; the intelligent scheduling algorithm selecting a target node for the computing task based on the predicted state vector; in step f, the operation of the edge gateway changing the scheduling policy flag from the bypass scheduling policy flag to the intelligent scheduling policy flag is a temporary modification, and the edge gateway will restore the scheduling policy flag to the bypass scheduling policy flag after a preset time.

[0019] Preferably, the edge nodes in the heavy-load node pool preempt or terminate the computation task of the light-load portrait at the process scheduling level, specifically including: the operating system of the edge node sets the computation task of the heavy-load portrait to a high priority; when the computation task of the heavy-load portrait arrives, the process scheduler of the edge node sends a termination signal to the process that is executing the computation task of the light-load portrait, which is a low-priority task; after terminating the computation task of the light-load portrait, the edge node executes the computation task of the heavy-load portrait.

[0020] Preferably, in step c, the stateless distribution algorithm used in the fast lane process is a static polling algorithm; the edge gateway maintains a polling list containing the addresses of all target nodes in the lightly loaded node pool, and sequentially delivers the received lightly loaded profile calculation tasks to the next target node in the polling list.

[0021] An intelligent scheduling system for edge nodes based on the Internet of Things, the system comprising:

[0022] The task profile lookup table is used to predefine task profiles. The task profiles are at least divided into light-load profiles and heavy-load profiles. The light-load profiles are associated with bypass scheduling policy flags and execution time baseline thresholds, and the heavy-load profiles are associated with intelligent scheduling policy flags.

[0023] The receiving and querying unit is used to receive computing tasks and use the task type ID of the computing task to query the task profile lookup table to obtain its associated scheduling policy flag.

[0024] The logic judgment unit is used to perform logical judgments based on the acquired scheduling policy flags;

[0025] The stateless distribution unit is used to bypass the intelligent scheduling unit and deliver the computing task to the target node in the lightly loaded node pool by using a preset stateless distribution algorithm when the logical judgment unit determines that the scheduling policy flag is the bypass scheduling policy flag.

[0026] The intelligent scheduling unit is used to select a target node for the computing task within the scope of the overloaded node pool when the logical judgment unit determines that the scheduling policy flag is the intelligent scheduling policy flag.

[0027] The execution time monitoring unit is used to monitor the execution time of the computing tasks performed by the target nodes in the lightly loaded node pool.

[0028] The downgrade request sending unit is used to send a profile downgrade request containing the task type ID to the profile update unit when the execution time detected by the execution time monitoring unit exceeds the execution time benchmark threshold associated with the light load profile.

[0029] The profile update unit is used to respond to profile downgrade requests, update the task profile lookup table, and change the scheduling strategy flag corresponding to the task type ID from the bypass scheduling strategy flag to the intelligent scheduling strategy flag.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] 1. By setting up a task profile lookup table at the edge gateway, the system performs a local key-value lookup when it receives a computing task. Based on the obtained policy flag, it determines the subsequent data processing path. For lightly loaded tasks, the gateway bypasses the local intelligent scheduling algorithm and directly delivers them to the lightly loaded node pool using a stateless distribution algorithm. Heavyly loaded tasks, on the other hand, are handed over to the intelligent scheduling algorithm, where decisions are made within the heavyly loaded node pool. This approach removes the decision-making overhead associated with massive, high-frequency lightly loaded tasks from the intelligent scheduling module, freeing up the scheduler's digital data processing resources to be dedicated to handling a small number of truly complex heavyly loaded tasks. This avoids headend congestion caused by the scheduler's computing resources being exhausted by high-frequency lightly loaded tasks, ensuring the timeliness of scheduling and the quality of decision-making for heavyly loaded tasks.

[0032] 2. The dynamic degradation mechanism for character profiles included in this method further ensures the robustness of the fast lane. When a target node in the lightly loaded node pool executes a computation task, the execution time of its digital data processing is monitored. When the execution time exceeds a preset threshold, i.e., a disguised abnormal lightly loaded task is identified, the target node will asynchronously send a character profile degradation request to the edge gateway. The gateway responds to this request by updating its task profile lookup table and changing the scheduling strategy corresponding to the abnormal task identifier from bypass to intelligent. This closed-loop adjustment method based on execution feedback utilizes the complex scheduling and resource isolation capabilities already existing in the slow lane to accommodate abnormal tasks. Without adding complex real-time monitoring to the fast lane, it achieves automatic identification and process degradation of abnormal tasks, avoiding the risk of a single abnormal task type causing a chain reaction of overload in the entire lightly loaded node pool.

[0033] 3. The idle resource reuse mechanism included in this method improves the overall utilization rate of the system's electrical and digital data processing resources while ensuring the service quality of heavy-load tasks. Edge nodes in the heavy-load node pool actively register with the gateway and temporarily join the stateless distribution target pool of the fast lane when their electrical and digital data processing load is below the idle threshold, based on their own electrical and digital data processing load status. At the same time, the heavy-load node is configured to immediately preempt or terminate any light-load tasks being executed on it at the electrical and digital data processing level when it receives a heavy-load task. This processing method, initiating registration by the node and relying on the node's local process scheduling mechanism to ensure priority, keeps the gateway's fast lane distribution logic simple and realizes the tidal reuse of idle resources of heavy-load services by light-load services, reducing the resource deployment cost required by the system to cope with light-load peaks. Attached Figure Description

[0034] Figure 1 This is a data flow and functional architecture diagram of the intelligent scheduling system of the present invention;

[0035] Figure 2 This is a graph showing the dynamic changes in failure rate and task lane under the circuit breaker mechanism of this invention.

[0036] Figure 3 The fishbone diagram represents the four key elements for achieving efficient and robust scheduling in this invention. Detailed Implementation

[0037] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention is further described below in conjunction with specific embodiments. However, the following embodiments are only preferred embodiments of this invention and are not intended to limit the scope of protection of this invention.

[0038] This invention provides an intelligent scheduling method for edge nodes based on the Internet of Things (IoT), executed in an electronic digital data processing system. This method addresses the problem in large-scale IoT scenarios where a large number of high-frequency light-load tasks coexist with a few low-frequency heavy-load tasks, easily leading to overload of the intelligent scheduler (as a computing unit) due to decision-making overhead and headend congestion. The method proposes a data processing flow. This flow mainly relies on a predefined task profile lookup table and an edge gateway as the system's processing entry point. This gateway integrates task reception, querying, logical judgment, and distribution functions, and works collaboratively with the backend edge node cluster, logically divided into light-load and heavy-load node pools. The method also includes an asynchronous feedback mechanism from the execution node to the target node and then to the edge gateway for dynamic correction of the data processing flow. In the deployment phase, corresponding to step a, the system administrator or maintenance personnel need to predefine a task profile lookup table, which can be used within the system... The system is implemented as a hash table or key-value storage structure to support efficient query operations. The lookup table uses the task type ID of the computation task as the key. This ID is metadata used to uniquely identify a type of task. One implementation method is that the ID originates from a specific name or identifier of the task submission API. The value of the lookup table is a data structure containing at least two key fields: a scheduling policy flag and an execution time baseline threshold. Based on business characteristics, the system classifies task profiles into light-load profiles, which correspond to high-frequency, repetitive, and computationally low-cost telemetry or heartbeat tasks, and associates them with a bypass scheduling policy flag. Simultaneously, an execution time baseline threshold is associated with this light-load profile. The determination of this threshold is a system calibration procedure, the specific steps of which are: during the system's offline testing phase, for the task type corresponding to the light-load profile, execute 1000 times on the baseline edge node, statistically analyze the distribution of its execution time, and take its 99.9th percentile (P99).9) The time consumed, plus a fixed safety margin as the threshold, is used to cover normal execution fluctuations while also identifying execution anomalies; correspondingly, for management or analysis tasks with high computational overhead and complex logic, they are defined as overload profiles and associated with intelligent scheduling policy flags; during the runtime phase, corresponding to step b, the edge gateway, as the system's data processing entry point, upon receiving any computing task, immediately extracts its task type ID from the task's metadata and uses this ID as the key to perform a local key-value query operation on the task profile lookup table defined in step a to obtain its associated scheduling policy flag; corresponding to step c, the edge gateway's processing logic, based on the scheduling policy flag obtained in the previous step, executes a logical... The decision-making process can be implemented as a switch-case or if-then-else flow control structure. If the obtained flag is a bypass scheduling policy flag, the system triggers the fast lane process. In this case, the edge gateway's processor bypasses the local intelligent scheduling algorithm module and instead uses a stateless distribution algorithm with extremely low computational overhead to select a target node from a pool of lightly loaded logical groups and immediately deliver the computation task. Conversely, if the obtained flag is a smart scheduling policy flag, the system triggers the slow lane process. In this case, the edge gateway transfers the computation task to its internal intelligent scheduling algorithm module, which performs its decision-making operations within a dedicated pool of heavily loaded logical groups to select a target node for the task.

[0039] In the aforementioned fast-track process, a stateless distribution algorithm is used for rapid distribution. In one implementation, this algorithm is a static polling algorithm. The edge gateway maintains a polling list in memory containing the addresses of all target nodes in the light-load node pool. Each time a light-load profile task is received, it sequentially delivers the task to the next target node in the list, returning the head of the list when the end is reached. In another implementation, this distribution algorithm can be used for data locality optimization. In this case, the data structure of the light-load profile defined in step a also includes an affinity key, which can be an identifier representing the data source. Simultaneously, the target nodes in the light-load node pool are also pre-marked with affinity keys. In this case, the stateless distribution algorithm used in the fast-track process is specifically a consistent hashing algorithm based on affinity keys for routing decisions. The edge gateway performs a hash calculation on the affinity key of the task, and the result of this calculation deterministically points to the target node in the light-load node pool with the same affinity key, thus achieving efficient routing. Lightly loaded tasks from the same data source are delivered to fixed nodes. In the aforementioned slow lane process, an intelligent scheduling algorithm is used to achieve refined resource matching for heavily loaded tasks. In one implementation, the algorithm is a load balancing algorithm based on real-time load query. After receiving a heavily loaded task, the edge gateway actively broadcasts a query request to the nodes in the heavily loaded node pool, collects the real-time resource utilization of each node, such as CPU and memory usage, performs calculations, and selects the node with the lowest current load as the target node. In another implementation, the algorithm can be a decision-making algorithm based on asynchronous predictive state vector (PSV) broadcasting. In this case, the nodes in the heavily loaded node pool actively and periodically broadcast their predictive state vectors to the edge gateway. These vectors represent the load trend of the corresponding node in the future. The edge gateway asynchronously receives and maintains these vectors. When it is necessary to schedule heavily loaded tasks, it directly makes decisions based on these predictive state vectors already in memory to avoid the latency caused by real-time queries.

[0040] This invention further provides a feedback closed-loop mechanism for handling an abnormal situation where a task marked as lightly loaded exhibits heavy loading behavior during actual execution on a node due to defects or malicious attacks. As in step d, when a target node deployed in the lightly loaded node pool executes a computational task received from the fast track, its local operating system or runtime environment initiates an execution time watchdog mechanism to monitor the actual execution time of the task. Then, as in step e, the node compares the monitored actual execution time with the execution time benchmark threshold associated with the task profile, defined in step a. When the node determines that the actual execution time exceeds the benchmark threshold, it determines that the task is an abnormally lightly loaded task and, while terminating the abnormal task, asynchronously sends a profile degradation request data packet to the edge gateway. This data packet contains the task type ID of the task that caused the timeout. Finally... As in step f, after receiving this profile downgrade request, the edge gateway's processor responds by performing a data update operation on the task profile lookup table, i.e., retrieving the entry corresponding to the task type ID and changing its scheduling policy flag from the bypass scheduling policy flag to the intelligent scheduling policy flag. In a preferred embodiment, the modification operation in step f is temporary. The edge gateway starts a timer, and after a preset duration (optional value of 1 hour), it automatically restores the flag to bypass, allowing the system to automatically resume fast lane processing after anomaly repair. To address potential overall failures of the lightly loaded node pool, such as network failures or attacks, the invention may also include a circuit breaker switchback mechanism. The edge gateway performs health status monitoring on the task delivery of its fast lane process. Specifically, the edge gateway sets a preset time window for the fast lane in memory (optional value of 5 seconds), and counts the total number of fast lane process deliveries within the window. and the number of delivery failures Delivery failures can manifest as connection timeouts or connection rejections; the gateway's processor periodically calculates the delivery failure rate. Its calculation formula is ,in Total number of deliveries This represents the number of delivery failures; when the gateway determines that... When the value exceeds a preset circuit breaker threshold (optional value is 0.5), the circuit breaker mechanism is triggered. This circuit breaker mechanism temporarily suspends the judgment logic of the bypass scheduling strategy flag in step c at the process level, and temporarily switches all computing tasks marked as bypass back to the slow lane process, which is then handled by the intelligent scheduling algorithm. This mechanism is used to automatically degrade the processing process to ensure the continuity of data processing when the fast lane fails.

[0041] The determination process for the preset time window and preset circuit breaker threshold in the circuit breaker mechanism may include: for the time window, its length is related to the service's tolerance time for fast lane failures. A shorter window, such as 1-3 seconds, can be used for services sensitive to transient failures, while a longer window, such as 5-10 seconds, is used to smooth network jitter and avoid false circuit breakers; for the circuit breaker threshold, its determination requires statistical analysis of the baseline delivery failure rate of the fast lane over a long period, such as 1 hour, under stable system conditions. ,Should Reflecting the network's inherent noise level, the circuit breaker threshold is set to a value higher than [a certain threshold]. And the value that can identify systemic faults is set as follows: The percentage can be set to 10 times or a fixed absolute percentage, such as 5%, to ensure that circuit breaking is only triggered when there is a large-scale connection timeout or denial of service in the lightly loaded node pool. Furthermore, for the heavy-load node pool during off-peak periods, such as early morning hours when resources are largely idle, this invention may also include a preemptive reuse mechanism for idle resources. Edge nodes deployed in the heavy-load node pool run a background process locally to monitor their own load status, such as CPU utilization. When the load status remains below a preset idle threshold (for example, CPU utilization below 10% for 60 consecutive seconds), the slow-lane node determines that its resources are idle and actively registers with the edge gateway, i.e., sends a registration message requesting temporary inclusion in the fast-lane stateless distribution algorithm. In the target pool; the edge gateway responds to this registration by dynamically adding the node's address to the target pool of the fast lane, which can be added to the polling list, so that the light-load portrait computing tasks are also delivered to these idle heavy-load nodes; to ensure the quality of service (QoS) of heavy-load tasks, the operating system or container of these heavy-load nodes is configured to give absolute priority to heavy-load tasks. When a heavy-load node receives its heavy-load portrait computing task, the node's process scheduler is configured to immediately preempt or terminate any light-load portrait computing tasks that are being executed on it at the process scheduling level. One implementation is that the process scheduler sends a termination signal (such as SIGKILL) to the process of the light-load portrait computing task and immediately executes the high-priority heavy-load portrait computing task.

[0042] Regarding the preset idle threshold and continuous monitoring duration of load status in the aforementioned idle resource reuse mechanism, the determination process may include: For the idle threshold, its set value should be higher than the silent load of the heavy-load node when no business tasks are executed. This silent load refers to the baseline CPU utilization rate consumed by the operating system and necessary background daemons. One calibration method is to run the node in an idle state for a long period of time, such as 1 hour, and take the 99th percentile (P99) reading of its CPU utilization rate, and then add a fixed safety margin, such as 2% to 5%, to avoid system jitter. For the continuous monitoring duration, its setting is used to prevent the heavy-load node from frequently registering and deregistering during the short interval between two heavy-load tasks. The value of this duration should refer to the typical arrival interval time of the heavy-load profile task, and can be set to the 75th percentile (P75) value of the task arrival interval time distribution, to ensure that the node only joins the fast lane when it truly enters a long period of idle time. The present invention also provides an intelligent scheduling system for edge nodes based on the Internet of Things. This system is used to implement any of the aforementioned methods. At the hardware level, the system includes a memory and a processor. At the logical level, the system includes: a task profile. The system comprises the following components: a lookup table, stored in memory, for pre-defining task profiles, distinguishing between light and heavy loads, associating bypass or intelligent scheduling policy flags, and setting execution time baseline thresholds; a receiving and querying unit, executed by the processor, for receiving computational tasks and querying the task profile lookup table using the task type ID to obtain scheduling policy flags; a logical judgment unit, executed by the processor, for performing logical judgments based on the obtained flags; a stateless distribution unit, executed by the processor, for bypassing the intelligent scheduling unit and delivering the task to the light load node pool using a stateless distribution algorithm when the task is determined to be a bypass; an intelligent scheduling unit, executed by the processor, for selecting a target node for the task in the heavy load node pool when the task is determined to be intelligent; an execution time monitoring unit, deployed on the target node in the light load node pool, for monitoring the execution time of the task; a degradation request sending unit, deployed on the target node in the light load node pool, for sending a profile degradation request to the profile update unit when the execution time times out; and a profile update unit, executed by the processor, for responding to the degradation request, updating the task profile lookup table, and changing the bypass flag to the intelligent flag.

[0043] Example 1: In a specific data processing scenario, a large-scale urban IoT monitoring system deployed the scheduling method and system of this invention. Its edge gateway, as the data processing entry point, is responsible for receiving and scheduling computing tasks from millions of front-end sensors. Under normal operating conditions, these computing tasks exhibit high heterogeneity, with the vast majority being telemetry-based light-load profiling tasks reported once per second, coexisting with a few data calibration-based heavy-load profiling tasks executed once per hour. At a certain moment, due to a sudden weather event, the arrival rate of light-load profiling tasks across the entire network increased by 50 times, forming a task flood. At the same time, a system management task, namely a log analysis computing task marked as a heavy-load profiling task used to respond to this weather event, was also submitted to the edge gateway. Under the existing processing method of using a single entry point and unified scheduling, the intelligent scheduling algorithm module of the edge gateway quickly reached resource exhaustion because its computing resources were used to perform decision calculations for each arriving light-load profiling task. As a result, the high-value heavy-load profiling task accumulated in the scheduling queue for a long time and could not be distributed to the heavy-load node pool by the processor in a timely manner, leading to head-end blockage.

[0044] The scheduling method provided by this invention operates differently under the same working conditions. When the edge gateway receives the first lightly loaded profiling task amidst a massive influx, its processor, according to step b, uses the task type ID to query a predefined task profiling lookup table and obtains its associated bypass scheduling policy flag. Based on this flag, the edge gateway's logical judgment unit, step c, triggers the fast-track process. Its processor bypasses the local intelligent scheduling algorithm and uses the aforementioned stateless distribution algorithm with static polling to deliver the task to the lightly loaded node pool. This processing path is effective for all subsequent lightly loaded profiling tasks, and the computing resources of the edge gateway's intelligent scheduling algorithm module are not consumed by the lightly loaded tasks throughout the process. When the log analysis task for the heavily loaded profiling arrives at the edge gateway, the gateway executes a task profiling lookup table query to obtain its associated intelligent scheduling policy flag, and the logical judgment unit (step c)... Triggering the slow lane process, the edge gateway transfers the computation task to the intelligent scheduling algorithm. Since the intelligent scheduling algorithm's computational resources are not currently occupied by lightly loaded profiling tasks, this algorithm module, such as an algorithm based on real-time load querying, can immediately perform load balancing computation within the heavy-load node pool. The processor selects a target node for the heavy-load profiling task and completes delivery in a very short time. In this instance, the bypass design of step c in the fast lane process decouples the decision-making overhead of massive lightly loaded tasks from the intelligent scheduling module. This refactoring of the data processing flow ensures that the scheduler's computational resources are reserved and dedicated to processing heavy-load profiling tasks, thus guaranteeing the Quality of Service (QoS) of step c in the slow lane process and resolving the headend congestion problem. Simultaneously, the lightly loaded profiling task achieves low-latency processing due to the low-overhead, stateless distribution of the fast lane, maintaining the overall data processing throughput and stability of the system.

[0045] Example 2: This example was conducted on a simulation test platform to quantify the specific effects of the method of the present invention on the data processing flow, particularly its ability to avoid blocking at the scheduler headend. The platform includes an edge gateway, simulated as a computing unit with an 8-core CPU and 16GB of memory, a load generator, and two logical node pools: a light-load node pool and a heavy-load node pool, each containing 20 nodes. Two test groups were set up: a control group, which adopted a single entry point and unified scheduling approach, where all computing tasks were processed by the edge gateway's intelligent scheduling algorithm based on real-time load query; and the present invention test group, which adopted the method of the present invention, where the edge gateway pre-configured a task profile lookup table, and the light-load profile task type (ID: Task_Telemetry) triggered the fast-track process and... A static polling algorithm was used. Heavy-load portrait task type ID Task_Analytics triggered the slow lane process and was transferred to the same intelligent scheduling algorithm as the control group. The experimental conditions were set as follows: the arrival rate of heavy-load portrait (Task_Analytics) tasks was kept constant at 10 TPS, while the arrival rate of light-load portrait (Task_Telemetry) tasks was gradually increased from 1000 TPS, 5000 TPS, and then to 10000 TPS. Under each load level, the CPU utilization of the process containing the intelligent scheduling algorithm module in the edge gateway, the average scheduling latency of heavy-load portrait tasks (which refers to the time from the task arriving at the gateway to the task being delivered to the target node), and the average end-to-end latency of light-load portrait tasks were recorded. The experimental data are summarized in Table 1.

[0046] Table 1: Data from the Scheduling Performance Comparison Test

[0047] Referring to Table 1, the data shows that in the control group, as the arrival rate of lightly loaded portrait tasks increased, the CPU utilization of the intelligent scheduling algorithm climbed from 15.3% to 85.5%, the average scheduling latency of heavily loaded portrait tasks worsened from 20.5ms to 1205.1ms, indicating head-end blocking, and the latency of the lightly loaded portrait tasks themselves also swelled to 1350.8ms due to queuing by the scheduler. In contrast, in the sample group of this invention, due to the bypass mechanism of the fast lane process (corresponding to step c), the high-concurrency lightly loaded portrait tasks did not consume the computing resources of the intelligent scheduling algorithm, and the CPU utilization of this algorithm was significantly reduced. U utilization remained at a low level of 5.1% to 5.3% throughout the test, a level that corresponds only to a constant 10 TPS heavy-load portrait task processing load. The average scheduling latency of the heavy-load portrait task remained stable in the range of 20.2ms to 21.1ms, thus avoiding head-end blocking. The light-load portrait task maintained a low latency level of 8.2ms to 8.8ms due to stateless distribution. This experimental data confirms that the method of the present invention, by separating the task portrait lookup table and the data processing flow of fast and slow lanes, releases the computing resources of the intelligent scheduling unit under high-concurrency mixed task loads.

[0048] Example 3: This example uses the same simulation test platform as Example 2, including an edge gateway, a load generator, and 20 nodes each in the light-load node pool and the heavy-load node pool, to verify the impact of steps d, e, and f of the dynamic profile degradation mechanism on the system robustness. A control group is set up, in which the edge gateway only implements steps a, b, and c of the task profile lookup table and the fast / slow lane diversion mechanism, but the nodes in its light-load node pool do not implement execution time monitoring and profile degradation feedback loop. A prototype of this invention is set up, in which the system fully implements all processing methods, including the dynamic profile degradation mechanism. In this experiment, a computational task with task type ID Task_Poison is incorrectly associated in advance in the task profile lookup table. The bypass scheduling strategy flag was set, meaning the system mistakenly identified the task as a light-load profile. However, the actual computation time (simulating processor getting stuck in a loop) of this task on the node was set to a constant 3000ms, far exceeding the execution time benchmark threshold of 50ms associated with the light-load profile. The test conditions were set as follows: the load generator sent light-load profile (Task_Telemetry) tasks at a constant rate of 5000TPS. After the system had been running stably for 30 seconds, the load generator injected an additional Task_Poison task stream with an arrival rate of 20TPS, which lasted for 60 seconds. The test platform monitored the average CPU load of the light-load node pool and the average end-to-end latency of the light-load profile (Task_Telemetry) tasks. The test data are summarized in Table 2.

[0049] Table 2: Comparative Experimental Data on the Robustness of Dynamic Image Degradation Mechanism

[0050] Referring to Table 2, the experimental data shows that during the 0-30 second phase of the experiment, the two test groups were in the same state. After the Task_Poison task was injected at the 30-second mark, the control group, due to the lack of a feedback mechanism in the edge gateway, had its processor continuously deliver the abnormal task as a light-load profile to the light-load node pool. Under the impact of the abnormal task flow of 20 TPS, the computing resources of the 20 nodes in the light-load node pool were occupied and exhausted one by one, and the average CPU load quickly climbed to 98.6%, causing the entire fast-track process to crash, and the latency of the normal Task_Telemetry task deteriorated to 1850.3ms. In contrast, the sample group of this invention... In the lightly loaded node pool, after the target node executes the first Task_Poison task and detects an execution timeout (corresponding to steps d and e), the processor immediately sends a profile downgrade request to the edge gateway. The edge gateway responds to this request and updates the task profile lookup table (corresponding to step f). The processor modifies the task type ID of Task_Poison to the intelligent scheduling policy flag. Therefore, all subsequent Task_Poison tasks are automatically transferred to the slow lane process by the edge gateway. The lightly loaded node pool is not affected, and its average CPU load and Task_Telemetry task latency remain stable.

[0051] Example 4: This example combines Figures 1 to 3 This document describes an intelligent scheduling method and system for edge nodes based on the Internet of Things (IoT). Figure 1 As shown, the intelligent scheduling algorithm engine serves as the core, receiving inputs from the data preprocessing module, the edge node resource awareness module, and the task priority evaluation module. The data preprocessing module processes edge node status data and task request data, the edge node resource awareness module processes network topology information, and the task priority evaluation module processes user preference settings. The output of the intelligent scheduling algorithm engine leads to scheduling instruction distribution, task execution monitoring, performance data feedback, and log recording and analysis. Among these, performance data feedback and log recording and analysis also dynamically adjust the intelligent scheduling algorithm engine through feedback optimization links.

[0052] like Figure 2 As shown in the figure, the graph uses the time window as the horizontal axis and the delivery failure rate and task volume as the vertical axis, clearly revealing the dynamic relationship between the delivery failure rate, the task volume in the fast lane, and the task volume in the slow lane. Specifically, within the 10-15s time window, when the delivery failure rate reaches its peak, the task volume in the fast lane correspondingly drops to a low point, while the task volume in the slow lane instantly jumps from zero to its peak. This intuitively demonstrates the system's process of automatically switching tasks back to the slow lane when an anomaly is detected in the fast lane. Figure 3As shown, the achievement of this goal mainly relies on the synergistic effect of four aspects: scheduling strategies and algorithms, including intelligent scheduling strategies, bypass scheduling strategies and fast and slow lane diversion; data profiling and input, including computing task requests, distinguishing between light and heavy loads and task profile lookup tables; system architecture and components, including edge gateways, intelligent scheduling units and logical judgment units; and feedback and guarantee mechanisms, including execution time benchmark thresholds, timeout triggering degradation and ensuring system robustness.

[0053] Example 5: This example discloses a calibration procedure for key thresholds in the task profile lookup table during the system deployment phase, as well as a specific processing logic for the slow lane intelligent scheduling algorithm. The calibration procedure executes the execution time benchmark threshold for the light-load profile, taking a telemetry task with task type ID Task_Telemetry as an example. The initial calibration state is defined as follows: a benchmark node is selected from a light-load node pool. This node has processor and memory specifications sufficient to run the task, and it is ensured that the node is idle during the calibration period and does not execute other computational tasks. The calibration operation flow is as follows: First, the Task_Telemetry task is continuously sent and executed 10,000 times to the benchmark node using a testing tool; Second, the benchmark node's monitoring... The measurement unit records the processor execution time for each execution in step d, forming an execution time sample set containing 10,000 data points. The third step involves statistical analysis of this sample set to obtain its percentile data. A typical analysis result is as follows: the 90th percentile (P90) has a time of 20.1ms, the 99th percentile (P99) has a time of 35.5ms, the 99.9th percentile (P99.9) has a time of 48.2ms, and the maximum time of 210.4ms is caused by occasional system jitter. The fourth step determines the threshold. To balance covering most normal jitter with timely detection of true execution anomalies, the threshold is set using the principle of P99.9 time plus a fixed safety margin, where the fixed safety margin is set to 26.8ms. This forms the final execution time baseline threshold. Calculated and set as: The fifth step is to write the Task_Telemetry task profile entry into the task profile lookup table and associate the bypass flag with the 75ms execution time baseline threshold.

[0054] Secondly, a specific implementation path of the slow lane intelligent scheduling algorithm is disclosed, which corresponds to load balancing calculation based on real-time load query. The operation flow of the algorithm, i.e., the load balancing calculation, is defined as follows: Step 1: Input reception. The intelligent scheduling unit receives a heavy load profile calculation task from the logic judgment unit; Step 2: Status query. The intelligent scheduling unit immediately sends the task to all nodes in the heavy load node pool. Each node broadcasts a real-time load query request; the third step is data collection, where the intelligent scheduling unit asynchronously collects data from all nodes within a very short time window (e.g., 5ms). The returned load vector The vector contains at least: ,in For nodes Current CPU usage percentage For nodes The number of heavy-load portrait tasks currently being executed or queued; the fourth step is load scoring calculation, where the intelligent scheduling unit's processor processes each returned task. Apply a preset weighted load scoring formula. ,in For nodes The overall load score (the lower the score, the more idle). and The preset weighting coefficients, and In this embodiment, it is set to and The fifth step (decision output) involves the intelligent scheduling unit's processor performing calculations, specifically selecting the option with the smallest... The node with the value is used as the final target node; taking a specific numerical calculation as an example, if node 1 returns... Node 2 returns ,but , ,because The intelligent scheduling unit will select node 2 as the delivery target for this heavy-load portrait task.

[0055] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An Internet of Things based edge node intelligent scheduling method, the method being executed in an electronic digital data processing system, characterized in that, The method comprises: Step a, predefining a task image lookup table, the task image lookup table at least distinguishes task images into light-load images and heavy-load images, and associates a bypass scheduling strategy flag and an execution time benchmark threshold with the light-load images, and associates an intelligent scheduling strategy flag with the heavy-load images; Step b, the edge gateway receives a computing task, and uses a task type ID of the computing task to query the task image lookup table to obtain the associated scheduling strategy flag; Step c, the edge gateway performs logical judgment based on the obtained scheduling strategy flag: if the scheduling strategy flag is the bypass scheduling strategy flag, a fast lane process is triggered: the edge gateway bypasses the local intelligent scheduling algorithm, adopts a preset stateless distribution algorithm, and delivers the computing task to a target node in a light-load node pool; if the scheduling strategy flag is the intelligent scheduling strategy flag, a slow lane process is triggered: the edge gateway hands over the computing task to the intelligent scheduling algorithm, and the intelligent scheduling algorithm selects a target node for the computing task within the scope of a heavy-load node pool; Step d, the target node in the light-load node pool monitors the execution time of the computing task when executing the computing task delivered in the fast lane process; Step e, when the execution time exceeds the execution time benchmark threshold associated with the light-load image, the target node sends an image degradation request containing the task type ID to the edge gateway; Step f, the edge gateway updates the task image lookup table in response to the image degradation request, and modifies the scheduling strategy flag corresponding to the task type ID from the bypass scheduling strategy flag to the intelligent scheduling strategy flag; In step a, the data structure of the light-load image further comprises an affinity key, and the target node in the light-load node pool is also marked with the affinity key; and in step c, the stateless distribution algorithm adopted in the fast lane process is a consistent hashing algorithm based on the affinity key for routing decision; In step c, the intelligent scheduling algorithm adopted in the slow lane process comprises: the intelligent scheduling algorithm performs load balancing calculation based on the query result of the real-time load of each node in the heavy-load node pool. 2.The method of claim 1, wherein, The method further comprises: the edge gateway performs health state monitoring on the task delivery of the fast lane process; when the delivery failure rate of the light-load node pool exceeds a preset fuse threshold, a fuse mechanism is triggered; the fuse mechanism temporarily suspends the execution of the bypass scheduling strategy flag, and temporarily switches all computing tasks marked with the bypass scheduling strategy flag to the slow lane process for processing by the intelligent scheduling algorithm. 3.The method of claim 1, wherein, The method further comprises: the edge node in the heavy-load node pool monitors its own load state; when the load state is lower than a preset idle threshold, the edge node registers with the edge gateway to request temporarily joining a target pool of the stateless distribution algorithm of the fast lane process; the edge gateway also delivers the computing task of the light-load image to the registered edge node in the heavy-load node pool; and when the registered edge node in the heavy-load node pool receives the computing task of the heavy-load image, the edge node preempts or terminates any computing task of the light-load image being executed on the edge node at the process scheduling level.

4. The edge node intelligent scheduling method based on the Internet of Things according to claim 2, characterized in that, Health status monitoring, specifically, the edge gateway counts the total delivery times and the delivery failure times of the fast lane process within a preset time window; calculates the delivery failure rate wherein , is the delivery failure times, is the total delivery times; the fuse mechanism is triggered when is greater than a preset fuse threshold.

5. The edge node intelligent scheduling method based on the Internet of Things according to claim 1, characterized in that, In step c, the intelligent scheduling algorithm adopted by the slow lane process includes: the intelligent scheduling algorithm asynchronously receives and maintains the prediction state vectors broadcast by each node in the heavy load node pool, and the prediction state vectors represent the future load of the corresponding node; the intelligent scheduling algorithm selects a target node for the computing task based on the prediction state vector; in step f, the edge gateway modifies the scheduling strategy flag from the bypass scheduling strategy flag to the intelligent scheduling strategy flag, which is a temporary modification, and the edge gateway restores the scheduling strategy flag to the bypass scheduling strategy flag after a preset time period.

6. The edge node intelligent scheduling method based on the Internet of Things according to claim 3, characterized in that, The edge nodes in the heavy load node pool preempt or terminate the computing tasks of the light load image at the process scheduling level, specifically including: the operating system of the edge node sets the computing task of the heavy load image to high priority; when the computing task of the heavy load image arrives, the process scheduler of the edge node sends a termination signal to the process that is executing the computing task of the light load image as a low priority; the edge node executes the computing task of the heavy load image after terminating the computing task of the light load image.

7. The edge node intelligent scheduling method based on the Internet of Things according to claim 1, characterized in that, In step c, the stateless distribution algorithm adopted by the fast lane process is a static polling algorithm; The edge gateway maintains a polling list containing the addresses of all target nodes in the light load node pool, and sequentially delivers the received computing task of the light load image to the next target node in the polling list.

8. An edge node intelligent scheduling system based on Internet of Things, used for realizing the edge node intelligent scheduling method based on Internet of Things in claim 1, characterized in that, The system comprises: A task image lookup table is used to predefine task images, and the task images are at least distinguished into light load images and heavy load images, and the light load images are associated with a bypass scheduling strategy flag and an execution time reference threshold, and the heavy load images are associated with an intelligent scheduling strategy flag; A receiving and querying unit is used to receive a computing task, and use the task type ID of the computing task to query the task image lookup table to obtain the associated scheduling strategy flag; A logical judgment unit is used to perform logical judgment based on the obtained scheduling strategy flag; A stateless distribution unit is used to, when the logical judgment unit judges that the scheduling strategy flag is a bypass scheduling strategy flag, bypass the intelligent scheduling unit, and use a preset stateless distribution algorithm to deliver the computing task to a target node in the light load node pool; An intelligent scheduling unit is used to, when the logical judgment unit judges that the scheduling strategy flag is an intelligent scheduling strategy flag, select a target node for the computing task within the scope of the heavy load node pool; An execution time monitoring unit is used to monitor the execution time of the computing task executed by the target node in the light load node pool; A downgrade request sending unit is used to, when the execution time monitored by the execution time monitoring unit exceeds the execution time reference threshold associated with the light load image, send an image downgrade request containing the task type ID to the image updating unit; An image updating unit is used to, in response to the image downgrade request, update the task image lookup table to modify the scheduling strategy flag corresponding to the task type ID from the bypass scheduling strategy flag to the intelligent scheduling strategy flag.

Images