An edge AI containerized model dynamic scheduling method

Abstract

The application discloses an edge AI containerized model dynamic scheduling method, which comprises the following steps: collecting multi-dimensional state data of a plurality of edge nodes in an edge network in real time, wherein the multi-dimensional state data comprises node computing resource data, node network resource data and task characteristic data of a to-be-processed task queue; inputting the multi-dimensional state data into a pre-trained time series prediction model to output predicted state data in a future preset time window, wherein the predicted state data comprises predicted node load and predicted network bandwidth; jointly forming a joint state space of a system at a current moment with the multi-dimensional state data and the predicted state data, and inputting the joint state space into a pre-trained deep reinforcement learning model to output a scheduling action for a to-be-scheduled AI model container; and controlling an edge scheduler to execute the scheduling action. The application combines time series prediction, multi-target deep reinforcement learning and a dynamic weight mechanism, and thus achieves lower task processing delay, more balanced cluster load distribution and more reliable privacy security guarantee.

Description

Technical Field

[0001] This invention relates to the field of edge AI technology, and in particular to a dynamic scheduling method for containerized edge AI models. Background Technology

[0002] With the deep integration of edge computing and artificial intelligence, deploying AI models in containerized form on edge network nodes to process data generated by IoT terminals locally has become a key paradigm for achieving low-latency, high-privacy intelligent applications. However, the edge network environment is highly dynamic and heterogeneous. The computing resources and network bandwidth of nodes fluctuate over time, and the influx of AI inference tasks also differ in terms of computing requirements, latency constraints, and privacy sensitivities. How to efficiently and intelligently schedule AI model containers in such a complex and dynamic environment is the core challenge for improving the overall performance of edge AI systems.

[0003] Existing edge AI task or container scheduling technologies suffer from the following limitations: First, most scheduling strategies make decisions based on system snapshots (such as current CPU utilization and queue length) at the current moment or recent historical moments, which is a passive, reactive scheduling approach. This method cannot anticipate short-term resource and load changes, easily leading to decision failures. For example, scheduling a new task to a currently idle node that is about to experience a surge in load due to predicted tasks may actually increase the overall task completion time. Second, existing methods typically optimize a single objective, such as minimizing average latency, or assign fixed weights to multiple competing objectives (such as latency, load balancing, and energy consumption). In real-world edge scenarios, the urgency of tasks, the degree of system load imbalance, and the remaining energy of nodes are constantly changing. Fixed-weight multi-objective optimization cannot make optimal trade-offs under different emergency situations, such as "task deadline approaching" and "system energy consumption critically low," and the scheduling strategy lacks scenario adaptability. Furthermore, traditional scheduling often treats tasks as homogeneous computing units and nodes as homogeneous pools of computing resources. In reality, AI inference tasks may have varying levels of privacy requirements for data processing, and the trustworthiness of edge nodes differs due to their deployment environment, security configuration, and historical behavior. Existing solutions lack a mechanism to match task privacy requirements with node trustworthiness, posing a risk of privacy breaches. Therefore, there is an urgent need in this field to provide an intelligent container scheduling method that can adapt to dynamic edge environments, perform multi-objective adaptive trade-offs, and ensure differentiated task requirements. Summary of the Invention

[0004] To address at least one of the aforementioned technical problems, this invention provides a dynamic scheduling method for edge AI containerization models.

[0005] In a first aspect, the present invention provides a method for dynamic scheduling of edge AI containerized models, the method comprising:

[0006] Real-time acquisition of multi-dimensional status data of multiple edge nodes in the edge network, including node computing resource data, node network resource data, and task feature data of the task queue to be processed;

[0007] Multidimensional state data is input into a pre-trained time series prediction model, and the predicted state data within a future preset time window is output. The predicted state data includes predicted node load and predicted network bandwidth.

[0008] The multidimensional state data and the predicted state data are combined to form the joint state space of the system at the current moment, which is then input into a pre-trained deep reinforcement learning model to output scheduling actions for the AI ​​model container to be scheduled.

[0009] The control edge scheduler executes scheduling actions, which include at least one of determining the target edge node, determining the model inference accuracy level, and determining the task execution priority.

[0010] Preferably, the deep reinforcement learning model is trained using a multi-objective adaptive reward mechanism; the scheduling action is a binary tuple, derived by solving a joint utility maximization problem, including:

[0011]

[0012] The maximization operation represents the goal of a deep reinforcement learning agent to find an optimal action in the entire possible action space A. This maximizes the long-term cumulative reward value within the parentheses;

[0013] Represents the mathematical expectation. As a discount factor, ;

[0014] Indicates time The further away the reward is, the lower its weight.

[0015] This represents the end-to-end latency of the task, which is the sum of transmission time, queuing time, and model inference computation time.

[0016] is the node load balancing metric, and is the reciprocal of the load variance;

[0017] This refers to system energy consumption;

[0018] For a moment The latency-sensitive dynamic weight is a function of the remaining task lifetime;

[0019] ;

[0020] As the set base weight, As a growth rate adjustment factor, ; These are the task deadline and the current system time, respectively. It is a natural constant;

[0021] For a moment The load balancing dynamic weight is a function of the variance of the load of all nodes in the network;

[0022] For a moment The energy-sensitive dynamic weight is a function of the node's remaining power.

[0023] Preferably, after the deep reinforcement learning model is deployed, online updates are performed using shadow mode, including:

[0024] Before deploying the new model, the system replays the real state sequence of the most recent M time steps in a sandbox environment;

[0025] Calculate the divergence of action distributions between the new model policy and the old model policy;

[0026] When the virtual cumulative reward of the new model is better than the preset ratio of the old model, and the divergence of the action distribution is lower than the safety threshold, the parameters of the new model are smoothly integrated into the online model using an exponential moving average method to update the model.

[0027] Preferably, the method further includes determining the inference accuracy level of the time series prediction model, including:

[0028] Construct accuracy-delay Pareto frontier curves to evaluate model file size and inference accuracy at different accuracy levels in real time;

[0029] When predicting network bandwidth fluctuations, calculate the marginal revenue ratio and determine whether the marginal revenue ratio is greater than the system's preset tolerance coefficient.

[0030] If so, execute the model compression command;

[0031] If not, schedule to a node with higher available bandwidth.

[0032] Preferably, the task feature data includes task privacy sensitivity tags;

[0033] The state space of the deep reinforcement learning model also includes task privacy sensitivity labels, and its action space includes constraints on the trust level of nodes.

[0034] When determining the target edge node, the scheduling action must ensure that the trust level of the target edge node is not lower than the threshold required by the task privacy sensitivity label.

[0035] Preferably, the trust level of the node is dynamically changing and is calculated by comprehensively considering the node's security baseline score, historical violation records, and the current network security level.

[0036] Preferably, the time-series prediction model is a long short-term memory network or a gated recurrent unit; the deep reinforcement learning model is a deep Q-network or a proximal policy optimization (PPO) algorithm model.

[0037] Secondly, the present invention also provides a dynamic scheduling system for edge AI containerized models, the system comprising:

[0038] The data acquisition module is used to collect multi-dimensional status data of multiple edge nodes in the edge network in real time. The multi-dimensional status data includes node computing resource data, node network resource data, and task feature data of the task queue to be processed.

[0039] The time series prediction module is used to input multidimensional state data into a pre-trained time series prediction model and output predicted state data within a future preset time window. The predicted state data includes predicted node load and predicted network bandwidth.

[0040] The reinforcement learning module is used to combine multi-dimensional state data and predicted state data to form the joint state space of the system at the current moment, input it into the pre-trained deep reinforcement learning model, and output the scheduling action for the AI ​​model container to be scheduled.

[0041] The scheduling execution module is used to control the edge scheduler to perform scheduling actions, which include at least one of determining the target edge node, determining the model inference accuracy level, and determining the task execution priority.

[0042] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect above and any possible implementation thereof.

[0043] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor of an electronic device, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention obtains predicted node load and predicted network bandwidth within a preset future time window by real-time acquisition of multi-dimensional state data and inputting it into a pre-trained time-series prediction model (such as LSTM or GRU). The current state and predicted state are combined to form a joint state space, which is then input into the decision model. This ensures that scheduling decisions are not only based on the current situation but also fully consider near-future trends. This effectively avoids assigning tasks to nodes that are about to become overloaded or experience network congestion, thus significantly reducing queuing time and task failure rates caused by resource contention and improving the long-term effectiveness of scheduling decisions. A deep reinforcement learning model trained based on a multi-objective adaptive reward mechanism is employed, and a dynamic weight function is designed to guide model training and decision-making: latency-sensitive weights increase exponentially as the remaining task lifetime decreases, ensuring that urgent tasks are prioritized for scheduling; load balancing weights increase as the overall network load variance increases, prompting the system to proactively balance loads when uneven; and energy-sensitive weights adjust as the remaining power of nodes decreases, guiding the system to conserve energy when power is scarce. This mechanism enables the scheduling strategy to perceive the "urgency" of different dimensions of the system in real time and automatically adjust the optimization focus to achieve a dynamic optimal balance between task latency, cluster load balancing and total system energy consumption. Compared with the fixed weight strategy, it significantly improves overall performance in complex and changing scenarios.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the accompanying drawings used in the embodiments of the present invention or the background art will be described below.

[0048] Figure 1 A flowchart illustrating a dynamic scheduling method for an edge AI containerized model provided in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of a dynamic scheduling system for an edge AI containerized model provided in an embodiment of the present invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] Please see Figure 1 , Figure 1 This is a flowchart illustrating a dynamic scheduling method for edge AI containerized models provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0053] S10. Real-time acquisition of multi-dimensional status data of multiple edge nodes in the edge network, the multi-dimensional status data including node computing resource data, node network resource data and task feature data of the task queue to be processed;

[0054] S20. Input the multidimensional state data into the pre-trained time series prediction model and output the predicted state data within the future preset time window. The predicted state data includes the predicted node load and the predicted network bandwidth.

[0055] S30. Combine the multidimensional state data and the predicted state data to form the joint state space of the system at the current moment, input it into the pre-trained deep reinforcement learning model, and output the scheduling action for the AI ​​model container to be scheduled.

[0056] S40. Control the edge scheduler to perform scheduling actions, the scheduling actions including at least one of determining the target edge node, determining the model inference accuracy level, and determining the task execution priority.

[0057] In this embodiment, the system continuously collects real-time computing and network resource data, as well as task queue characteristics, from edge nodes. Then, it uses a time-series prediction model to analyze this historical and current data, predicting the load and network bandwidth trends of each node in the near future. Next, it combines the current real-time state with the predicted future state to form a joint state containing both current and forward-looking information, which is then input into a deep reinforcement learning model. This model, based on a multi-objective optimization strategy, outputs a comprehensive optimal scheduling action instruction. Finally, the scheduler executes this instruction to dynamically schedule the AI ​​model container, specifically selecting the most suitable target node, determining the model's inference accuracy level, and prioritizing task execution.

[0058] Preferably, the time-series prediction model is a Long Short-Term Memory (LSTM) network or a gated recurrent unit (GRU); the deep reinforcement learning model is a deep Q-network or a proximal policy optimization (PPO) algorithm model. LSTM, with its unique forget gate, input gate, and output gate structure, can effectively capture the load cycle patterns over long periods. When deploying GRUs on resource-constrained edge nodes, it can quickly output predicted state data for future time windows while using less memory, thus gaining valuable response time for subsequent scheduling decisions. When using a deep Q-network (DQN), the experience replay pool and target network mechanism effectively solve the problems of sample correlation and target fluctuation during reinforcement learning training, making it suitable for discrete scheduling scenarios (such as selecting a target node from multiple edge nodes or selecting a level from multiple preset accuracy levels). When using a proximal policy optimization (PPO), the pruned alternative objective function limits the single update step size during policy updates, avoiding performance crashes caused by policy mutations. PPO is particularly suitable for continuous control scenarios (such as dynamically adjusting the CPU share and network bandwidth ratio allocated to each container), enabling more refined resource allocation while ensuring training stability.

[0059] Through the above process, this method achieves intelligent forward-looking scheduling and dynamic trade-offs among multiple objectives. First, prediction-based scheduling overcomes the short-sightedness of traditional methods that rely solely on the current state. By anticipating resource change trends, it effectively avoids assigning tasks to nodes that are about to become overloaded or experience network congestion, thereby reducing task queuing latency and failure rates. Second, deep reinforcement learning-based decision-making comprehensively considers multiple competing objectives such as latency, load balancing, and energy consumption. Its adaptive reward mechanism dynamically adjusts the optimization focus; for example, it automatically prioritizes latency when task deadlines are approaching and proactively guides load balancing when system load is uneven, thus achieving continuous optimization of overall system performance in complex dynamic environments. Third, flexible scheduling actions allow the system not only to choose where tasks are executed but also to determine the level of precision required. This enables it to make optimal trade-offs between transmission time and inference accuracy when network bandwidth fluctuates, improving task processing success rate and system efficiency under resource-constrained conditions.

[0060] In one embodiment, the purpose of collecting node computing and network resource data is to assess real-time processing and transmission capabilities. CPU / GPU / memory data directly reflects the current "computing power" saturation level of a node, serving as a core basis for determining its ability to receive new tasks and the expected execution time of tasks. Network bandwidth and latency data characterize the "transmission channel" status of data, determining the speed of model container distribution or task data return. The purpose of collecting task queue feature data is to achieve differentiated and constraint-aware scheduling: different tasks vary greatly in computing requirements, urgency (deadlines), and security requirements. Collecting these features allows the scheduling system to differentiate tasks rather than treating them all the same. Specifically, node computing resource data is collected by deploying lightweight monitoring agents on various edge nodes and reading the performance interfaces provided by the operating system in real time; node network resource data is collected by combining local node monitoring and network probing, for example, uplink / downlink bandwidth availability is estimated by periodically sending standardized data packets to neighboring nodes or a central measurement point. Network round-trip time (RTT) and jitter are measured by sending ICMP Ping packets or specific probe messages to measure the latency to critical nodes or gateways. Network interface status and packet loss rate are obtained by reading / proc / net / dev or network interface counters. Task characteristic data for the pending task queue is collected directly from task metadata extracted from the task queue manager or message broker on the edge nodes.

[0061] In one embodiment, the deep reinforcement learning model is trained using a multi-objective adaptive reward mechanism; the scheduling action is a binary tuple, derived by solving a joint utility maximization problem, including:

[0062]

[0063] The maximization operation represents the goal of a deep reinforcement learning agent to find an optimal action in the entire possible action space A. This maximizes the long-term cumulative reward value within the parentheses;

[0064] Represents the mathematical expectation. As a discount factor, ;

[0065] Indicates time The further away the reward is, the lower its weight.

[0066] This represents the end-to-end latency of the task, which is the sum of transmission time, queuing time, and model inference computation time.

[0067] is the node load balancing metric, and is the reciprocal of the load variance;

[0068] This refers to system energy consumption;

[0069] For a moment The latency-sensitive dynamic weight is a function of the remaining task lifetime;

[0070] ;

[0071] As the set base weight, As a growth rate adjustment factor, ; These are the task deadline and the current system time, respectively. It is a natural constant;

[0072] For a moment The load balancing dynamic weight is a function of the variance of the load of all nodes in the network;

[0073] For a moment The energy-sensitive dynamic weight is a function of the node's remaining power.

[0074] In the above formula, due to the randomness of the edge environment (such as network jitter, sudden tasks), the future state and reward are uncertain. Next, we take the mathematical expectation. This indicates that the optimization aims to improve the long-term average return, rather than the instantaneous return of a specific run, reflecting the algorithm's robustness. Cumulative summation emphasizes that the scheduling strategy not only focuses on the "current" optimality but also on the long-term global optimality over the entire time window, avoiding short-sighted behavior (such as causing widespread timeouts in subsequent tasks to save a little power). This means that the smaller the delay, the greater the reward. The reciprocal form is used to transform the goal of "minimizing delay" into the mathematical form of "maximizing reward". It is used to measure the uniformity of resource utilization across edge nodes in the system, preventing all tasks from flooding the same high-performance node and causing congestion while other nodes remain idle. This indicates that energy consumption is a penalty; the higher the energy consumption, the lower the total reward. This forces the model to choose low-power nodes or low-precision models as much as possible while meeting latency requirements. In computation... During the process, as the task approaches the deadline, the denominator approaches 0, and the exponent increases sharply. It is growing exponentially. This indicates the level of importance placed on system load balancing. When the system detects that some nodes are overloaded while others are idle (high variance), The weight is automatically increased, guiding the scheduler to distribute new tasks to idle nodes; when the system load is relatively even, this weight is reduced to give way to latency or energy consumption targets. This indicates the level of importance attached to energy consumption control; when the edge node's battery power is low, or during peak electricity consumption periods, The increased size forces the model to prioritize lightweight compressed models (low precision) or dormant nodes to extend the system's lifespan.

[0075] In this embodiment, the system no longer mechanically executes fixed optimization formulas. For example, when an emergency task (such as obstacle recognition in autonomous driving) arises, It will automatically take over, and the scheduling strategy will immediately switch to "delay-first" mode to ensure that critical tasks are completed in a timely manner. After urgent tasks are completed, the system load may be uneven. As the weight increases, the strategy switches back to "balanced resource" mode. This dynamic priority switching is something a fixed-weight strategy cannot achieve. The exponential growth mechanism gives tasks nearing their deadlines extremely high scheduling priority. This effectively reduces the probability of tasks failing due to queuing or improper scheduling, especially for latency-sensitive AI applications (such as interactive AR and industrial quality inspection), significantly improving service reliability and user experience. This mechanism enables the system to have negative feedback regulation capabilities. Once a load hotspot occurs, the system will automatically increase the incentive to schedule to idle nodes, preventing individual nodes from overloading and crashing, thus improving the stability and throughput of the cluster. The introduction of this mechanism enables energy-conscious scheduling strategies. In scenarios where edge nodes are powered by batteries (such as mobile vehicle networks and drones), this mechanism can intelligently avoid concentrating the load on nodes that are about to run out of power, achieving a balanced distribution of energy across the load and thus extending the sustainable operating time of the entire edge network.

[0076] Furthermore, deep reinforcement learning models make decisions by maximizing the long-term cumulative discounted reward starting from the current state. This means that when choosing a current action, the agent has already inherently considered the potential impact of that action on the future state of the system (such as future load and future energy consumption). It may accept a slight short-term increase in latency for a task in order to achieve long-term load balancing and energy savings. In other words, the model possesses forward-looking global optimization capabilities.

[0077] In one embodiment, after the deep reinforcement learning model is deployed, online updates are performed using a shadow mode, including:

[0078] Before deploying the new model, the system replays the real state sequence of the most recent M time steps in a sandbox environment;

[0079] Calculate the divergence of action distributions between the new model policy and the old model policy;

[0080] When the virtual cumulative reward of the new model is better than the preset ratio of the old model, and the divergence of the action distribution is lower than the safety threshold, the parameters of the new model are smoothly integrated into the online model using an exponential moving average method to update the model.

[0081] Step 1: Sandbox Environment Construction and Historical State Replay:

[0082] Maintaining a historical state buffer: When the online scheduling system is running continuously, the joint state space data observed at each scheduling time t (i.e., the combination of real-time collected multidimensional state data and predicted state data) is synchronously stored in a fixed-capacity first-in-first-out buffer.

[0083] Create a sandbox environment: Establish a sandbox environment on the deployment server that is logically identical to the online scheduling environment but isolated from the actual production nodes. This environment includes a simulated task generator and a node state simulator, but its core function is to load and run both old and new policy models.

[0084] Replay Evaluation: After training a new candidate model based on new data, the online model is not directly replaced. Instead, the real state sequences of the most recent M time steps (e.g., the state sequences corresponding to the past hour) in the historical buffer are sequentially input into the sandbox environment.

[0085] Virtual simulation: In a sandbox, decision-making simulations are performed on the same state sequence using both the online old model and candidate new models. The system records the actions (i.e., scheduling decisions) output by each model in each state and calculates the virtual cumulative reward for each model on this historical sequence based on the results calculated by the environment simulator (such as actual task latency, energy consumption, etc.).

[0086] Step Two: Dual Security Assessment

[0087] Performance Evaluation: Compare the virtual cumulative rewards of the new and old models on the replay sequence. Set a threshold for the increase ratio (e.g., requiring the new model's reward to increase by at least 5%). The first evaluation is passed only if (new model reward - old model reward) / old model reward > the preset ratio.

[0088] Behavioral safety assessment: Calculate the divergence of the action distribution output by the old and new models under the same state sequence. This is typically measured using KL divergence or JS divergence to measure the difference between the two policy distributions.

[0089] Specific calculations: For each state in the replay sequence, the model outputs an action probability distribution (which can be considered a one-hot distribution for deterministic policies). Calculate the average action distribution divergence of the old and new models across all M states.

[0090] Set a safety threshold: Set a small divergence threshold (e.g., KL divergence < 0.05). This threshold ensures that the decision-making behavior of the new model does not deviate drastically or unpredictably from that of the old model, avoiding abnormal scheduling of the production system due to sudden policy changes (such as suddenly concentrating all tasks on a single node).

[0091] Decision: The new model will only proceed to the update process if it passes both the performance improvement and behavioral safety assessments; otherwise, the old model will be retained, and the update failure will be recorded for analysis.

[0092] Step 3: Smooth parameter fusion update:

[0093] Exponential Moving Average Update: Instead of direct model replacement, this method uses an exponential moving average to smoothly integrate the parameters of the new model into the online model. This approach is equivalent to injecting only a small portion of the new model's "knowledge" into the old model each time, achieving a gradual update. The performance and behavior of the online model slowly and steadily converge to the new model, avoiding performance fluctuations or service interruptions caused by sudden parameter changes. After parameter fusion is complete, the online scheduler immediately begins using the updated model for decision-making. The old model file is archived and saved.

[0094] By employing a shadow mode for online updates, the new model is first validated in a sandbox environment using recent real-world state sequences before deployment. The improvement in virtual cumulative rewards and the divergence in action distribution compared to the old strategy are then rigorously evaluated. Only when the new strategy is confirmed to be superior and the behavior does not deviate drastically is the parameter fusion smoothed using an exponential moving average. This process significantly reduces the risk of performance regression or unstable behavior introduced by model updates, ensuring the continuous and stable operation of the production system and achieving a "safe evolution" of the AI ​​scheduling brain.

[0095] In one embodiment, the method further includes determining the inference accuracy level of the time series prediction model, including:

[0096] Construct accuracy-delay Pareto frontier curves to evaluate model file size and inference accuracy at different accuracy levels in real time;

[0097] When predicting network bandwidth fluctuations, calculate the marginal revenue ratio and determine whether the marginal revenue ratio is greater than the system's preset tolerance coefficient.

[0098] If so, execute the model compression command;

[0099] If not, schedule to a node with higher available bandwidth.

[0100] In this embodiment, multiple versions with different precisions (such as 32-bit floating-point, 16-bit floating-point, 8-bit integer quantization, and sparse models with different pruning rates) are prepared in advance for each deployable AI containerized model. Each version needs to have its key parameters calibrated: model file size (directly affecting transmission time); theoretical computational cost (affecting inference computation time on the target node); and baseline accuracy (the accuracy evaluated on a standard dataset). Then, on typical edge node hardware, the "end-to-end inference latency" (including the time from loading to output) and "inference accuracy" corresponding to each precision version model are measured and recorded. Points for all versions are plotted on a "latency-accuracy" coordinate graph, and those points with the highest accuracy at the same latency or the lowest latency at the same accuracy are selected and connected to form a Pareto front curve. This curve shows the optimal "latency-accuracy" tradeoff boundary achievable with the current hardware. Finally, the model versions located on the Pareto front are officially registered as dynamically selectable precision levels, and a profile is created for each level, including its model file size, expected accuracy, and inference latency.

[0101] In the real-time evaluation and decision-making phase, the triggering condition is typically that the time-series prediction model predicts a significant decrease in the network available bandwidth of the target node (or the current optimal node) at a certain future time, potentially becoming a bottleneck for task completion time. For the current task to be scheduled and the candidate accuracy levels, performance predictions for two alternative schemes are calculated:

[0102] Strategy 1 (Model Compression): Keep the scheduling to the original target node unchanged, but use a model version with a lower precision level.

[0103] Calculate the benefits: Evaluate the network transmission time saved by switching to a lower-precision model due to the reduced model file size.

[0104] Computational cost: Evaluate the extent to which inference accuracy decreases due to reduced model precision.

[0105] Strategy 2 (Node Switching): Keep the original high-precision model in use, but reschedule the task to another available node with more current prediction bandwidth.

[0106] Calculate the benefits: Evaluate the time savings in transferring high-precision model files after switching to a high-bandwidth node.

[0107] Computational cost: Evaluate the additional inference computation or queuing delay that a new target node may cause due to different computing capabilities or higher load.

[0108] Decisions are made based on the "marginal benefit ratio": The system calculates the "marginal benefit ratio" of Strategy One (compressed model). This ratio can be understood as: "the reduction in total task latency that can be achieved by sacrificing 1 percentage point of model accuracy." This ratio is then compared with a system-preset tolerance coefficient. This tolerance coefficient is determined by business needs and represents the level of "trading accuracy for speed" that the business can accept.

[0109] If the marginal benefit ratio is greater than the tolerance coefficient, it means that the "compressed model" strategy is very efficient, trading a small amount of accuracy loss for a large amount of time savings. The system then executes the model compression instruction and uses a lower-precision version for scheduling.

[0110] If the marginal revenue ratio is less than or equal to the tolerance coefficient, it means that the "compressed model" is not cost-effective. The system will then abandon compression, execute a node switching command, and find and schedule a node with higher idle bandwidth that is sufficient to support the rapid transmission of the original precision model.

[0111] If neither of the above two strategies can meet the task deadline requirements (for example, all nodes have extremely low bandwidth and even using the lowest precision model is not enough time), the system will trigger the task failure or degradation process in advance to avoid unnecessary resource consumption.

[0112] By constructing a precision-latency Pareto front curve and calculating the marginal benefit ratio of different decisions (such as model compression or selecting high-bandwidth nodes) when network bandwidth fluctuations are predicted, the system can intelligently choose between "reducing model precision to decrease transmission volume and shorten transmission time" and "finding a better network path to transmit the original model." This mechanism fully utilizes the scalability of AI models, ensuring timely task processing even when network resources are limited through the coordinated optimization of precision and resources, thereby improving the overall throughput and service availability of the system under harsh network conditions.

[0113] In one embodiment, the task feature data includes a task privacy sensitivity label;

[0114] The state space of the deep reinforcement learning model also includes task privacy sensitivity labels, and its action space includes constraints on the trust level of nodes.

[0115] When determining the target edge node, the scheduling action must ensure that the trust level of the target edge node is not lower than the threshold required by the task privacy sensitivity label.

[0116] When a task is submitted, the system automatically or manually assigns a privacy sensitivity label based on the task's data content, industry regulations, and user settings. This label is typically a discrete level (e.g., L1 Public Data, L2 General Privacy, L3 Sensitive Privacy, L4 Highly Sensitive). The criteria for determination include: the type of data processed by the task (e.g., face, voice, medical images, location tracking), the compliance scope of the data, and the security requirements explicitly specified by the task owner.

[0117] Each edge node maintains a dynamically updated trust level. This level is not fixed but is calculated periodically by a trust assessment engine. The assessment inputs include:

[0118] Security baseline score: Scoring is based on the security configuration of the node (such as firewall rules, encryption protocol strength, and intrusion detection system status).

[0119] Historical violation records: Whether the node has a history of unauthorized data access, abnormal external connections, or violations of scheduling policies.

[0120] Environmental security level: The security of the physical and network environment in which the node is located (such as a data center deployed within a firewall, or a gateway deployed in a public place).

[0121] The evaluation engine takes into account the above factors and calculates the current trust level of the node (for example, corresponding to the sensitivity label, it is also divided into level 1 to 4 or higher).

[0122] When a deep reinforcement learning model makes decisions, the input state information must include not only resource data but also the privacy sensitivity label of the task to be scheduled. The model's output action (i.e., "which target node to select") is subject to a hard constraint: a node is only considered if its current dynamic trust level is greater than or equal to the minimum trust level threshold required by the task's privacy sensitivity label. For example, a task labeled "L3 sensitive privacy" requires a minimum trust level of 3. Therefore, the model can only select targets from nodes with a current trust level of 3 or higher. A node with very idle resources but a trust level of only 2 will be automatically excluded.

[0123] During the training of a deep reinforcement learning model, the aforementioned constraints are reflected through a reward function or action masking mechanism. During the exploratory learning process, the model is significantly penalized for attempting to schedule highly sensitive tasks to low-trust nodes, thus learning to proactively seek out and prioritize nodes that satisfy both resource optimization and security trust conditions. When the system evaluation finds that the trust level of all currently available nodes cannot meet the requirements of a certain highly sensitive task, an alternative strategy will be triggered:

[0124] Local processing or rejection: It is strongly recommended that the task initiator perform local processing at the data source (such as the terminal device). If local processing is not possible, the task may be delayed or return a failure to ensure that the data never enters a low-security environment.

[0125] Cloud rollback: When conditions permit and compliance requirements are met, the task is routed to a central cloud with a higher trust level for processing.

[0126] By introducing task privacy sensitivity labels and dynamic node trust levels, and incorporating both into the state and action constraint space of a deep reinforcement learning model, scheduling mandates that the target node's trust level must not be lower than the threshold required by the task. The node's trust level is dynamically calculated from multiple dimensions, including security baseline and historical behavior. This constructs a fine-grained, dynamic trust and demand matching framework, ensuring that highly privacy-sensitive tasks are always scheduled to high-trust nodes for execution. This fundamentally reduces the risk of privacy leaks caused by sensitive data being processed on low-security-level nodes, and enhances the system's ability to support tasks with diverse security requirements.

[0127] In one embodiment, the trust level of the node is dynamically changed and is calculated by combining the node's security baseline score, historical violation records, and the environmental security level of the currently accessed network.

[0128] The system deploys a lightweight security monitoring agent on each edge node to continuously collect data from three aspects.

[0129] Security baseline scoring: The agent periodically performs local security scans to check and evaluate the security configuration status of nodes, such as: operating system patch levels, firewall rule strength, container runtime security configuration, and encryption protocol enabling status. Compliance checks are performed based on predefined security baselines (such as CIS standards), and a quantitative score is generated.

[0130] Historical violation records: Security event logs for system maintenance nodes. Records include: unauthorized data access attempts, abnormal network connections, resource abuse, and scheduling policy violations (such as attempting to process tasks exceeding their trust level). Each record is marked with its severity level and the time of occurrence.

[0131] Environmental security level: Assess the physical and logical environment in which the node is located. The system dynamically assigns an environmental security level based on the type of network the node accesses (such as a private VPN, corporate intranet, or public cellular network), its physical location (such as a controlled server room, a public camera post, or a moving vehicle), and current network threat intelligence (such as whether the subnet is under scanning attack).

[0132] A centralized or regional trust assessment engine periodically (e.g., every 5 minutes) performs the following calculations:

[0133] Data standardization: The collected multidimensional data is uniformly quantified into standard scores. For example, the safety baseline score directly adopts a percentage system; historical violation records are weighted and deducted points based on the severity and recentity of the incident (more recent and serious violations are penalized more severely); environmental safety level is mapped to a base score.

[0134] Comprehensive score calculation: The three standardized scores are combined into a single comprehensive trust score using either a weighted average or a rule-based engine approach. For example, "security baseline" and "historical violations" are given higher weights because they directly reflect the node's own reliability; "environmental security level" serves as an adjustment factor, temporarily lowering the score in adverse environments.

[0135] Level mapping: The calculated overall trust score is mapped to discrete trust levels (e.g., levels 1-5). This level is the label directly used in scheduling decisions.

[0136] The assessment engine runs periodically, generating the latest trust level for each node after each calculation. Trust level updates are synchronized in real-time to the scheduling decision center and all relevant nodes via a secure signaling channel. When a node experiences a major security incident (such as detected intrusion), an immediate assessment can be triggered, enabling a downgrade and real-time update of the trust level.

[0137] This embodiment achieves real-time perception and quantification of security posture, transforming the originally ambiguous question of "whether a node is secure" into a clear, quantifiable, and comparable dynamic level. This allows the scheduling system to monitor the constantly changing security status of the entire edge network, providing precise data input for decision-making. Trust levels are dynamic. When a node's trust level drops due to lax security configuration, attack, or entry into an insecure environment, the scheduling system can immediately detect it. Subsequent high-privacy-sensitive tasks will automatically avoid this node, greatly limiting the scope of security risk impact. Static trust levels cannot cope with the dynamic changes in the edge environment's security, potentially leading to highly sensitive tasks being scheduled to a compromised but unnoticed "statically high-trust" node. The dynamic mechanism ensures that the "node trust level" upon which the matching rules rely always accurately reflects the current security status, thus guaranteeing the real-time effectiveness of this security constraint mechanism. Nodes perceive that their trust level directly affects the tasks they are assigned (higher levels receive more high-value tasks), which incentivizes nodes to proactively maintain good security practices, such as timely patch updates and standardized behavior, thereby improving the overall security baseline level of the edge network.

[0138] See Figure 2 In one embodiment, the present invention also provides a dynamic scheduling system for edge AI containerized models, the system comprising:

[0139] The data acquisition module 100 is used to collect multi-dimensional status data of multiple edge nodes in the edge network in real time. The multi-dimensional status data includes node computing resource data, node network resource data, and task feature data of the task queue to be processed.

[0140] The time series prediction module 200 is used to input multidimensional state data into a pre-trained time series prediction model and output predicted state data within a future preset time window. The predicted state data includes predicted node load and predicted network bandwidth.

[0141] The reinforcement learning module 300 is used to combine multi-dimensional state data and predicted state data to form the joint state space of the system at the current moment, input it into the pre-trained deep reinforcement learning model, and output the scheduling action for the AI ​​model container to be scheduled.

[0142] The scheduling execution module 400 is used to control the edge scheduler to perform scheduling actions, the scheduling actions including at least one of determining the target edge node, determining the model inference accuracy level, and determining the task execution priority.

[0143] It is understood that the system provided in this embodiment has functions or includes modules that can be used to execute the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0144] The present invention also provides an electronic device including a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs a method as described in any of the above possible implementations.

[0145] The present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor of an electronic device, cause the processor to perform a method as described in any of the above possible implementations.

[0146] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

Claims

1. A dynamic scheduling method for edge AI containerized models, characterized in that, The method includes: Real-time acquisition of multi-dimensional status data of multiple edge nodes in the edge network, including node computing resource data, node network resource data, and task feature data of the task queue to be processed; Multidimensional state data is input into a pre-trained time series prediction model, and the predicted state data within a future preset time window is output. The predicted state data includes predicted node load and predicted network bandwidth. The multidimensional state data and the predicted state data are combined to form the joint state space of the system at the current moment, which is then input into a pre-trained deep reinforcement learning model to output scheduling actions for the AI ​​model container to be scheduled. The control edge scheduler executes scheduling actions, which include at least one of determining the target edge node, determining the model inference accuracy level, and determining the task execution priority.

2. The dynamic scheduling method for edge AI containerized models according to claim 1, characterized in that, The deep reinforcement learning model is trained using a multi-objective adaptive reward mechanism; The scheduling action is a binary tuple, derived by solving the joint utility maximization problem, and includes: ； The maximization operation represents the goal of a deep reinforcement learning agent to find an optimal action in the entire possible action space A. This maximizes the long-term cumulative reward value within the parentheses; Represents the mathematical expectation. As a discount factor, ; Indicates time The further away the reward is, the lower its weight. This represents the end-to-end latency of the task, which is the sum of transmission time, queuing time, and model inference computation time. is the node load balancing metric, and is the reciprocal of the load variance; This refers to system energy consumption; For a moment The latency-sensitive dynamic weight is a function of the remaining task lifetime; ； As the set base weight, As a growth rate adjustment factor, ; These are the task deadline and the current system time, respectively. It is a natural constant; For a moment The load balancing dynamic weight is a function of the variance of the load of all nodes in the network; For a moment The energy-sensitive dynamic weight is a function of the node's remaining power.

3. The dynamic scheduling method for edge AI containerized models according to claim 1, characterized in that, After the deep reinforcement learning model is deployed, online updates are performed using shadow mode, including: Before deploying the new model, the system replays the real state sequence of the most recent M time steps in a sandbox environment; Calculate the divergence of action distributions between the new model policy and the old model policy; When the virtual cumulative reward of the new model is better than the preset ratio of the old model, and the divergence of the action distribution is lower than the safety threshold, the parameters of the new model are smoothly integrated into the online model using an exponential moving average method to update the model.

4. The dynamic scheduling method for edge AI containerized models according to claim 1, characterized in that, The method further includes determining the inference accuracy level of the time-series prediction model, including: Construct accuracy-delay Pareto frontier curves to evaluate model file size and inference accuracy at different accuracy levels in real time; When predicting network bandwidth fluctuations, calculate the marginal revenue ratio and determine whether the marginal revenue ratio is greater than the system's preset tolerance coefficient. If so, execute the model compression command; If not, schedule to a node with higher available bandwidth.

5. The dynamic scheduling method for edge AI containerized models according to claim 1, characterized in that, The task feature data includes task privacy sensitivity tags; The state space of the deep reinforcement learning model also includes task privacy sensitivity labels, and its action space includes constraints on the trust level of nodes. When determining the target edge node, the scheduling action must ensure that the trust level of the target edge node is not lower than the threshold required by the task privacy sensitivity label.

6. The dynamic scheduling method for edge AI containerized models according to claim 5, characterized in that, The trust level of the node is dynamically changing and is calculated by comprehensively considering the node's security baseline score, historical violation records, and the security level of the current network environment.

7. The dynamic scheduling method for edge AI containerized models according to claim 1, characterized in that, The time-series prediction model is a long short-term memory network or a gated recurrent unit; the deep reinforcement learning model is a deep Q-network or a proximal policy optimization (PPO) algorithm model.

8. A dynamic scheduling system for edge AI containerized models, characterized in that, The system includes: The data acquisition module is used to collect multi-dimensional status data of multiple edge nodes in the edge network in real time. The multi-dimensional status data includes node computing resource data, node network resource data, and task feature data of the task queue to be processed. The time series prediction module is used to input multidimensional state data into a pre-trained time series prediction model and output predicted state data within a future preset time window. The predicted state data includes predicted node load and predicted network bandwidth. The reinforcement learning module is used to combine multi-dimensional state data and predicted state data to form the joint state space of the system at the current moment, input it into the pre-trained deep reinforcement learning model, and output the scheduling action for the AI ​​model container to be scheduled. The scheduling execution module is used to control the edge scheduler to perform scheduling actions, which include at least one of determining the target edge node, determining the model inference accuracy level, and determining the task execution priority.

9. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs the edge AI containerization model dynamic scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor of an electronic device, cause the processor to perform the dynamic scheduling method for edge AI containerization models as described in any one of claims 1 to 7.

Images