An edge AI computing task scheduling method and device for cloud edge collaboration

Abstract

The application discloses an edge AI computing task scheduling method and device for cloud edge cooperation, relates to the technical field of edge computing, and comprises the following steps: continuously acquiring a real-time task set, classifying and priority sorting each randomly generated task sequence to obtain a sorted sequence; performing task scheduling according to each sorted sequence and recording the corresponding task execution time; predicting each sorted sequence corresponding to a given to-be-allocated task set through a first model, and performing task scheduling on the sorted sequence with the smallest task execution time prediction value; predicting a newly added to-be-allocated task set through a second model, obtaining an optimal task queue, and performing task scheduling according to the optimal task queue. The application improves the task waiting time and improves the resource utilization rate of the edge server, and solves the problem that when a large number of task requests are sent by users and a cloud computing center, a great burden and energy consumption are brought to the edge server.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and in particular to a method and apparatus for scheduling edge AI computing tasks for cloud-edge collaboration. Background Technology

[0002] The main characteristic of edge computing is that it extends cloud services to the network edge, significantly reducing latency while improving the resource utilization of network and computing devices. As a natural extension of cloud computing, edge computing continues various research challenges and raises new problems that need to be solved. Task scheduling is a key aspect of edge computing; it is the process by which optimized tasks are mapped to appropriate resource pools for execution. Scheduling algorithms aim to optimize various service parameters, such as cost, completion time, scalability, reliability, resource utilization, and energy consumption.

[0003] Because frequent data requests in edge environments can lead to network congestion and latency, edge scheduling technology is a crucial method for reducing latency and the latency sensitivity of video transmission tasks. Therefore, edge computing scheduling and allocation technology is a hot topic in mobile edge computing, providing possibilities for the development of 5G. With the combination of 5G and edge computing meeting the needs of autonomous driving, whether facing the demands of high-performance computing or the need for network latency in computationally intensive application services, the large number of task requests sent by users and cloud computing centers places a significant burden and consumes considerable energy on edge servers. In edge computing, task scheduling not only affects the time and cost for users and cloud computing centers but also plays a significant role in the performance of edge servers. Therefore, achieving efficient task scheduling has become an important issue in the field of edge cloud research.

[0004] Existing edge computing task scheduling technologies have the following core shortcomings:

[0005] a. Insufficient dynamic resource matching capability:

[0006] Traditional scheduling algorithms (such as FIFO, priority queues, and round-robin scheduling) rely on static rules (such as task arrival time and preset priority) and do not consider the dynamic relationship between task type and resource requirements. For example, CPU-intensive tasks and NPU-intensive tasks in the same queue may experience a surge in execution latency due to resource contention (as experimental data shows, the latency of a hybrid queue is 40%-60% higher than that of a dedicated queue).

[0007] b. Execution efficiency bottleneck under massive tasks:

[0008] When edge nodes face a sudden surge of tasks (such as multiple cameras simultaneously triggering AI analysis in smart cities), traditional strategies (such as shortest job first) lack the ability to predict the overall execution time of the queue and cannot generate a globally optimal arrangement. According to an industry report (IDC, 2023), as AI computing power migrates to the edge, the burden of edge computing tasks will become increasingly severe, and massive tasks will require better execution scheduling technology.

[0009] c. The value of historical data has not been tapped:

[0010] Existing methods (such as rule-based heuristic algorithms) do not utilize historical task execution data to model the complex nonlinear relationship between task characteristics, resource consumption, and execution time, making it difficult for scheduling strategies to adapt to environmental changes (such as node resource fluctuations and task type distribution shifts).

[0011] Therefore, a method and device for scheduling edge AI computing tasks for cloud-edge collaboration were developed to solve the above problems. Summary of the Invention

[0012] This invention proposes an edge AI computing task scheduling method and device for cloud-edge collaboration, in order to solve the problem that when users and cloud computing centers send a large number of task requests, it will bring a huge burden and energy consumption to the edge server.

[0013] The present invention achieves the above objectives through the following technical solutions:

[0014] This invention provides a method for scheduling edge AI computing tasks for cloud-edge collaboration, comprising:

[0015] Continuously acquire real-time task sets, randomly generate several different task sequences based on the task sets, and classify and prioritize each randomly generated task sequence according to task type and task source to obtain a sorted sequence;

[0016] In the first stage, online, tasks are scheduled and the corresponding task execution time is recorded according to each sorted sequence. Offline, all sorted sequences are used as input and the corresponding task execution time is used as output to train the first model.

[0017] Once the confidence level of the first model reaches the first preset threshold, the second stage begins. The trained first model is then deployed online. Online, the first model is used to predict each sorted sequence corresponding to the given set of tasks to be assigned, and the sorted sequence with the smallest predicted task execution time is selected for task scheduling. Offline, the second model is trained based on the given set of tasks to be assigned and the sorted sequence with the smallest predicted task execution time.

[0018] Once the confidence level of the second model reaches the second preset threshold, the third stage begins. The trained second model is then deployed online. The second model is used to predict the newly added set of tasks to be assigned, obtain the optimal task queue, and schedule tasks based on the optimal task queue.

[0019] Both the first model and the second model are preset machine learning prediction models.

[0020] Furthermore, the task types include CPU-intensive tasks, disk I / O-intensive tasks, NPU-intensive tasks, and network I / O-intensive tasks, and the task sources include Master tasks, Partner tasks, and user tasks.

[0021] Furthermore, randomly generated task sequences are classified based on task type, including:

[0022] Based on the characteristics of task resource consumption and dependencies, tasks are categorized into different types to obtain task resource demand rates.

[0023] Calculate the average and variance of resource demand rates for different tasks in historical data, where the historical data consists of collected task logs that have been executed in the system in the past, including:

[0024] Calculate the historical average and average variance of resource demand rates for CPU-intensive tasks:

[0025]

[0026]

[0027] =

[0028] =

[0029] =

[0030] =

[0031] =

[0032] =

[0033] in, This represents the average CPU utilization of CPU-intensive tasks in historical data. The average variance of the CPU. This represents the average disk utilization rate for CPU-intensive tasks in historical data. The disk average variance This represents the average NPU utilization for CPU-intensive tasks in historical data. The average variance of the NPU. This represents the average bandwidth utilization of CPU-intensive tasks in historical data. The average variance of the bandwidth. , , and The following are the CPU utilization, disk utilization, NPU utilization, and bandwidth utilization of the i-th CPU-intensive task, in order. This represents the number of times CPU-intensive tasks occurred in historical data.

[0034] Similarly, calculate the average demand rate and average variance in historical data for disk I / O intensive tasks;

[0035] Similarly, calculate the average demand rate and average variance in historical data for NPU-intensive tasks;

[0036] Set the maximum values ​​for the resource demand metrics to be: CPU maximum utilization (maxc), NPU maximum utilization (maxp), disk maximum utilization (maxd), bandwidth maximum utilization (maxn), and set the variance threshold for all resource demand metrics.

[0037] If the resource demand rate does not meet the requirements, and the occupancy rate of a certain resource demand exceeds the maximum value or the threshold, the task cannot proceed to the next step and will enter a reload state.

[0038] Furthermore, the randomly generated task sequences are prioritized based on their source, including:

[0039] For each specific task queue, the actual execution time of each task j is recorded. M is the number of tasks, and the actual execution time includes the queuing time of the tasks in the queue and the actual time spent processing them.

[0040] Calculate the priority score for each task j. ;

[0041] All tasks are prioritized based on their origin, with the Master task having the highest priority. Partner and User tasks are prioritized according to their arrival time at the edge, following a first-in, first-out (FIFO) principle. The task priority ranking is as follows:

[0042]

[0043] All Master tasks have a priority of 0 to ensure they are executed first. The normalization coefficient is... The maximum timestamp supported by the system. The timestamp of the task arriving at the edge;

[0044] The task waiting cost corresponding to the sequence is calculated using a preset cost function. ;

[0045] Cost function:

[0046]

[0047] Among them, the time penalty factor To control the non-linear effects of actual execution time; priority weighting. To adjust priority sensitivity.

[0048] Furthermore, the first model includes:

[0049] The encoding module includes a sequence transformation layer and a FeedForward layer. The sequence transformation layer is composed of a Self-Attention layer or a BiLSTM layer and several normalization layers. The sequence transformation layer uses a residual network structure. The FeedForward layer is composed of several fully connected network layers and several dropout layers. The FeedForward layer uses a residual network structure.

[0050] A feature fusion layer is used to concatenate task-level features and queue-level features and then input them into a fully connected network.

[0051] The prediction layer, which includes convolutional layers and fully connected layers, is used to output the execution time of the task queue.

[0052] Furthermore, the first model includes:

[0053] The encoding module includes a sequence transformation layer and a FeedForward layer. The sequence transformation layer is composed of a Self-Attention layer or a BiLSTM layer and several normalization layers. The sequence transformation layer uses a residual network structure. The FeedForward layer is composed of several fully connected network layers and several dropout layers. The FeedForward layer uses a residual network structure.

[0054] A feature fusion layer is used to concatenate task-level features and queue-level features and then input them into a fully connected network.

[0055] The prediction layer, which includes convolutional layers and fully connected layers, is used to output the execution time of the task queue.

[0056] Furthermore, the second model is based on the RankNet model structure.

[0057] Furthermore, after entering the second stage, the sorted sequence with the smallest predicted task execution time and the actual task execution time of the task scheduling are recorded in real time. The first model is continuously corrected based on the sorted sequence with the smallest predicted task execution time and the corresponding actual task execution time of the task scheduling.

[0058] Furthermore, upon entering the third stage, the optimal task queue is used to schedule tasks and the actual task execution time of the corresponding task is recorded in real time. The second model is then continuously revised based on the optimal task queue and the actual task execution time of the corresponding task.

[0059] The present invention also provides an apparatus for scheduling edge AI computing tasks for cloud-edge collaboration, comprising:

[0060] The acquisition module is used to continuously acquire a real-time task set, randomly generate several different task sequences based on the task set, and classify and prioritize each randomly generated task sequence according to task type and task source to obtain a sorted sequence.

[0061] The first scheduling module is used to enter the first stage. Online, it schedules tasks according to each sorted sequence and records the corresponding task execution time. Offline, it uses all sorted sequences as input and the corresponding task execution time as output to train the first model.

[0062] The second scheduling module is used to enter the second stage after the confidence of the first model reaches the first preset threshold. The trained first model is launched online. Online, the first model is used to predict each sorted sequence corresponding to the given set of tasks to be assigned, and the sorted sequence with the smallest predicted task execution time is selected for task scheduling. Offline, the second model is trained based on the given set of tasks to be assigned and the sorted sequence with the smallest predicted task execution time.

[0063] The third scheduling module is used to enter the third stage after the confidence of the second model reaches the second preset threshold. The trained second model is then launched online. The second model is used to predict the newly added set of tasks to be assigned, obtain the optimal task queue, and schedule tasks according to the optimal task queue.

[0064] Both the first model and the second model are preset machine learning prediction models.

[0065] The beneficial effects of this invention are as follows:

[0066] This invention proposes an edge AI computing task scheduling method and device for cloud-edge collaboration, which improves the actual task execution time, increases the resource utilization of edge servers, and solves the problem of huge burden and energy consumption on edge servers when users and cloud computing centers send a large number of task requests in existing edge computing. Attached Figure Description

[0067] Figure 1 This is a flowchart of the task scheduling method provided in an embodiment of the present invention;

[0068] Figure 2 This is a schematic diagram of the parameters of the training model A provided in an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram illustrating the relationship between Model A and Model B provided in an embodiment of the present invention;

[0070] Figure 4 This is a flowchart of the prediction process for Model A provided in an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0072] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0073] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0074] In one embodiment, the present invention provides an edge AI computing task scheduling method for cloud-edge collaboration, comprising:

[0075] Continuously acquire real-time task sets, randomly generate several different task sequences based on the task sets, and classify and prioritize each randomly generated task sequence according to task type and task source to obtain a sorted sequence;

[0076] In the first stage, online, tasks are scheduled and the corresponding task execution time is recorded according to each sorted sequence. Offline, all sorted sequences are used as input and the corresponding task execution time is used as output to train the first model.

[0077] Once the confidence level of the first model reaches the first preset threshold, the second stage begins. The trained first model is then deployed online. Online, the first model is used to predict each sorted sequence corresponding to the given set of tasks to be assigned, and the sorted sequence with the smallest predicted task execution time is selected for task scheduling. Offline, the second model is trained based on the given set of tasks to be assigned and the sorted sequence with the smallest predicted task execution time.

[0078] Once the confidence level of the second model reaches the second preset threshold, the third stage begins. The trained second model is then deployed online. The second model is used to predict the newly added set of tasks to be assigned, obtain the optimal task queue, and schedule tasks based on the optimal task queue.

[0079] Both the first model and the second model are preset machine learning prediction models.

[0080] In one embodiment, the task types include CPU-intensive tasks, disk I / O-intensive tasks, NPU-intensive tasks, and network I / O-intensive tasks, and the task sources include Master tasks (i.e., main tasks), Partner tasks (i.e., sub-tasks), and user tasks.

[0081] In one embodiment, classifying randomly generated task sequences based on task type includes:

[0082] Based on the characteristics of task resource consumption and dependencies, tasks are categorized into different types to obtain task resource demand rates.

[0083] Calculate the average and variance of resource demand rates for different tasks in historical data, where the historical data consists of collected task logs that have been executed in the system in the past, including:

[0084] Calculate the historical average and average variance of resource demand rates for CPU-intensive tasks:

[0085]

[0086] =

[0087] =

[0088] =

[0089] =

[0090] =

[0091] =

[0092] =

[0093] in, This represents the average CPU utilization of CPU-intensive tasks in historical data. The average variance of the CPU. This represents the average disk utilization rate for CPU-intensive tasks in historical data. The disk average variance This represents the average NPU utilization for CPU-intensive tasks in historical data. The average variance of the NPU. This represents the average bandwidth utilization of CPU-intensive tasks in historical data. The average variance of the bandwidth. , , and The following are the CPU utilization, disk utilization, NPU utilization, and bandwidth utilization of the i-th CPU-intensive task, in order. This represents the number of times CPU-intensive tasks occurred in historical data.

[0094] Similarly, calculate the average demand rate and average variance in historical data for disk I / O intensive tasks;

[0095] Similarly, calculate the average demand rate and average variance in historical data for NPU-intensive tasks;

[0096] Set the maximum values ​​for the resource demand metrics to be: CPU maximum utilization (maxc), NPU maximum utilization (maxp), disk maximum utilization (maxd), bandwidth maximum utilization (maxn), and set the variance threshold for all resource demand metrics.

[0097] If the resource demand rate does not meet the requirements, and the occupancy rate of a certain resource demand exceeds the maximum value or the threshold, the task cannot proceed to the next step and will enter a reload state.

[0098] In one embodiment, prioritizing a randomly generated task sequence based on its source includes:

[0099] For each specific task queue, which is the sequence of tasks that have entered the task scheduling and execution state, the actual execution time of each task j is recorded. M is the number of tasks, and the actual execution time includes the queuing time of the tasks in the queue and the actual time spent processing them.

[0100] Calculate the priority score for each task j. ;

[0101] All tasks are prioritized based on their origin, with the Master task having the highest priority. Partner and User tasks are prioritized according to their arrival time at the edge, following a first-in, first-out (FIFO) principle. The task priority ranking is as follows:

[0102]

[0103] All Master tasks have a priority of 0 to ensure they are executed first. The normalization coefficient is... The maximum timestamp supported by the system. The timestamp of the task arriving at the edge;

[0104] The task waiting cost corresponding to the sequence is calculated using a preset cost function. ;

[0105] Cost function:

[0106]

[0107] Among them, the time penalty factor To control the non-linear effects of actual execution time; priority weighting. To adjust priority sensitivity.

[0108] In one embodiment, the first model includes:

[0109] The encoding module includes a sequence transformation layer and a FeedForward layer (i.e., a feedforward layer). The sequence transformation layer is composed of a Self-Attention layer or a BiLSTM layer and several normalization layers. The sequence transformation layer uses a residual network structure. The FeedForward layer is composed of several fully connected network layers and several dropout layers. The FeedForward layer uses a residual network structure.

[0110] A feature fusion layer is used to concatenate task-level features and queue-level features and then input them into a fully connected network.

[0111] The prediction layer, which includes convolutional layers and fully connected layers, is used to output the execution time of the task queue.

[0112] In one embodiment, the first model includes:

[0113] The encoding module includes a sequence transformation layer and a FeedForward layer. The sequence transformation layer is composed of a Self-Attention layer (i.e., a self-attention mechanism layer) or a BiLSTM layer (i.e., a bidirectional long short-term memory network layer) combined with several normalization layers. The sequence transformation layer uses a residual network structure. The FeedForward layer is composed of several fully connected network layers and several dropout layers (dropout layers are a regularization technique used in neural networks). The FeedForward layer also uses a residual network structure.

[0114] A feature fusion layer is used to concatenate task-level features and queue-level features and then input them into a fully connected network.

[0115] The prediction layer, which includes convolutional layers and fully connected layers, is used to output the execution time of the task queue.

[0116] In one embodiment, the second model is a RankNet model structure.

[0117] In one embodiment, after entering the second stage, the sorting sequence with the smallest predicted task execution time and the actual task execution time of the task scheduling are recorded in real time, and the first model is continuously corrected based on the sorting sequence with the smallest predicted task execution time and the corresponding actual task execution time of the task scheduling.

[0118] In one embodiment, after entering the third stage, the optimal task queue for task scheduling and the actual task execution time of the corresponding task scheduling are recorded in real time, and the second model is continuously corrected based on the optimal task queue for task scheduling and the actual task execution time of the corresponding task scheduling.

[0119] In one embodiment, the present invention also provides an apparatus for scheduling edge AI computing tasks for cloud-edge collaboration, comprising:

[0120] The acquisition module is used to continuously acquire a real-time task set, randomly generate several different task sequences based on the task set, and classify and prioritize each randomly generated task sequence according to task type and task source to obtain a sorted sequence.

[0121] The first scheduling module is used to enter the first stage. Online, it schedules tasks according to each sorted sequence and records the corresponding task execution time. Offline, it uses all sorted sequences as input and the corresponding task execution time as output to train the first model.

[0122] The second scheduling module is used to enter the second stage after the confidence of the first model reaches the first preset threshold. The trained first model is launched online. Online, the first model is used to predict each sorted sequence corresponding to the given set of tasks to be assigned, and the sorted sequence with the smallest predicted task execution time is selected for task scheduling. Offline, the second model is trained based on the given set of tasks to be assigned and the sorted sequence with the smallest predicted task execution time.

[0123] The third scheduling module is used to enter the third stage after the confidence of the second model reaches the second preset threshold. The trained second model is then launched online. The second model is used to predict the newly added set of tasks to be assigned, obtain the optimal task queue, and schedule tasks according to the optimal task queue.

[0124] Both the first model and the second model are preset machine learning prediction models.

[0125] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0126] like Figure 1 As shown, the task scheduling method provided by the present invention includes the following steps:

[0127] Step S101: Obtain the task set. In a distributed task scheduling system, the task set is mainly obtained through a dynamic task selection mechanism. When a batch of tasks arrive simultaneously, the system initiates a sampling process based on a sliding time window. This mechanism maintains a fixed-length time window (e.g., 500ms), continuously captures the task set within the window, and selects M tasks to form a candidate subset according to a predetermined sampling strategy.

[0128] Step S102: Analyze the task requests. Different types of tasks have different resource requirements. Tasks can be divided into CPU-intensive tasks, disk I / O-intensive tasks, NPU-intensive tasks, and network I / O-intensive tasks.

[0129] Step S103: The task generator sorts and categorizes tasks according to task type, task source, etc.

[0130] Step S104: Construct the initial training dataset. Randomly generate a task queue, and calculate the waiting cost for each task in the queue.

[0131] Step S105: Select the optimal task queue based on the prediction of the second-order machine learning.

[0132] Step S106: The task queue selects the relevant task executor based on the type of resource being called.

[0133] Step S107: The task executor assigns tasks to different task execution engines according to their type.

[0134] Steps S101-S103, which analyze task requests and categorize and sort tasks, include:

[0135] Step 1: Divide tasks into different types based on their resource consumption and dependencies. Each type has its specific optimization strategies and application scenarios to obtain the task resource demand rate.

[0136] Step 2: Calculate the average and variance of resource demand rates for different tasks in historical data, where historical data refers to collected task logs that have been executed in the system in the past. For example:

[0137] CPU-intensive tasks primarily consume CPU resources, requiring a significant amount of computation. Because CPU-intensive tasks involve increased task switching time during multitasking, the efficiency of CPU task execution decreases.

[0138]

[0139]

[0140]

[0141]

[0142]

[0143]

[0144]

[0145]

[0146] in, This represents the average CPU utilization of CPU-intensive tasks in historical data. The average variance of the CPU. This represents the average disk utilization rate for CPU-intensive tasks in historical data. The disk average variance This represents the average NPU utilization for CPU-intensive tasks in historical data. The average variance of the NPU. This represents the average bandwidth utilization of CPU-intensive tasks in historical data. The average variance of the bandwidth. , , and The following are the CPU utilization, disk utilization, NPU utilization, and bandwidth utilization of the i-th CPU-intensive task, in order. This represents the number of times CPU-intensive tasks occurred in historical data.

[0147] Disk I / O intensive tasks involve a large number of data read and write operations. These tasks require a considerable amount of time to wait for disk read and write operations to complete. Disk I / O intensive tasks primarily rely on disk read / write speed and data storage capacity. The average and average variance of historical resource demand rates for disk I / O intensive tasks are calculated; the same applies to CPU intensive tasks.

[0148] NPU-intensive tasks typically refer to high-performance computing tasks performed using Neural Processing Units (NPUs), such as training and inference of deep learning models. NPUs improve computational efficiency through specialized hardware acceleration. The historical average and mean variance of resource requirements for NPU-intensive tasks are calculated, similar to calculations for CPU-intensive tasks.

[0149] Network I / O intensive tasks primarily involve data transmission and communication over a network. These tasks require handling network latency and bandwidth limitations to ensure efficient data transmission. Network I / O intensive tasks depend on the performance and stability of the network infrastructure. The average and mean variance of historical resource demand rates for network I / O intensive tasks are calculated; similarly, the calculations are performed for CPU-intensive tasks.

[0150] Step 3: Set the maximum values ​​of the resource demand indicators to be maximum CPU utilization (maxc), maximum NPU utilization (maxp), maximum disk utilization (maxd), maximum bandwidth utilization (maxn), and set the variance thresholds for all resource demand rates (a1: CPU threshold, a2: NPU threshold, a3: disk threshold, a4: bandwidth threshold).

[0151] Step 4: If the resource demand rate does not meet the requirements, and the occupancy rate of a certain resource demand exceeds the maximum value or is greater than the threshold, the next step of the process cannot be entered, and the task enters the reload state.

[0152] Step 5: Divide tasks into Master tasks, Partner tasks, and User tasks based on their source. Tasks issued by the Master have the highest priority, while tasks from the other two sources are prioritized according to their arrival time at the edge, following a first-in-first-out (FIFO) principle.

[0153] Step 6: Push the tasks to the machine learning model in descending order of priority, and let the model predict the best task queue.

[0154] In step S104, the task queue is randomly generated, the actual execution time is recorded, and the initial training dataset is constructed. The specific construction process is as follows.

[0155] Step 1: Randomly generate task sequences. M tasks were collected within a time window. By randomly arranging these tasks, M! different task sequences can be generated. .

[0156] Step 2: For each specific task queue, the system will process the tasks in a predetermined order and record the actual execution time of each task j. The actual execution time includes the queuing time of the task in the queue and the actual time spent processing it.

[0157] Step 3: Calculate the priority score for each task j.

[0158] Tasks originate from Master tasks, Partner tasks, and User tasks. All tasks are prioritized based on their origin, with Master tasks having the highest priority. Partner and User tasks are prioritized according to their arrival time at the edge, following a first-in, first-out (FIFO) principle. The task priority ranking is as follows:

[0159]

[0160] All Master tasks have a priority of 0, ensuring they are executed first. Normalization coefficient ( ,make sure ), The maximum timestamp supported by the system. This is the timestamp when the task arrives at the edge.

[0161] Step 4: Calculate the task waiting cost corresponding to the sequence using a preset cost function. .

[0162] Cost function:

[0163]

[0164] Among them, the time penalty factor To control the nonlinear effects of actual execution time (typically) (This reflects the increasing marginal cost of delay). Priority weight. To adjust priority sensitivity ( (At the same time, reduce the dominance of high-priority tasks).

[0165] In step S105, the optimal task queue is determined based on the prediction results of the machine learning prediction model.

[0166] In the following text, Model A refers to the first machine learning prediction model, and Model B refers to the second machine learning prediction model.

[0167] Step 1: Train model A using the initial dataset (e.g., ...) Figure 2 Model A takes a task queue as input, including the type of new task, the length of the task queue, and the edge node NPU utilization, CPU utilization, disk utilization, and bandwidth utilization. The output is the task waiting cost.

[0168] Step 2: Generate M! types of task queues based on the task set of length M.

[0169] Step 3: Use Model A to predict the task waiting cost for each type of task queue. Record the task queue with the lowest task waiting cost among the M! task queues, and use it as the label for the dataset used to train Model B.

[0170] Step 4: Use the prediction results of model A as the training data for model B (e.g., Figure 3 Model B is trained using a set of tasks (unordered), including new task types, task queue lengths, task queue types, and edge node NPU utilization, CPU utilization, disk utilization, and bandwidth utilization. The output of model B is the optimal task queue. Initially, a single-run machine learning approach was used. Figure 4 After model A has been running for a period of time, its prediction results are collected as training data for model B. Model B is then trained for a period of time. Model B is then used to predict the optimal task queue. If the confidence level of the prediction result reaches 0.9, the prediction result of model B is adopted; otherwise, the prediction method of model A is still used. At the same time, data is continuously collected to train model B until the prediction result of model B reaches maturity.

[0171] Step 5: Use Model B to predict the optimal task queue.

[0172] Step Six: Determine the confidence level of Model B's prediction. If the confidence level is greater than 0.9, use Model B's prediction; if it is less than 0.9, use Model A's prediction. Then, sort the predictions and select the best one to obtain the optimal task queue.

[0173] The specific process of training Model A in step one.

[0174] Step a: Encode the input data.

[0175] Numerical features such as CPU utilization, NPU utilization, disk utilization, bandwidth utilization, and task queue length are standardized using Z-score.

[0176] One-hot encoding is used to convert task type and task queue type into binary vectors.

[0177] Step b: The model structure design mainly includes the encoding module, feature fusion layer and prediction layer.

[0178] The encoding module includes a sequence transformation layer and a FeedForward layer. The sequence transformation layer is composed of a Self-Attention layer or a BiLSTM layer and several normalization layers, and the sequence transformation layer uses a residual network structure. The FeedForward layer is composed of several fully connected network layers and several dropout layers, and the FeedForward layer uses a residual network structure.

[0179] Feature fusion layer: The task-level features and queue-level features are concatenated and then input into the fully connected network.

[0180] Prediction layer: Convolutional layer + fully connected layer, outputs task queue execution time.

[0181] The specific process of training Model B in step four.

[0182] Step a: Encode the input data. This is similar to the data encoding process in step a during the training of model A in step one.

[0183] Step b: For model structure design, the model uses a ranking module based on the RankNet model structure.

[0184] RankNet consists of two main neural networks: one for predicting sample scores and the other for ranking based on those scores. It typically employs a dual-tower structure, effectively separating feature extraction and ranking tasks to improve efficiency. For example, it models the probability of an item's value for a given value in terms of its corresponding score. The sorting relationship, that is Ranked The probability before:

[0185]

[0186] if Compare More relevant (should be ranked higher), then there should be ,otherwise Its loss function is:

[0187]

[0188] In the loss function L, That is, cross-entropy requires normalization in probabilistic form, so 0.5 is needed as a threshold to indicate which side is more relevant.

[0189]

[0190] The above It's the model's prediction of the labels on the training data. It also needs to meet the requirement of a normalized probability distribution. In actual data, the correlation level is usually discretely labeled as... , representing: irrelevant, neutral, highly relevant, respectively. The following mapping can meet the requirements:

[0191]

[0192] In step S106, the task queue selects the relevant task executor based on the type of resource it calls. Different task executors are encapsulations of different resources; a total of three types of task executors are created here: CPU executors, NPU executors, and IO executors. CPU executors handle tasks with high CPU computational requirements, NPU executors mainly handle AI-related tasks, and IO executors mainly handle tasks related to IO operations.

[0193] In step S107, the task executor assigns tasks to different task execution engines according to their type. Different task executors have different execution engines, and different execution engines execute different tasks. For example, execution engine 1: dockerrun (creates and starts containers); execution engine 2: docker pull (pulls images); execution engine 3: crontab (a tool for scheduled tasks) and adding tasks. To better select the task execution engine, tasks are further divided into the following 11 types: 1. App deployment (download / update); 2. App configuration update; 6. App startup; 4. App stop; 5. App periodic scheduling (scheduled start and stop); 6. App uninstallation; 7. Local resource statistics reporting; 8. Local resource periodic statistics reporting; 9. Switching master (switching master nodes); 10. Adding controller (adding a controller); 11. Adding cluster_task_queue (adding a cluster task queue).

[0194] The general inventive concept of this invention is as follows:

[0195] Phase 1: Initial Stage (approximately 3 months after the scheduling system goes live)

[0196] For newly added tasks to be assigned, they are randomly sorted under priority constraints (i.e., higher priority tasks are placed first, and tasks of the same priority are randomly sorted), and the tasks are scheduled according to the random sorting results. The task vector sequence and the actual time consumption of scheduling execution are recorded.

[0197] After accumulating a certain number of records, train the first model: given a sequence of task vectors, predict the time consumption.

[0198] Continuously record subsequent task scheduling data: task vector sequence and actual time consumption, and compare the actual time consumption with the predicted time consumption to calculate the accuracy of the first model.

[0199] Once the accuracy of the first model reaches a certain threshold, the second stage begins.

[0200] Phase Two: Optimization Period (lasting approximately 3 to 6 months)

[0201] For newly added tasks to be assigned, use strategy S1 for online task scheduling. The steps of strategy S1 are as follows:

[0202] Given all possible sorting options under priority constraints, for each possible sorting option, predict its time consumption online using the first model, select the sorting result with the smallest predicted time consumption value, and then schedule the task.

[0203] Record: The given set of tasks to be assigned is sorted by the minimum predicted time consumption value given by the first model; the task vector sequence and the actual time consumption of scheduling execution.

[0204] The first model is continuously and incrementally trained using the "task vector sequence and actual time consumption of scheduling execution" to ensure its accuracy.

[0205] The second model is trained by "sorting the given set of tasks to be assigned with the smallest predicted time consumption values ​​given by the first model": given the set of tasks to be assigned, the scheduling sequence is directly predicted.

[0206] The scheduling sequence predicted by the second model is compared with the result of strategy S1, and their accuracy is calculated.

[0207] Once the accuracy of the second model reaches a certain threshold, the third stage begins.

[0208] Phase Three: Maturity

[0209] For newly added tasks to be assigned, the second model is used to predict the task order, and the tasks are scheduled and executed according to the order.

[0210] Simultaneously, record: (a) the set of tasks to be assigned; and (b) the task sequence and actual time consumption.

[0211] The first model is continuously and incrementally trained using the "task vector sequence and actual time consumption of scheduling execution" to ensure its accuracy.

[0212] For the record "(a) set of tasks to be assigned", a task sequence is generated offline using strategy S1 based on the first model. The second model is then continuously trained using the prediction results of "(a) set of tasks to be assigned" and the offline strategy S1.

[0213] Through continuous iterative optimization, the generalization ability and stability of the second model in real-world scenarios are ensured. When a new task arrives, the system prioritizes using the second model to generate a scheduling sequence, improving decision-making efficiency. Simultaneously, during actual execution, feedback data is continuously collected to verify model prediction biases and update the training set. This entire process forms a closed loop, supporting long-term dynamic evolution and ultimately achieving a highly efficient and adaptive task scheduling system.

[0214] The advantages of this invention compared to the prior art are:

[0215] Compared to traditional first-in-first-out (FIFO) task scheduling methods, this invention first categorizes tasks based on their resource consumption type: CPU-intensive tasks, NPU-intensive tasks, disk I / O-intensive tasks, and network I / O-intensive tasks. Based on task type, task origin, and associated resources, the optimal task queue is selected to improve actual task execution time and increase edge server resource utilization.

[0216] For the selection of the optimal task queue, a secondary machine learning approach is adopted. First, model A is trained using historical data from the edge computing side. Model A is then used to predict the actual execution time of tasks under different task queues. The prediction results of model A are collected to train model B, which directly predicts the optimal task queue.

[0217] Regarding the selection of model B and model A, model A is first enabled for a period of time, and then model B is enabled. The confidence level is used to determine which model's prediction result to choose. Once the confidence level of model B is greater than 0.9, the prediction result of model B is used. While the confidence level of model B's prediction result is less than 0.9, the prediction results of model A are continuously collected to train model B and continuously optimize model B.

[0218] When creating and selecting task executors, viewing them as encapsulations of different resources can effectively improve resource utilization.

[0219] The training dataset for Model A takes into account task type, current edge node utilization, current task queue type and length, and incorporates multiple factors that change in real time, thus improving the accuracy and real-time performance of predictions.

[0220] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for scheduling edge AI computing tasks for cloud-edge collaboration, characterized in that, include: Continuously acquire real-time task sets, randomly generate several different task sequences based on the task sets, classify them into different types according to the characteristics of task resource consumption and dependencies, and classify and prioritize each randomly generated task sequence based on task type and task source to obtain a sorted sequence; In the first stage, online, tasks are scheduled and the corresponding task execution time is recorded according to each sorted sequence. Offline, all sorted sequences are used as input and the corresponding task execution time is used as output to train the first model. Once the confidence level of the first model reaches the first preset threshold, the second stage begins. The trained first model is then deployed online. Online, the first model is used to predict each sorted sequence corresponding to the given set of tasks to be assigned, and the sorted sequence with the smallest predicted task execution time is selected for task scheduling. Offline, the second model is trained based on the given set of tasks to be assigned and the sorted sequence with the smallest predicted task execution time. Once the confidence level of the second model reaches the second preset threshold, the third stage begins. The trained second model is then deployed online. The second model is used to predict the newly added set of tasks to be assigned, obtain the optimal task queue, and schedule tasks based on the optimal task queue. Both the first model and the second model are preset machine learning prediction models.

2. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 1, characterized in that, The task types include CPU-intensive tasks, disk I / O-intensive tasks, NPU-intensive tasks, and network I / O-intensive tasks, and the task sources include Master tasks, Partner tasks, and user tasks.

3. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 2, characterized in that, Randomly generated task sequences are classified based on task type, including: Acquire task resource demand rate; Calculate the average and variance of resource demand rates for different tasks in historical data, where the historical data consists of collected task logs that have been executed in the system in the past, including: Calculate the historical average and average variance of resource demand rates for CPU-intensive tasks: ， = ， = ， = ， = ， = ， = ， = ， in, This represents the average CPU utilization of CPU-intensive tasks in historical data. The average variance of the CPU. This represents the average disk utilization rate for CPU-intensive tasks in historical data. The disk average variance This represents the average NPU utilization for CPU-intensive tasks in historical data. The average variance of the NPU. This represents the average bandwidth utilization of CPU-intensive tasks in historical data. The average variance of the bandwidth. , , and The CPU utilization, disk utilization, NPU utilization, and bandwidth utilization of the i-th CPU-intensive task are respectively, and N is the number of times the CPU-intensive task appears in the historical data. Similarly, calculate the average demand rate and average variance in historical data for disk I / O intensive tasks; Similarly, calculate the average demand rate and average variance in historical data for NPU-intensive tasks; Set the maximum values ​​for the resource demand metrics to be: CPU maximum utilization (maxc), NPU maximum utilization (maxp), disk maximum utilization (maxd), bandwidth maximum utilization (maxn), and set the variance threshold for all resource demand metrics. If the resource demand rate does not meet the requirements, and the occupancy rate of a certain resource demand exceeds the maximum value or the threshold, the task cannot proceed to the next step and will enter a reload state.

4. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 2, characterized in that, Prioritize randomly generated task sequences based on their source, including: For each specific task queue, record the actual execution time of each task j. M is the number of tasks. The actual execution time includes the queuing time of the tasks in the queue and the actual time spent being processed. A specific task queue is the sequence of tasks that enters the task scheduling and execution state. Calculate the priority score for each task j. ; All tasks are prioritized based on their origin, with the Master task having the highest priority. Partner and User tasks are prioritized according to their arrival time at the edge, following a first-in, first-out (FIFO) principle. The task priority ranking is as follows: ， All Master tasks have a priority of 0 to ensure they are executed first. The normalization coefficient is... The maximum timestamp supported by the system. The timestamp of the task reaching the edge; The task waiting cost corresponding to the sequence is calculated using a preset cost function. ; Cost function: ， Among them, the time penalty factor To control the non-linear effects of actual execution time; priority weighting. To adjust priority sensitivity.

5. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 1, characterized in that, The first model includes: The encoding module includes a sequence transformation layer and a FeedForward layer. The sequence transformation layer is composed of a Self-Attention layer or a BiLSTM layer and several normalization layers. The sequence transformation layer uses a residual network structure. The FeedForward layer is composed of several fully connected network layers and several dropout layers. The FeedForward layer uses a residual network structure. A feature fusion layer is used to concatenate task-level features and queue-level features and then input them into a fully connected network. The prediction layer, which includes convolutional layers and fully connected layers, is used to output the execution time of the task queue.

6. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 1, characterized in that, The second model is the RankNet model structure.

7. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 1, characterized in that, After entering the second stage, the sorted sequence with the smallest predicted task execution time and the actual task execution time of the task scheduling are recorded in real time. The first model is continuously corrected based on the sorted sequence with the smallest predicted task execution time and the corresponding actual task execution time of the task scheduling.

8. The edge AI computing task scheduling method for cloud-edge collaboration according to claim 1, characterized in that, After entering the third stage, the optimal task queue is recorded in real time for task scheduling and the actual task execution time of the corresponding task scheduling. The second model is continuously corrected based on the optimal task queue for task scheduling and the actual task execution time of the corresponding task scheduling.

9. An apparatus for scheduling edge AI computing tasks in a cloud-edge collaborative manner as described in any one of claims 1-8, characterized in that, include: The acquisition module is used to continuously acquire a real-time task set, randomly generate several different task sequences based on the task set, and classify and prioritize each randomly generated task sequence according to task type and task source to obtain a sorted sequence. The first scheduling module is used to enter the first stage. Online, it schedules tasks according to each sorted sequence and records the corresponding task execution time. Offline, it uses all sorted sequences as input and the corresponding task execution time as output to train the first model. The second scheduling module is used to enter the second stage after the confidence of the first model reaches the first preset threshold. The trained first model is launched online. Online, the first model is used to predict each sorted sequence corresponding to the given set of tasks to be assigned, and the sorted sequence with the smallest predicted task execution time is selected for task scheduling. Offline, the second model is trained based on the given set of tasks to be assigned and the sorted sequence with the smallest predicted task execution time. The third scheduling module is used to enter the third stage after the confidence of the second model reaches the second preset threshold. The trained second model is then launched online. The second model is used to predict the newly added set of tasks to be assigned, obtain the optimal task queue, and schedule tasks according to the optimal task queue. Both the first model and the second model are preset machine learning prediction models.

Images