Resource scheduling method, program product, electronic equipment and storage medium

Abstract

The invention discloses a resource scheduling method, a program product, electronic equipment and a storage medium. The resource scheduling method comprises the following steps: inputting task data of a task to be executed and system state data into a first model to obtain a task priority and a resource allocation strategy output by the first model; the first model is obtained through training based on historical task data and historical system state data, and a label of training data of the first model is an index reflecting task scheduling quality; sorting a task list based on the task priorities, wherein the task list comprises a plurality of to-be-executed tasks; and according to the task list and the resource allocation strategy of each to-be-executed task, sequentially executing the plurality of to-be-executed tasks.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a resource scheduling method, program product, electronic device and storage medium. Background Technology

[0002] In related technologies, cloud computing resource scheduling methods include static scheduling and dynamic scheduling. Static scheduling methods determine the scheduling strategy before the task starts, making it difficult to cope with load changes during task execution. While dynamic scheduling methods can adjust resource allocation according to real-time conditions, their scheduling strategies often rely on predefined rules or simple feedback mechanisms, making it difficult to achieve globally optimal scheduling in complex and ever-changing environments. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a resource scheduling method, a program product, an electronic device, and a storage medium.

[0004] The technical solution of this invention is implemented as follows: On one hand, embodiments of the present invention provide a resource scheduling method, the method comprising: The task data and system status data of the task to be executed are input into the first model to obtain the task priority and resource allocation strategy output by the first model; the first model is trained based on historical task data and historical system status data, and the labels of the training data of the first model are indicators reflecting the quality of task scheduling; The task list is sorted based on the task priority, and the task list includes multiple tasks to be executed; Multiple tasks are executed sequentially according to the task list and the resource allocation strategy for each task.

[0005] In the above scheme, the resource allocation strategy includes the task waiting time of the task to be executed, which refers to the waiting time of the task to be executed in the system from submission to start execution.

[0006] In the above scheme, the first model is used to select a target waiting time from a preset waiting time set based on the probability vector obtained through training.

[0007] In the above scheme, during the training of the first model, the method further includes: Based on the probability vector, a waiting time is selected from a preset set of waiting times, and the loss function is determined; The probability vector is updated based on the loss function, so that the probability vector converges to the optimal waiting time.

[0008] In the above scheme, the first model adopts a hybrid model structure based on convolutional neural network (CNN) and long short-term memory network (LSTM).

[0009] In the above scheme, the indicators reflecting the quality of task scheduling include one or more of the following: actual task waiting time and system resource idle time.

[0010] In the above scheme, the method further includes: Obtain feedback data after executing each task to be executed, the feedback data including task execution data and system status data during task execution; The parameters of the first model are adjusted based on the feedback data.

[0011] On the other hand, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described resource scheduling method.

[0012] On the other hand, embodiments of the present invention provide an electronic device including a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the steps of the resource scheduling method provided in the first aspect of the present invention.

[0013] On the other hand, embodiments of the present invention provide a computer-readable storage medium, comprising: the computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the resource scheduling method provided in the first aspect of the present invention.

[0014] The solution of this embodiment of the invention involves inputting task data and system status data of the tasks to be executed into a first model to obtain the task priority and resource allocation strategy output by the first model. The first model is trained based on historical task data and historical system status data, and the labels of the training data of the first model are indicators reflecting the quality of task scheduling. A task list is sorted based on task priority, and the task list includes multiple tasks to be executed. Multiple tasks to be executed are executed sequentially according to the task list and the resource allocation strategy of each task to be executed. This embodiment achieves more refined and intelligent resource scheduling by dynamically adjusting task priority and resource allocation strategy according to system status and task characteristics through the first model, thereby improving resource utilization and task scheduling efficiency. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating the implementation process of a resource scheduling method provided in an embodiment of the present invention; Figure 2This is a schematic diagram of the workflow of an ASA algorithm provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a CNN network structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an LSTM network structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a hybrid model based on CNN and LSTM provided in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the workflow of a resource scheduling method based on an adaptive neural network provided in an embodiment of the present invention. Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] With the rapid development and widespread application of cloud computing technology, the efficient scheduling and management of cloud computing resources has become a critical issue. Traditional resource scheduling methods often struggle to adapt to dynamically changing loads and diverse application requirements when dealing with complex and ever-changing cloud computing environments. Therefore, there is an urgent need for a method that can dynamically adjust and optimize resource scheduling strategies to improve resource utilization and system performance.

[0018] Resource scheduling methods in related technologies include static scheduling and dynamic scheduling. Static scheduling methods determine the scheduling strategy before the task starts, making it difficult to cope with load changes during task execution. While dynamic scheduling methods can adjust resource allocation according to real-time conditions, their scheduling strategies often rely on predefined rules or simple feedback mechanisms, making it difficult to achieve globally optimal scheduling in complex and ever-changing environments.

[0019] An adaptive scheduling algorithm (ASA) is an algorithm that dynamically adjusts the allocation of tasks or resources based on changes in system state and environment. Below are some common adaptive scheduling algorithms that are widely used in various fields, including cloud computing, data centers, and embedded systems: 1. Dynamic Priority Scheduling (DPS): This algorithm dynamically adjusts task priorities based on their importance and urgency. During system operation, task priorities can be adjusted according to predefined rules or real-time conditions. For example, the priority of some critical tasks may increase when system load increases, while the priority of non-critical tasks may decrease.

[0020] 2. Adaptive Feedback Scheduling (AFS): This algorithm continuously adjusts the task scheduling strategy through a feedback mechanism. Based on task execution results and system performance feedback, it adjusts task allocation and priority in real time to optimize overall system performance.

[0021] 3. Adaptive Genetic Algorithm (AGA): Based on the fundamental principles of genetic algorithms, the AGA optimizes task scheduling and resource allocation by adaptively adjusting the parameters of the genetic algorithm (such as crossover rate and mutation rate). This algorithm progressively optimizes the scheduling strategy through selection, crossover, and mutation operations.

[0022] 4. Adaptive Particle Swarm Optimization (APSO): APSO simulates the social behavior of a particle swarm, dynamically adjusting the velocity and position of each particle to find the optimal resource scheduling strategy. The adaptive mechanism can dynamically adjust the search range and velocity of particles based on system performance.

[0023] 5. Adaptive Fuzzy Logic Scheduling (AFLS): AFLS uses a fuzzy logic system to fuzzify system states and task characteristics, dynamically adjusting task priorities and resource allocation strategies. The adaptive mechanism adjusts fuzzy rules and parameters based on system feedback.

[0024] 6. Adaptive Neural Network Scheduling (ANNS): This algorithm utilizes the learning capabilities of neural networks to dynamically adjust task priorities and resource allocation strategies based on system state and task characteristics. Through an adaptive learning mechanism, it continuously optimizes the scheduling strategy.

[0025] A drawback of dynamic priority scheduling algorithms is the priority inversion problem. Dynamic priority scheduling algorithms are prone to priority inversion, meaning that high-priority tasks may wait for low-priority tasks to complete for extended periods, impacting overall system performance. The adjustment mechanism of dynamic priority scheduling algorithms is complex to implement, requiring the design and implementation of sophisticated priority adjustment mechanisms, especially in multi-task, multi-priority scenarios. When handling sudden loads, priority adjustment may lag, failing to respond quickly to changes in system state.

[0026] Adaptive feedback scheduling algorithms rely on complex feedback mechanism designs, which can result in slow response times, especially when handling sudden loads, where feedback adjustment strategies may lag. Under high load or complex environments, the feedback mechanism may fail to adjust strategies in a timely and effective manner, impacting system stability.

[0027] Adaptive genetic algorithms and adaptive particle swarm optimization algorithms have high computational complexity and slow convergence speed, which increases the difficulty of implementation and maintenance and affects real-time scheduling performance.

[0028] Adaptive fuzzy logic scheduling algorithms require detailed definitions of fuzzy rules and parameters, resulting in high design and implementation complexity. Similarly, fuzzy logic systems incur significant computational overhead, especially in scenarios with high real-time requirements. For example, fuzzy logic systems may suffer from insufficient real-time performance when handling dynamically changing system states. There are many types of adaptive scheduling algorithms, each with its own advantages and disadvantages. Choosing the right adaptive scheduling algorithm requires a trade-off based on the specific application scenario and requirements. In dynamic and changing system environments, adaptive scheduling algorithms can provide more flexible and efficient resource scheduling solutions, improving system performance and stability. Adaptive Neural Networks (ANKs) are neural networks that can automatically adjust their structure and parameters. Traditional neural networks typically require manual configuration of the network structure and parameters before training, while adaptive neural networks can automatically select and adjust their structure and parameters through learning and optimization algorithms to adapt to different environments and task requirements. Adaptive neural networks can dynamically add, delete, or modify neurons, connection weights, and the number of layers based on changes in input data and loss functions, thereby improving the network's adaptability and performance.

[0029] To address the shortcomings of the aforementioned related technologies, this invention provides a resource scheduling method. Specific embodiments are described below to illustrate the technical solution described in this invention.

[0030] Figure 1 This is a schematic diagram illustrating the implementation flow of a resource scheduling method provided in an embodiment of the present invention. The executing entity of the resource scheduling method is an electronic device. (Reference) Figure 1 Resource scheduling methods include: S101, Input the task data and system status data of the task to be executed into the first model to obtain the task priority and resource allocation strategy output by the first model; The first model is trained based on historical task data and historical system status data, and the labels of the training data of the first model are indicators reflecting the quality of task scheduling.

[0031] In a cloud computing resource management system, after a task is submitted to the system, cloud computing resources are allocated to execute the task. Task data for tasks to be executed may include task execution time, task priority, and task dependencies. Tasks have an initial priority upon submission, such as user-specified priority or rule-based calculation. System status data may include the utilization rates of the cloud computing system's Central Processing Unit (CPU), Graphics Processing Unit (GPU), memory utilization, and disk utilization.

[0032] The system can monitor system status and task characteristics in real time during system operation, collecting real-time data such as CPU load and memory usage using system monitoring tools and log management systems. This data is transmitted to the first model, supporting real-time task priority calculation and adjustment.

[0033] The first model can be a neural network model. System status and task data are collected beforehand from monitoring and log systems. The collected data is cleaned, standardized, and normalized for use in subsequent neural network model training. During the data collection phase, the real-time nature and accuracy of the data must be ensured to support subsequent decision-making. The preprocessed training dataset is used to train the first model, and model parameters are optimized through supervised learning methods. Model training is conducted on a dedicated model training server or cloud platform, and is regularly updated to adapt to new workloads and system environment changes. During neural network model training, it is necessary to ensure the representativeness and diversity of the training data to improve the model's generalization ability and accuracy. Simultaneously, the training error and validation error of the model need to be monitored during training to prevent overfitting or underfitting.

[0034] When training the first model, the labels of the training data are indicators that reflect the quality of task scheduling, such as the actual waiting time of a task in the task queue, the idle time of system resources, etc.

[0035] The input to the first model is task data and system state data. The output of the first model is task priority and resource scheduling strategy. Task priority can represent the execution order of tasks in the task queue, or the priority when allocating resources. Tasks with higher priority will receive computing resources and other critical resources first. The resource scheduling strategy can include the waiting time of the tasks to be executed in the system from submission to start execution.

[0036] S102, sort the task list based on the task priority, the task list including multiple tasks to be executed.

[0037] In the cloud resource management system, a task list is set up, and the tasks in the task list are sorted according to their priority. Based on the task priority output by the first model, the position of the task to be executed in the task list is determined.

[0038] Here, the task priorities of multiple tasks to be executed can be calculated simultaneously. For example, when the system load changes, the task priorities of all tasks in the task list are recalculated, and the task list is reordered. Dynamically adjusting task priorities ensures that high-priority tasks receive computing resources and other critical resources first.

[0039] S103, execute multiple tasks in sequence according to the task list and the resource allocation strategy of each task to be executed.

[0040] Based on the task list, resources are allocated to tasks with high priority in the task list. Combined with the resource allocation strategy, appropriate resources (such as compute nodes, storage resources and network bandwidth) are allocated to the tasks to be executed. The goal is to maximize system efficiency and task execution performance, and ensure that tasks are completed on time and meet the Service Level Agreement (SLA).

[0041] The solution of this embodiment of the invention involves inputting task data and system status data of the tasks to be executed into a first model to obtain the task priority and resource allocation strategy output by the first model. The first model is trained based on historical task data and historical system status data, and the labels of the training data of the first model are indicators reflecting the quality of task scheduling. A task list is sorted based on task priority, and the task list includes multiple tasks to be executed. Multiple tasks to be executed are executed sequentially according to the task list and the resource allocation strategy of each task to be executed. This embodiment achieves more refined and intelligent resource scheduling by dynamically adjusting task priority and resource allocation strategy according to system status and task characteristics through the first model, thereby improving resource utilization and task scheduling efficiency.

[0042] In one embodiment, the resource allocation strategy includes the task waiting time of the task to be executed, which refers to the waiting time of the task to be executed in the system from submission to start execution.

[0043] The first model in this embodiment can also output the task waiting time. The first model can use an adaptive scheduling algorithm to proactively submit resource change requests in advance by estimating the task waiting time, thereby minimizing the waiting time between task job stages.

[0044] ASA learns the distribution of queue wait times and makes dynamic adjustments based on real-time feedback to maximize resource utilization and minimize wait times between jobs.

[0045] The first model is used to select a target waiting time from a preset waiting time set based on the probability vector obtained during training.

[0046] The ASA algorithm works by maintaining a fixed distribution of waiting times for a set of queues. For example, for m=4, ASA attempts to learn which of the four options (indices) in the vector (1s, 10s, 100s, 1000s) is best suited as an estimate of the queue waiting time for a given resource allocation request.

[0047] Instead of focusing on a specific time (such as the average queue wait time per sense), ASA assigns options based on a probability vector. The first model selects a target wait time from a pre-defined set of wait times based on the probability vector. The probability vector is updated during the training of the first model. First, m wait time tiers are defined, and then the probability vector is used to "select the index of 'a'". After execution, the probability vector is updated again based on the loss score feedback, so that the probability of the optimal 'a' eventually converges to 1.

[0048] In one embodiment, during the training of the first model, the method further includes: Based on the probability vector, a waiting time is selected from a preset set of waiting times, and the loss function is determined; The probability vector is updated based on the loss function, so that the probability vector converges to the optimal waiting time.

[0049] ASA selects one of multiple discrete waiting time tiers (preset waiting time set) as the "timing estimation strategy a for advance resource application" through a probability vector. After execution, the loss is calculated based on the actual scheduling waiting time and resource idle time. The probability is then updated with a gradually decreasing learning rate, so that the better strategy a is more likely to be selected in the future, and eventually converges to the optimal strategy.

[0050] For example, for a value of m = 4, where m represents the number of available waiting time estimation strategies, and the preset waiting time set is (1s, 10s, 100s, 1000s), ASA uses the probability vector... Assignment options, among which .

[0051] In other words, if a particular option (e.g., the 1s index or m1, where m1 refers to the index of the first policy, i.e., 1s) works well (as evaluated by the loss function), then as the number of trials t approaches infinity (t→∞), we hope that... probability tends to ( ;exist The first unit vector in the middle.

[0052] The overall process of the ASA algorithm is as follows: Figure 2 As shown, in the first phase of task 1, task 2 is submitted ahead of schedule, and in the second phase of task 2, task 3 is submitted ahead of schedule. The advance time is calculated by the ASA algorithm.

[0053] For example, for each stage of iteration t, estimate a waiting time a for task y, and use it in... Submit assignment on time That is, submitting a resource change request, where It is an ongoing task y The expected end date of Y. If everything goes according to plan, this proactive job submission strategy is expected to minimize perceived latency between all workflow stages. However, if a workflow stage ends late, or is assigned earlier than expected, it may remain idle for a period before resources are effectively utilized. Conversely, if a workflow stage ends early, or if resources become available later than expected, the entire workflow may take longer to complete. This can all be addressed using a loss function associated with workflow stage Y. Let represent that the loss function is based on the waiting time 'a'.

[0054] For example, there are m different time estimates for queuing time. It is also assumed that after applying action a to a specific situation, its loss can be calculated. Scoring. The Adaptive Scheduling Algorithm (ASA) mainly consists of a double loop. The outer loop processes cases through mini-batches, while the inner loop ensures that the loss vector is updated appropriately within each mini-batch. and probability vector The algorithm uses a non-increasing sequence. To ensure convergence, and to ensure that the probability vector is sufficiently learned, Able to converge to the optimal waiting time .

[0055] In one embodiment, a monitoring and logging system is used by an agent to collect system status and task characteristic data in real time, ensuring the real-time nature and accuracy of the data. For example, data such as CPU load, memory usage, task execution time, and task priority are collected. The collected data is then cleaned, standardized, and normalized. Data cleaning removes outliers and noise (such as extreme values ​​and missing data), and interpolation is used to fill in missing values. Standardization ensures the data has a uniform proportion, and normalization maps the data to a predefined range for the training and prediction of the first model.

[0056] In one embodiment, the first model employs a hybrid model structure based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM).

[0057] This embodiment selects a hybrid CNN and LSTM model as the neural network structure, which can effectively handle spatial features and time series data. Gradient descent is used to optimize the model parameters during the model training phase. Gradient descent calculates the loss function... Regarding model parameters gradient The parameters are updated in the opposite direction of the gradient to minimize the loss function. Specifically, the parameter update rule can be expressed as: ,in, The learning rate controls the step size for each parameter update. The goal of gradient descent is to update the parameters through iterative iterations. This makes the loss function It reaches the minimum value.

[0058] To further improve training efficiency and convergence speed, this embodiment can introduce the Adam optimizer. The Adam optimizer incorporates first-moment estimation of the gradient. (i.e., the moving average of the gradient) and second-moment estimation (i.e., the moving average of the squared gradient), by adaptively adjusting the learning rate for each parameter. This allows for more precise control over the parameter update step size. The update rule of the Adam optimizer can be expressed as:

[0059]

[0060]

[0061]

[0062]

[0063] in, and It is the exponential decay rate (usually set to a value close to 1). It is a small constant (usually 1000). This is used to avoid division by zero errors. Initially, the learning rate... The settings are configured according to preset values. During training, a learning rate decay strategy is employed to gradually reduce the learning rate. This ensures the model optimizes more stably as it approaches convergence. Simultaneously, the learning rate is adjusted in real-time based on changes in the validation set loss. For example, when the validation set loss stops improving or starts to fluctuate, the learning rate can be appropriately reduced to avoid overfitting or training instability.

[0064] These optimizations not only help the model adapt to new data patterns and task requirements, but also ensure that the model can accurately predict real-time task priorities and reorder the task list, thereby ensuring that high-priority tasks can obtain computing resources first and maximizing the efficiency and performance of the system.

[0065] A convolutional neural network (CNN) is a deep neural network that can learn numerous features simultaneously. A CNN can consist of one or more convolutional layers, pooling layers, and fully connected layers. Generally, one-dimensional CNNs are sufficient for processing time-series data and natural language tasks, while two-dimensional CNNs are often used in image processing to effectively extract image features. This embodiment selects a one-dimensional convolutional neural network, the structure of which is as follows: Figure 3 As shown, it includes convolutional layers, pooling layers, and fully connected layers. Here, `sample` refers to the number of input data samples, and `step` refers to the data dimension. Since CNNs excel at extracting local features, they can perform preliminary processing on the input data, reducing redundant information. Simultaneously, the special convolutional layers in convolutional neural networks can mine deeper feature information from historical data. The convolutional and pooling layers progressively reduce the data size, extracting high-level features. Convolution formula:

[0066] in, Indicates input data, The symbol represents the convolution kernel, and * represents the convolution operation.

[0067] LSTM excels at handling time-series data, capturing long-term dependencies in task execution. Stacking two LSTM layers enhances the model's expressive power. Compared to traditional recurrent neural networks, its internal "gate" structure avoids the vanishing or exploding gradient phenomena that occur when learning medium- to long-term time series data. A memory unit in an LSTM network consists of four key elements: a forget gate, an input gate, an output gate, and the memory unit itself. Its network structure is as follows: Figure 4 As shown, the calculation formulas for each key element in the learning process are as follows:

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] in , , , These represent the forget gate, input gate, memory unit, and output gate, respectively.

[0074] Hybrid model structure based on CNN and LSTM, such as Figure 5 As shown, it includes convolutional layers, pooling layers, flattening layers, and LSTM layers. The main function of the flattening layer is to transform the data dimension, solving the compatibility problem between the high-dimensional features output by the convolutional layers and the one-dimensional input required by the fully connected layers.

[0075] During the model inference phase, the preprocessed input data is used to extract local features using a CNN layer, reducing redundant information. The output data from the CNN layer is then passed through pooling and flattening layers before being fed into an LSTM layer. The LSTM layer captures long-term dependencies in the data, enhancing its ability to remember historical information. Finally, the data processed by the LSTM layer is fed into the output layer to predict task priority and / or resource allocation decisions (task waiting time).

[0076] In one embodiment, the indicators reflecting the quality of task scheduling include one or more of the following: actual task waiting time and system resource idle time.

[0077] The labels of the training data for the first model are indicators that reflect the quality of task scheduling. When collecting training data, the actual waiting time and resource usage can be recorded after historical tasks are completed.

[0078] It should be understood that the system state data in the training data includes the system state before the task is executed, the system state during the task execution, and the system state after the task is executed.

[0079] In one embodiment, the method further includes: Obtain feedback data after executing each task to be executed, the feedback data including task execution data and system status data during task execution; The parameters of the first model are adjusted based on the feedback data.

[0080] The Adaptive Scheduling Algorithm (ASA) records the actual waiting time and resource usage after each task is completed (including statistics on system resource load and resource contention during task execution, as well as the actual resource consumption of the task itself). This data is fed back into the model training process. The first model uses this feedback data to adjust parameters, improving the accuracy of priority predictions and resource allocation decisions for future tasks. For example, if a queue waiting time estimate shows higher accuracy in historical data, ASA will increase the probability of selecting that estimate. Accuracy refers to the degree of matching between the waiting time estimation strategy and the actual queue latency and resource integration effect; the metric is the combined effect of prediction error, resource idleness, task blocking, and SLA satisfaction.

[0081] This approach ensures that ASA not only converges to a superior resource allocation strategy initially, but also adapts quickly to changes in environmental conditions or workload, continuing to optimize resource utilization. This adaptive scheduling algorithm, through continuous learning and adjustment, enables the system to effectively optimize resource allocation under varying workloads and resource changes, minimizing task execution wait times and thus improving overall system efficiency and performance.

[0082] Figure 6 This is a flowchart illustrating the workflow of a resource scheduling method based on an adaptive neural network provided in an embodiment of the present invention, including: 1. Data collection and preprocessing.

[0083] In a cloud computing resource management system, the data collection unit is responsible for gathering system status and task characteristic data from monitoring and log systems. This data includes CPU load, memory usage, task execution time, task priority, and task dependencies. The collected data undergoes cleaning, standardization, and normalization for use in subsequent neural network model training. Ensuring the real-time nature and accuracy of the data during the data collection phase is crucial to supporting subsequent decision-making processes.

[0084] 2. Training the neural network model.

[0085] The neural network model (the first model) selected a hybrid model of CNN and LSTM as the neural network structure.

[0086] Model training is performed using a preprocessed dataset, including historical task scheduling data and system performance data. Supervised learning methods are used to optimize model parameters. Model training is conducted on a dedicated model training server or cloud platform and is regularly updated to adapt to new workloads and system environment changes. During neural network model training, it is crucial to ensure the representativeness and diversity of the training data to improve the model's generalization ability and accuracy. Simultaneously, training and validation errors must be monitored during training to prevent overfitting or underfitting.

[0087] 3. Real-time monitoring and data input.

[0088] The real-time monitoring unit monitors system status and task characteristics in real time during system operation. It collects real-time data, such as real-time CPU load and memory usage, using system monitoring tools and a log management system. This data is then transmitted to a neural network model to support real-time task priority calculation and adjustment.

[0089] 4. Priority Calculation and Adjustment The priority calculation module reorders the task list based on the task priority information output by the neural network model. When the system load changes, it dynamically adjusts the task priorities to ensure that high-priority tasks receive computing resources and other critical resources first.

[0090] 5. Task scheduling and resource allocation.

[0091] Based on the prioritized task list, allocate appropriate compute nodes, storage resources, and network bandwidth. The goal is to maximize system efficiency and task execution performance, ensuring tasks are completed on time and meet SLAs.

[0092] It should be understood that the model output layer is responsible for decision suggestions (such as task waiting time), and the task scheduling model and resource allocation execute scheduling strategies based on these suggestions.

[0093] Task waiting time is merely a scheduling decision variable within the ASA algorithm, designed to reduce queuing between stages. The model output layer provides task priority predictions or resource allocation bias decisions. The task scheduling and resource allocation steps execute scheduling based on these criteria and allocate specific computing nodes, storage resources, and network bandwidth.

[0094] 6. System Feedback and Adaptive Learning The system collects task execution results and system performance feedback. This feedback is used not only to adjust the parameters of the neural network model in real time to optimize subsequent scheduling strategies and resource allocation, but also to support the implementation of Adaptive Scheduling (ASA) algorithms. ASA learns the distribution of queue waiting times and dynamically adjusts based on real-time feedback to maximize resource utilization and minimize waiting times between jobs. This comprehensive feedback and learning mechanism ensures that the system can continuously optimize resource scheduling efficiency in the face of constantly changing workloads and environmental conditions.

[0095] This embodiment achieves efficient and intelligent cloud computing resource management by integrating key steps such as data collection, model training, real-time monitoring, priority calculation, task scheduling and resource allocation, as well as system feedback and adaptive learning.

[0096] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0097] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0098] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0099] In addition, in the embodiments of the present invention, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0100] In practical applications, the method steps of the above embodiments can be implemented by processors in electronic devices, such as central processing units (CPUs), digital signal processors (DSPs), microcontroller units (MCUs), or field-programmable gate arrays (FPGAs).

[0101] To implement the methods of the embodiments of this application, an electronic device is also provided. Figure 7 This is a schematic diagram of the hardware structure of the electronic device according to an embodiment of this application, as shown below. Figure 7 As shown, the electronic device includes: A communication interface enables information exchange with other devices, such as network devices. The processor, connected to the communication interface, enables information interaction with other devices and, when running a computer program, executes the methods provided by one or more technical solutions on the electronic device side. The computer program is stored in memory.

[0102] Of course, in practical applications, the various components in an electronic device are coupled together through a bus system. It can be understood that the bus system is used to achieve communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 7 The general will label all buses as bus systems.

[0103] The memory in this application embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0104] It is understood that memory can be volatile or non-volatile, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0105] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads the program from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0106] Optionally, when the processor executes the program, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0107] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor of an electronic device to perform the steps described in the method of this application embodiment.

[0108] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory storing a computer program, which can be executed by a processor of an electronic device to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0110] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0112] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0113] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0114] It should be noted that the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0115] In addition, in this application example, terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A resource scheduling method, characterized in that, The method includes: The task data and system status data of the task to be executed are input into the first model to obtain the task priority and resource allocation strategy output by the first model; the first model is trained based on historical task data and historical system status data, and the labels of the training data of the first model are indicators reflecting the quality of task scheduling; The task list is sorted based on the task priority, and the task list includes multiple tasks to be executed; Multiple tasks are executed sequentially according to the task list and the resource allocation strategy for each task.

2. The method according to claim 1, characterized in that, The resource allocation strategy includes the task waiting time for the task to be executed, which refers to the waiting time for the task to be executed in the system from submission to start execution.

3. The method according to claim 2, characterized in that, The first model is used to select a target waiting time from a preset waiting time set based on the probability vector obtained during training.

4. The method according to claim 1, characterized in that, During the training of the first model, the method further includes: Based on the probability vector, a waiting time is selected from a preset set of waiting times, and the loss function is determined; The probability vector is updated based on the loss function, so that the probability vector converges to the optimal waiting time.

5. The method according to claim 1, characterized in that, The first model adopts a hybrid model structure based on convolutional neural network (CNN) and long short-term memory network (LSTM).

6. The method according to claim 1, characterized in that, The indicators reflecting the quality of task scheduling include one or more of the following: actual task waiting time and system resource idle time.

7. The method according to claim 1, characterized in that, The method further includes: Obtain feedback data after executing each task to be executed, the feedback data including task execution data and system status data during task execution; The parameters of the first model are adjusted based on the feedback data.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the resource scheduling method as described in any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the resource scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the steps of the resource scheduling method as described in any one of claims 1 to 7.

Images