Operating system adaptive task scheduling method and device, computer equipment

Abstract

This invention provides an adaptive task scheduling method and apparatus for an operating system, as well as a computer device, belonging to the field of computer technology. The method includes: pre-training a task scheduling model embedded in the operating system during the operating system's factory setup phase to obtain parameters for the task scheduling model, including: electricity cost incentive, resource utilization incentive, service quality incentive, and incentive weight parameters; distributing the pre-trained task scheduling model parameters to the user-side operating system, optimizing the parameters of the task scheduling model in the user-side operating system to obtain a task scheduling model matching the current user's preferences; and in the user-side operating system, performing task scheduling based on the task scheduling mode selected by the current user, using the task scheduling model matching the current user's preferences. This invention embeds a deep reinforcement learning task scheduling model into the operating system kernel to achieve closed-loop optimization of dynamic resource allocation.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to an operating system adaptive task scheduling method and apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Existing operating system task scheduling methods mainly include various classic algorithms, aiming to balance goals such as resource utilization, response time, and fairness. First-Come, First-Served (FCFS) is the most basic scheduling strategy, allocating resources according to the order in which tasks arrive. While simple to implement, it can lead to short tasks waiting too long due to being blocked by longer tasks. Shortest Job First (SJF) optimizes average turnaround time by prioritizing the execution of tasks with the shortest estimated runtime, but requires predicting task durations and is prone to causing long tasks to "starve." For interactive systems, round-robin scheduling allocates a fixed time slice to each task to achieve fairness, but frequent context switching can reduce throughput. Priority scheduling divides task execution order through static or dynamic priorities, allowing high-priority tasks to preempt resources, making it suitable for real-time systems, but it needs to address priority inversion and "starvation" issues. Furthermore, multi-level feedback queues, combining priority and time-slice mechanisms, dynamically adjust the task queue hierarchy to balance response speed and throughput, and are a complex scheduling scheme widely used in general-purpose systems.

[0003] In the operating system of power grids, traditional scheduling algorithms use CPU utilization or deadlines as core indicators, failing to incorporate dynamic energy costs such as time-of-use pricing into the decision-making system. This leads to unnecessary electricity expenditures when high-load tasks are executed during peak electricity price periods, and lacks the ability to collaboratively optimize grid load balancing. Furthermore, existing methods employ static or single-dimensional priority strategies, making it difficult to dynamically adapt to differentiated user needs (such as mobile users preferring low power consumption, desktop users seeking high performance, and edge computing scenarios requiring a trade-off between latency and cost). Especially when multiple objectives conflict (such as energy efficiency versus real-time performance), personalized scheduling cannot be achieved. Summary of the Invention

[0004] To address the aforementioned technical deficiencies, this invention provides an operating system adaptive task scheduling method and apparatus, a computer device, a computer-readable storage medium, and a computer program product, which embeds a deep reinforcement learning task scheduling model into the operating system kernel to achieve closed-loop optimization of dynamic resource allocation.

[0005] This invention provides an adaptive task scheduling method for an operating system, comprising:

[0006] During the operating system factory setup phase, the task scheduling model embedded in the operating system is pre-trained to obtain the parameters of the task scheduling model. The task scheduling model is configured with multiple task scheduling modes, and the parameters of the task scheduling model include: electricity cost reward, resource utilization reward, service quality reward, and reward weight parameters. The reward weight parameters include: electricity cost reward weight, resource utilization reward weight, and service quality reward weight. Among them, the electricity cost reward is determined based on the average electricity cost required for the task scheduling model to execute task scheduling. The average electricity cost is the product of the average execution time of all tasks to be executed within a unit of time determined by the task scheduling model and the normalized peak-valley electricity price.

[0007] The parameters of the pre-trained task scheduling model are sent to the user-side operating system. Based on the current user's preference for electricity cost, resource utilization and service quality, the parameters of the task scheduling model in the user-side operating system are optimized to obtain a task scheduling model that matches the current user's preferences.

[0008] In the user-side operating system, task scheduling is performed using a task scheduling model that matches the current user's preferences, based on the task scheduling mode selected by the current user.

[0009] In this embodiment of the invention, the relationship between the electricity price cost incentive, resource utilization incentive, service quality incentive, and incentive weight parameter is as follows:

[0010] ;

[0011] Where, r t For immediate rewards, Cp t Cr is the electricity price cost bonus generated when the action is performed at time t. t Cq is the resource utilization reward at time t. t As a reward for service quality;

[0012] α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. α1+α2+α3=1, and α1, α2, and α3≥0.

[0013] In this embodiment of the invention, the electricity price cost incentive Cp t It is a monotonically decreasing function with the average electricity cost required for the task scheduling model to execute task scheduling as the independent variable;

[0014] The average electricity cost is determined based on the average execution time of all tasks that have obtained resource allocation per unit time, and the impact of the execution time of tasks that have not obtained resource allocation on the average execution time is eliminated by a sign function.

[0015] Electricity cost incentive Cp tThe calculation expression is:

[0016] ;

[0017] in, It is a monotonically decreasing function. [0,1], e is a constant;

[0018] This represents the average electricity cost;

[0019] EP t N represents the normalized electricity price at time t. t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1.

[0020] For symbolic functions, This represents the CPU resource allocation rate for each task in the ready queue at time t. Indicates the first i The CPU resource rate required for each ready task.

[0021] In this embodiment of the invention, the resource utilization rate reward Cr t Let S be the state space at time t. t and action space A t The environmental state S obtained by the action t+1 CPU load rate CL t+1 ;

[0022] Wherein, the state space S at time t t Includes: CPU load rate CL at time t t Normalized electricity price EP at time t t and ready queue information RQ t ;

[0023] Action space A at time t t Let t be the CPU resource allocation rate for each task in the ready queue. .

[0024] In this embodiment of the invention, the service quality reward Cq t The calculation expression is:

[0025] ;

[0026] ;

[0027] in, [0,1], e is a constant;

[0028] N tLet N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1.

[0029] Represents the time t. i The service quality of each ready task Represents the time t+1. i Service quality of each ready task;

[0030] when If the service quality does not improve after implementing adaptive task scheduling, the service reward will be 0.

[0031] when If the service quality improves after implementing adaptive task scheduling, then the service reward is greater than 0.

[0032] In this embodiment of the invention, the task scheduling model uses a multi-objective reward weight optimization model based on user preferences to optimize the reward weight parameters;

[0033] The multi-objective reward weight optimization model is as follows:

[0034] ;

[0035] Wherein, α1 is the electricity price cost incentive weight, α2 is the resource utilization rate incentive weight, and α3 is the service quality incentive weight. , These are mathematical symbols used to represent constraints. For sorting functions, This indicates user preference constraints.

[0036] In this embodiment of the invention, the task scheduling model includes: an actor network, a critic network, a target actor network, and a target critic network. The parameters of the task scheduling model also include: actor network parameters, critic network parameters, target actor network parameters, and target critic network parameters.

[0037] The expression for the loss function of the commentator network is:

[0038] ;

[0039] ;

[0040] in, L Q This represents the loss function of the critic network. This refers to the network of critics. , , S represents the network parameters. i and Si+1 Indicates the environmental state, A i Indicates an action, N b Indicates the number of samples in the batch;

[0041] y i For the loss function parameters, r i Indicates immediate reward, γ represents discount factor; Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; Indicates a network of target commentators. This represents the corresponding target actor network;

[0042] This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

[0043] In this embodiment of the invention, the task scheduling model is pre-trained during the operating system factory setup phase, including:

[0044] Initialize the actor network, critic network, target actor network, and target critic network; initialize the reward weights, the experience replay area for the corresponding task scheduling mode, the network update step size, and the reward weight update step size.

[0045] Select an action in the environment, perform the action to receive an immediate reward and environmental status, and save it to the experience replay area;

[0046] Randomly sample from the experience playback area;

[0047] Update the critic network by minimizing the loss function;

[0048] Update the actor network by maximizing the output of the critic network;

[0049] By maximizing cumulative rewards and user preference levels, the reward weights are solved using gradient descent.

[0050] Update the target actor network and target critic network according to the network update step size.

[0051] In this embodiment of the invention, the expression for updating the commentator network using the minimum loss function is as follows:

[0052] ;

[0053] ;

[0054] in, L Q This represents the loss function of the critic network. Let S represent the critic network for the m-th task scheduling mode. i and S i+1 N represents the environmental state. b Indicates the number of samples in the batch;

[0055] y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; Describes the target commentator network for the m-th task scheduling mode. This represents the corresponding target actor network;

[0056] This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

[0057] In this embodiment of the invention, the expression for updating the actor network by maximizing the output of the critic network is:

[0058] ;

[0059] in, L A Let N represent the loss function of the actor network. b Indicates the number of samples in the batch;

[0060] Represents the actor network for the m-th task scheduling mode;

[0061] Indicates correspondence A network of critics.

[0062] In this embodiment of the invention, the expression for solving the reward weight using the gradient descent method is as follows:

[0063] ;

[0064] Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses;

[0065] This represents the i-th sample randomly sampled from the experience replay area, r i The corresponding electricity price cost incentives, resource utilization incentives, and service quality incentives, This represents the electricity price cost reward weight, resource utilization rate reward weight, and service quality reward weight for the m-th task scheduling mode.

[0066] This represents the maximum long-term cumulative reward for the m-th task scheduling mode;

[0067] This represents user preference constraints. , This represents the user preference constraint parameters.

[0068] In this embodiment of the invention, the parameters of the task scheduling model in the user-side operating system are optimized, including:

[0069] Initialize the task scheduling mode of the task scheduling model, initialize the reward weight, the long-term experience replay area and short-term experience replay area corresponding to the task scheduling mode, the network update step size and the reward weight update step size, and initialize the experience accumulation start time t0.

[0070] Freeze the underlying parameters of the task scheduling model;

[0071] Select an action in the environment, execute the action to obtain immediate rewards and environment status, and store them in the short-term experience replay area or the long-term experience replay area;

[0072] Randomly sample from either the short-term or long-term experience playback area;

[0073] The non-lower-level parameters of the commentator network are updated using a loss function based on elastic weight integration.

[0074] The non-lower-level parameters of the actor network are updated by maximizing the output of the critic network, and the loss function is adjusted by elastic weight integration.

[0075] By maximizing cumulative rewards and user preference levels, the reward weights are solved using gradient descent.

[0076] Update the target actor network and target critic network according to the network update step size.

[0077] In this embodiment of the invention, selecting an action in the environment, executing the action to obtain an immediate reward and environmental state, and storing them in a short-term experience replay area or a long-term experience replay area includes:

[0078] If the difference between the current time t and the empirical accumulation start time t0 is not the empirical time interval T L Multiples of this value will store immediate rewards and environmental states in the short-term experience replay area.

[0079] If the difference between the current time t and the empirical accumulation start time t0 is the empirical time interval T L Multiples of this value will store immediate rewards and environmental states in the long-term experience replay area.

[0080] In this embodiment of the invention, randomly sampling samples from the short-term experience replay area or the long-term experience replay area includes: if the sample size in both the current short-term experience replay area and the long-term experience replay area is greater than half of the batch processing sample size, randomly sampling samples from the short-term experience replay area or the long-term experience replay area; if the sample size in the current short-term experience replay area or the long-term experience replay area is less than half of the batch processing sample size, reselecting an action in the environment.

[0081] In this embodiment of the invention, when optimizing the task scheduling model in the user-side operating system, the expression for minimizing the loss function based on elastic weight integration is:

[0082] ;

[0083] ;

[0084] in, L Q This represents the loss function of the critic network. The commentator network represents the incremental learning of the m-th task scheduling mode. , , , For network parameters, S i and S i+1 N represents the environmental state. b F represents the number of samples in a batch. i Here, ρ is the Fisher information matrix, and ρ is the regularization coefficient.

[0085] y iFor the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; This represents the target critic network for incremental learning of the m-th task scheduling mode. This represents the corresponding target actor network;

[0086] This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

[0087] In this embodiment of the invention, when optimizing the task scheduling model in the user-side operating system, the expression for adjusting the loss function using elastic weight integration is as follows:

[0088] ;

[0089] in, L A Let N represent the loss function of the actor network. b Indicates the number of samples in the batch;

[0090] The actor network represents the incremental learning of the m-th task scheduling mode. Indicates correspondence Critics network , , For network parameters, S i F represents the environmental state. i Let ρ be the Fisher information matrix, and ρ be the regularization coefficient.

[0091] In this embodiment of the invention, when optimizing the task scheduling model in the user-side operating system, the expression for solving the reward weight using the gradient descent method is as follows:

[0092] ;

[0093] Among them, L Re This represents the reward-weighted loss function. This indicates the update step size of the reward weight for user u. This represents the update amount of the reward weight for user u, and λ is the adjustment parameter for the two losses;

[0094] This represents the i-th sample randomly sampled from the experience replay area, r i The corresponding electricity price cost incentives, resource utilization incentives, and service quality incentives, This represents the electricity cost reward weight, resource utilization reward weight, and service quality reward weight for user u's m-th task scheduling mode.

[0095] Let m represent the maximum long-term cumulative reward for user u in the m-th task scheduling mode;

[0096] This represents the preference constraints for user u. , These are the preference constraint parameters for user u.

[0097] In this embodiment of the invention, in the user-side operating system, task scheduling is performed using a task scheduling model that matches the current user's preference, based on the task scheduling mode currently selected by the user. This includes:

[0098] Based on the task scheduling mode selected by the user, the current computer state is input into the actor network of the task scheduling model to obtain the action to be executed.

[0099] Resources are allocated to each task in the ready queue based on the action to be performed, thereby achieving dynamic task scheduling.

[0100] Another aspect of the present invention provides an adaptive task scheduling device applied in an operating system, the device comprising: a task scheduling model;

[0101] The task scheduling model is configured with multiple task scheduling modes. The parameters of the task scheduling model include: electricity cost reward, resource utilization reward, service quality reward, and reward weight parameters. The reward weight parameters include: electricity cost reward weight, resource utilization reward weight, and service quality reward weight. The electricity cost reward is determined based on the average electricity cost required for the task scheduling model to execute task scheduling. The average electricity cost is the product of the average execution time of all tasks to be executed within a unit of time determined by the task scheduling model and the normalized peak-valley electricity price.

[0102] The task scheduling model is pre-trained during the operating system's factory setup phase;

[0103] After the parameters of the pre-trained task scheduling model are sent to the user-side operating system, the user-side operating system optimizes the parameters of the task scheduling model in the user-side operating system based on the current user's preference for electricity cost, resource utilization and service quality, so as to obtain a task scheduling model that matches the current user's preferences. According to the task scheduling mode selected by the current user, the task scheduling is performed using the task scheduling model that matches the current user's preferences.

[0104] In this embodiment of the invention, the relationship between the electricity price cost incentive, resource utilization incentive, service quality incentive, and incentive weight parameter is as follows:

[0105] ;

[0106] Where, r t For immediate rewards, Cp t Cr is the electricity price cost bonus generated when the action is performed at time t. t Cq is the resource utilization reward at time t. t As a reward for service quality;

[0107] α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. α1+α2+α3=1, and α1, α2, and α3≥0.

[0108] In this embodiment of the invention, the electricity price cost incentive Cp t The calculation expression is:

[0109] ;

[0110] in, It is a monotonically decreasing function. [0,1], e is a constant;

[0111] This represents the average electricity cost;

[0112] EP t N represents the normalized electricity price at time t. t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1.

[0113] For symbolic functions, This represents the CPU resource allocation rate for each task in the ready queue at time t. Indicates the first i The CPU resource rate required for each ready task.

[0114] In this embodiment of the invention, the resource utilization rate reward Cr tLet S be the state space at time t. t and action space A t The environmental state S obtained by the action t+1 CPU load rate CL t+1 ;

[0115] Wherein, the state space S at time t t Includes: CPU load rate CL at time t t Normalized electricity price EP at time t t and ready queue information RQ t ;

[0116] Action space A at time t t Let t be the CPU resource allocation rate for each task in the ready queue. .

[0117] In this embodiment of the invention, the service quality reward Cq t The calculation expression is:

[0118] ;

[0119] ;

[0120] in, [0,1], e is a constant;

[0121] N t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1.

[0122] Represents the time t. i The service quality of each ready task Represents the time t+1. i Service quality of each ready task;

[0123] when If the service quality does not improve after implementing adaptive task scheduling, the service reward will be 0.

[0124] when If the service quality improves after implementing adaptive task scheduling, then the service reward is greater than 0.

[0125] In this embodiment of the invention, the task scheduling model employs a multi-objective reward weight optimization model based on user preferences to optimize the reward weight parameters; the multi-objective reward weight optimization model is as follows:

[0126] ;

[0127] Wherein, α1 is the electricity price cost incentive weight, α2 is the resource utilization rate incentive weight, and α3 is the service quality incentive weight. , These are mathematical symbols used to represent constraints. For sorting functions, This indicates user preference constraints.

[0128] In this embodiment of the invention, the loss functions for the electricity price cost reward weight, resource utilization rate reward weight, and service quality reward weight are as follows:

[0129] ;

[0130] Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses;

[0131] This refers to incentives for electricity cost reduction, resource utilization, and service quality. This indicates the weighting of electricity price cost incentives, resource utilization incentives, and service quality incentives.

[0132] This indicates maximizing long-term cumulative rewards;

[0133] This represents user preference constraints. , This represents the user preference constraint parameters.

[0134] In this embodiment of the invention, the task scheduling model includes: an actor network, a critic network, a target actor network, and a target critic network. The parameters of the task scheduling model also include: actor network parameters, critic network parameters, target actor network parameters, and target critic network parameters.

[0135] The task scheduling model is pre-trained during the operating system's factory setup phase, including:

[0136] Update the critic network by minimizing the loss function;

[0137] Update the actor network by maximizing the output of the critic network;

[0138] By maximizing the cumulative reward and the degree of alignment with user preferences, the reward weights are solved using the gradient descent method.

[0139] In this embodiment of the invention, the parameters of the task scheduling model are optimized in the user-side operating system, including:

[0140] The non-lower-level parameters of the commentator network are updated using a loss function based on elastic weight integration.

[0141] The non-lower-level parameters of the actor network are updated by maximizing the output of the critic network, and the loss function is adjusted by elastic weight integration.

[0142] By maximizing cumulative rewards and user preference levels, the reward weights are solved using gradient descent.

[0143] In this embodiment of the invention, when optimizing the task scheduling model in the user-side operating system, the expression for minimizing the loss function based on elastic weight integration is:

[0144] ;

[0145] ;

[0146] in, L Q This represents the loss function of the critic network. The commentator network represents the incremental learning of the m-th task scheduling mode. , , , For network parameters, S i and S i+1 N represents the environmental state. b F represents the number of samples in a batch. i Here, ρ is the Fisher information matrix, and ρ is the regularization coefficient.

[0147] y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; This represents the target critic network for incremental learning of the m-th task scheduling mode. This represents the corresponding target actor network;

[0148] This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

[0149] The present invention also provides a computer device, including: a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described operating system adaptive task scheduling method.

[0150] The present invention also provides a machine-readable storage medium having stored computer program instructions thereon, which, when executed by a processor, implement the above-described operating system adaptive task scheduling method.

[0151] The present invention also provides a computer program product comprising a computer program that, when executed by a processor, implements the above-described operating system adaptive task scheduling method.

[0152] By embedding a deep reinforcement learning-based task scheduling model into the operating system kernel, a closed-loop optimization of dynamic resource allocation is achieved. The task scheduling model incorporates a multi-objective reward weight optimization algorithm based on user preferences, used to measure different users' preferences for electricity cost, resource utilization, and service quality. This invention, through deep integration of electricity market signals and user intent understanding, deeply couples electricity cost, computing resources, service quality, and user preferences into an AI-driven operating system task scheduling algorithm, achieving joint optimization of energy, computing power, service, and users, and providing a technical path for green computing and experience optimization.

[0153] Other features and advantages of the technical solution of the present invention will be described in detail in the following detailed embodiments section. Attached Figure Description

[0154] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0155] Figure 1 This is a flowchart of the operating system adaptive task scheduling method provided in the embodiments of the present invention;

[0156] Figure 2 This is a block diagram of the adaptive task scheduling device provided in an embodiment of the present invention. Detailed Implementation

[0157] To make the technical solutions and advantages of the embodiments of the present invention clearer, the exemplary embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0158] Figure 1 This is a flowchart of the operating system adaptive task scheduling method provided in an embodiment of the present invention. Figure 1 As shown, the operating system adaptive task scheduling method provided in this embodiment includes the following steps:

[0159] In the S100, during the operating system factory setup phase, the task scheduling model embedded in the operating system is pre-trained to obtain the parameters of the task scheduling model. The task scheduling model is configured with multiple task scheduling modes. The parameters of the task scheduling model include: electricity cost reward, resource utilization reward, service quality reward, and reward weight parameters. Among them, the electricity cost reward is determined based on the average electricity cost required for the task scheduling model to execute task scheduling. The average electricity cost is the product of the average execution time of all tasks to be executed within a unit of time determined by the task scheduling model and the normalized peak-valley electricity price. The reward weight parameters are determined based on the user's preference for electricity cost, resource utilization, and service quality.

[0160] S200 sends the parameters of the pre-trained task scheduling model to the user-side operating system. Based on the current user's preference for electricity cost, resource utilization and service quality, the task scheduling model in the user-side operating system is optimized and trained to obtain a task scheduling model that matches the current user's preferences.

[0161] In the S300 operating system, task scheduling is performed based on the task scheduling mode selected by the current user and using a task scheduling model that matches the current user's preferences.

[0162] This invention proposes an adaptive task scheduling method for operating systems based on multi-objective reinforcement learning. It embeds a deep reinforcement learning task scheduling model into the operating system kernel to achieve closed-loop optimization of dynamic resource allocation. The task scheduling model incorporates a multi-objective reward weight optimization algorithm based on user preferences, used to measure different users' preferences for electricity cost, resource utilization, and service quality. By deeply integrating electricity market signals and user intent understanding, this invention deeply couples electricity cost, computing resources, service quality, and user preferences into an AI-driven operating system task scheduling algorithm, achieving joint optimization of energy, computing power, service, and users, and providing a technical path for green computing and experience optimization.

[0163] This method first pre-trains the task scheduling model embedded in the operating system during the factory setup phase. Then, it optimizes and trains the task scheduling model in the user-side operating system based on the current user's preferences for electricity cost, resource utilization, and service quality, resulting in a task scheduling model that matches the current user's preferences. This enables adaptive task scheduling from the operating system factory setup to scenarios with rapidly changing user preferences. Specifically, a multi-objective reward function based on reinforcement learning is designed to balance electricity cost, resource utilization, and service quality. A "user preference" constraint index is also designed, and a multi-objective reward weight optimization method based on user preferences is constructed to measure the different users' preferences for electricity cost, resource utilization, and service quality. Furthermore, considering the urgency of periodic tasks in real-world scenarios, the service quality of these tasks is used to reshape the reinforcement learning loss function. Finally, an incremental learning-based reinforcement learning model optimization method is used to achieve adaptive task scheduling from the operating system factory setup to scenarios with rapidly changing user preferences.

[0164] This invention presents a task scheduling model based on deep reinforcement learning. Building upon the existing Deep Deterministic Policy Gradient (DDPG) algorithm, it designs a multi-objective reward function, a user-preference-based multi-objective reward weight optimization method, and a value network loss function based on periodic task constraints. This provides an innovative and improved reinforcement learning model optimization method based on incremental learning. The DDPG algorithm, a policy gradient-based approach, combines the advantages of deep neural networks and deterministic policies, making it suitable for control tasks with continuous action spaces, such as robotic arm control and autonomous driving. The DDPG algorithm optimizes the policy by simultaneously training an actor network and a critic network.

[0165] In the operating system adaptive task scheduling method based on multi-objective reinforcement learning, the state space at time t... Including CPU load rate at time t Normalized electricity price at time t Ready queue information The ready queue information includes the CPU resource ratio required by the i-th ready task. (Required CPU resources / Total CPU resources), Waiting time at time t Deadline Service quality .

[0166] The electricity price at time t can be determined according to the peak-valley electricity price stipulated in the electricity market transaction, and the electricity price at time t is normalized: normalized electricity price = electricity price at time t / peak electricity price.

[0167] Service quality needs to be determined for different types of tasks: when the task is aperiodic, service quality is inversely proportional to waiting time, and Q(waiting time at time t) is a monotonically decreasing function. ;

[0168] ;

[0169] When the task is a periodic task, the quality of service is:

[0170] ;

[0171] In the operating system adaptive task scheduling method based on multi-objective reinforcement learning, the action space at time t... This refers to the CPU resource allocation to each task in the ready queue. The allocated CPU resource rate ranges from [0,1]. If a ready task is allocated a CPU resource rate of 0, it means that the task needs to continue waiting.

[0172] The task scheduling model of this invention includes: an actor network, a critic network, a target actor network, and a target critic network. A multi-objective reward function is designed based on the network model to balance electricity cost, resource utilization, and service quality. Let the actor network be... Target actor network Critics Network Target Commentator Network The state at time t Input actor network In the middle, we obtained ,in , Let t be the number of tasks in the ready queue, and t be the number of actions to execute. Then, the environmental state is obtained. .but Instant Rewards The expression is:

[0173] ;

[0174] Where, r t For immediate rewards, Cp t Execute the action at time t The electricity price cost incentive generated during the period, Cr t Cq is the resource utilization reward at time t. t As a reward for service quality;

[0175] α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. α1+α2+α3=1, and α1, α2, and α3≥0.

[0176] Electricity cost reward measures the electricity cost bonus generated at a given moment after executing an adaptive task scheduling strategy. The electricity cost reward is related to the average electricity cost required to execute the adaptive task scheduling strategy at a given moment under a peak-valley electricity pricing system; the higher the average electricity cost, the lower the electricity cost reward. Higher electricity prices also lead to higher average electricity costs; longer average execution times also result in higher average electricity costs. Average execution time is affected by the CPU resource allocation rate of the task scheduling model to ready tasks. Therefore, how to reasonably allocate CPU resource rates to ready tasks under a peak-valley electricity pricing system determines the size of the electricity cost reward.

[0177] The electricity cost incentive is a monotonically decreasing function with the average electricity cost required to execute task scheduling in the task scheduling model as the independent variable. The average electricity cost is determined based on the average execution time of all tasks that have received resource allocation per unit time, and the impact of the execution time of tasks that have not received resource allocation on the average execution time is eliminated through a sign function.

[0178] Execute the action at time t The electricity price cost incentive generated during this period is expressed as follows:

[0179] ;

[0180] in, It is a monotonically decreasing function. [0,1], e is a constant;

[0181] The average electricity cost is the product of the average execution time of all tasks to be executed as determined by the task scheduling model at time t and the normalized peak-valley electricity price. A higher average electricity cost corresponds to a lower electricity cost incentive.

[0182] EP t N represents the normalized electricity price at time t. t Let S be the environmental state at time t. t The number of remaining tasks in the ready queue, N t+1 This represents the number of tasks remaining in the ready queue at time t+1. For symbolic functions, This represents the CPU resource allocation rate for each task in the ready queue at time t. Indicates the first i The CPU resource rate required for each ready task.

[0183] This represents the reward for resource utilization at time t. Resource utilization is essentially... The environmental state obtained by action CPU load rate ,Right now: ,in, [0,1], the state space S at time t t Includes: CPU load rate CL at time t t Normalized electricity price EP at time t t and ready queue information RQ t Action space A at time t t Let t be the CPU resource allocation rate for each task in the ready queue. .

[0184] CPU load rate CL t+1 The CPU resource allocation to each task in the ready queue at time t is determined by the task scheduling model. The higher the total resource allocation rate, the higher the CPU load rate (CL). t+1 The higher the resource utilization rate, the greater the reward Cr. t The higher.

[0185] Indicating service quality rewards actually reflects the status. and environmental conditions The service quality (SQ) changes between the two periods. The SQ reward depends on the improvement in SQ after implementing adaptive task scheduling; the improvement refers to the difference in average SQ before and after implementing adaptive task scheduling. Service Quality Reward (Cq) t The calculation expression is:

[0186] ;

[0187] ;

[0188] in, Indicates the degree of improvement in service quality. [0,1], e is a constant;

[0189] N t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1.

[0190] Represents the time t. i The service quality of each ready task Represents the time t+1. i Service quality of each ready task;

[0191] This represents the average quality of service after adaptive task scheduling is performed.

[0192] This represents the average quality of service before adaptive task scheduling is executed.

[0193] when If the service quality does not improve after implementing adaptive task scheduling, the service reward will be 0.

[0194] when If the service quality improves after implementing adaptive task scheduling, then the service reward is greater than 0.

[0195] Service quality rewards depend on the degree of service quality improvement after implementing adaptive task scheduling. If there is no improvement in service quality after implementing adaptive task scheduling, the service reward is 0; if there is an improvement in service quality after implementing adaptive task scheduling, the service reward is proportional to the degree of service quality improvement, and the greater the degree of service quality improvement, the greater the service reward.

[0196] This invention designs a service quality reward by measuring the average difference in service quality before and after adaptive task scheduling. The service reward only takes effect when the service quality improves after the adaptive task scheduling is executed.

[0197] As can be seen from the above, in terms of instant rewards The reward weight parameters α1, α2, and α3 can be used to balance electricity cost, resource utilization, and service quality. The reward weight parameters are optimized by minimizing the weighted sum of electricity cost reward, resource utilization reward, and service quality reward. The constraint requires that the order of these three weights aligns with the user's preference order for electricity cost, resource utilization, and service quality. The immediate reward is the weighted sum of the electricity cost reward, resource utilization reward, service quality reward, and their corresponding reward weight parameters. The effectiveness of the immediate reward in the task scheduling model during the solution process is determined by the periodic reward, which is characterized by whether there are any periodic tasks in the ready queue that have reached their deadline but have not yet been executed after the adaptive task scheduling is executed.

[0198] In order to rationally select reward weight parameters and incorporate user preferences, this invention designs a "user preference" constraint index and constructs a multi-objective reward weight optimization method based on user preferences to measure the degree of preference of different users for electricity cost, resource utilization and service quality.

[0199] First, three operating system task scheduling modes are set: "low power consumption," "high performance," and "strong responsiveness," corresponding to user preferences of low electricity cost, high resource utilization, and high service quality, respectively. The user preference constraints are set as follows: These are user preference scores for electricity cost, resource utilization rate, and service quality, respectively. When a user selects "low energy consumption," When the user selects "high performance", When the user selects "Strong Response", Parameters α1, α2, and α3 are learnable parameters. Then, a multi-objective reward weight optimization model based on user preferences is constructed as follows:

[0200] ;

[0201] Wherein, α1 is the electricity price cost incentive weight, α2 is the resource utilization rate incentive weight, and α3 is the service quality incentive weight. , These are mathematical symbols used to represent constraints. For sorting functions, This indicates user preference constraints.

[0202] The task scheduling model employs a multi-objective reward weight optimization model based on user preferences to optimize the reward weight parameters. This multi-objective reward weight optimization model aims to minimize the sum of weighted rewards for electricity cost, resource utilization, and service quality. The constraint requires that the order of these three rewards—electricity cost reward weight, resource utilization reward weight, and service quality reward weight—aligns with the user's preference order for electricity cost, resource utilization, and service quality.

[0203] Based on the aforementioned multi-objective reward weight optimization model, the gradient descent method is used to solve the problem, and the reward weights are designed. The loss function is:

[0204] ;

[0205] ;

[0206] ;

[0207] Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses;

[0208] This refers to incentives for electricity cost reduction, resource utilization, and service quality. This indicates the weighting of electricity price cost incentives, resource utilization incentives, and service quality incentives.

[0209] This indicates maximizing long-term cumulative rewards; , Indicates user preference constraint parameters, This represents user preference constraints, corresponding to the multi-objective reward weight optimization model. The constraints.

[0210] In scenarios such as aerospace, power, and vehicles, periodic tasks are required to be processed within a specified time to ensure that regular, prescribed actions are completed on time. However, existing operating systems lack a priority processing mechanism for periodic tasks, resulting in a lack of guarantee for the timely completion of periodic tasks in critical scenarios.

[0211] This invention takes into account the urgency of periodic tasks in real-world scenarios and designs a hard index for periodic task rewards to reshape the reinforcement learning loss function. The periodic task reward is related to the number of periodic tasks with poor service quality in the ready queue after the execution of adaptive task scheduling, so as to ensure that periodic tasks can be executed on schedule.

[0212] This invention utilizes the service quality reshaping loss function of reinforcement learning based on periodic tasks, and incorporates the commenter network... The loss function of a network is defined as:

[0213] ;

[0214] ;

[0215] in, L Q This represents the loss function of the critic network. This refers to the network of critics. , , S represents the network parameters. i and S i+1 Indicates the environmental state, A i Indicates an action, N b Indicates the number of samples in the batch;

[0216] y i For the loss function parameters, r i Indicates immediate reward, γ represents discount factor; Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; Indicates a network of target commentators. This represents the corresponding target actor network;

[0217] This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state Si+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i Invalid. This loss function can be used to constrain the model to prioritize the execution of periodic tasks on schedule.

[0218] The validity of the immediate reward in the critic network loss function is determined based on the periodic reward. The periodic reward is characterized by whether there are any periodic tasks in the ready queue that have reached their deadline but have not yet been executed after adaptive task scheduling. If there are still periodic tasks in the ready queue that have reached their deadline but have not yet been executed after adaptive task scheduling, the periodic reward is 0, and the immediate reward is invalid; if there are no periodic tasks in the ready queue that have reached their deadline but have not yet been executed after adaptive task scheduling, the periodic reward is 1, and the immediate reward is valid. Periodic tasks that have reached their deadline but have not yet been executed are characterized by a service quality of 0.

[0219] This invention utilizes a reinforcement learning model optimization method based on incremental learning to achieve adaptive task scheduling in scenarios where user preferences change rapidly, from initial operating system settings to user preferences. The model training and optimization consist of two stages.

[0220] First, in the initial stage, during the operating system's factory setup, the model parameters for three operating system task scheduling modes—"low power consumption," "high performance," and "strong responsiveness"—are pre-trained. Let the reinforcement model for the m-th mode be: Actor Network. and the network of critics Target actor network and the target critic network Reward weight The training process is as follows:

[0221] Step 1: Initialization: Randomly initialize the actor network and the network of critics and will , The initial target actor network is obtained. and the target critic network Randomly initialize reward weights Initialize the experience replay area for the m-th mode. Initialize network update step size and reward weight update step size .

[0222] Step 2: For each round:

[0223] (1) Selecting actions in the environment , Noise is used for exploratory actions;

[0224] (2) Performing actions Receive instant rewards and environmental conditions ;

[0225] (3) Store in the experience replay area ;

[0226] (4) Random sampling from the experience replay area One sample;

[0227] (5) Update using minimizing the loss function network:

[0228] ;

[0229] ;

[0230] in, L Q This represents the loss function of the critic network. Let S represent the critic network for the m-th task scheduling mode. i and S i+1 N represents the environmental state. b Indicates the number of samples in the batch; y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; Describes the target commentator network for the m-th task scheduling mode. This represents the corresponding target actor network; This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid;

[0231] (6) By maximizing Network output to update actor network :

[0232] ;

[0233] in,L A Let N represent the loss function of the actor network. b Indicates the number of samples in the batch; Represents the actor network for the m-th task scheduling mode; Indicates correspondence A network of critics;

[0234] (7) Solve for the reward weight parameters by maximizing the cumulative reward and the degree of alignment with user preferences using the gradient descent method. :

[0235] ;

[0236] Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses;

[0237] This represents the i-th sample randomly sampled from the experience replay area, r i The corresponding electricity price cost incentives, resource utilization incentives, and service quality incentives, This represents the electricity price cost reward weight, resource utilization rate reward weight, and service quality reward weight for the m-th task scheduling mode.

[0238] This represents the maximum long-term cumulative reward for the m-th task scheduling mode;

[0239] This represents user preference constraints. , Indicates user preference constraint parameters; Represents the user preference constraint for the m-th task scheduling mode;

[0240] (8) With step size Soft update target network:

[0241]

[0242]

[0243] Step 3: Repeat step 2 until convergence.

[0244] The objective of this invention for optimizing reward weights is to maximize the alignment between long-term cumulative rewards and user preferences, and the reward weights are solved using gradient descent. Specifically, the long-term cumulative reward is designed as the weighted sum of long-term electricity cost rewards, long-term resource utilization rewards, and long-term service quality rewards; the alignment with user preferences is designed as the ranking loss between user preference scores for electricity cost, resource utilization, and service quality and the reward weights.

[0245] Then, in the second stage, when the operating system's task scheduling algorithm is deployed on the user side, incremental learning is used for model fine-tuning to better adapt to scenarios with rapidly changing user preferences. This enables rapid adjustment of the task scheduling strategy and prevents catastrophic model amnesia. Since the task scheduling strategy essentially remains unchanged from the operating system's factory settings to scenarios with rapidly changing user preferences, the only difference lies in the hardware conditions. Therefore, this invention achieves incremental learning in three ways: freezing the model's underlying parameters to prevent excessive disruption during user-side fine-tuning; constructing a dynamic experience replay zone on the user side to lengthen the model fine-tuning timeline and slow down the rate of model change; and introducing regularization based on elastic weight integration to encourage the new model parameters trained on the user side to be as close as possible to the factory-set model parameters.

[0246] The model parameters for three operating system task scheduling modes—"low power consumption," "high performance," and "strong responsiveness"—pre-trained during the first-stage operating system factory setup are distributed to the user side. Let the reinforcement model for the m-th factory setting be: Actor Network. and the network of critics Target actor network and the target critic network Reward weight Let the reinforcement model for the m-th user-side mode be: actor network. and the network of critics Target actor network and the target critic network Set user-side reward weights. Set up a long-term experience playback area on the user side. The empirical time interval is Short-term experience replay area on the user side .

[0247] Step 1: Initialize the reinforcement model for the m-th user-side mode. , , , Initialize reward weights Initialize the long-term experience playback area on the user side for the m-th mode. and user-side short-term experience replay area Initialize network update step size and reward weight update step size Initialize the experience accumulation start time .

[0248] Step 2: Freeze the underlying parameters of the user-side mode reinforcement model. Let... , For low-level parameters, only non-low-level parameters are updated during user-side mode training. Similarly, let's assume... , , During user-side training, only non-lower-level parameters are updated. , , .

[0249] Step 3: Training the task scheduling strategy from factory settings to user-side settings:

[0250] For each round:

[0251] (1) Selecting actions in the environment , Noise is used for exploratory actions;

[0252] (2) Performing actions Receive instant rewards and environmental conditions ;

[0253] (3) Store in short-term experience replay area If the current time t-t0 is T L multiples of an integer, Store in long-term experience playback area Both long-term and short-term experience playback areas adopt the first-in, first-out principle. When the number of samples stored in the playback area exceeds the maximum capacity, the samples that entered each playback area first are discarded.

[0254] (4) If the sample size in both the current long-term and short-term experience replay areas is greater than Then continue to step (5); if the sample size of the current long-term or short-term experience replay area is less than If so, return to step (1);

[0255] (5) Long-term experience playback area from the user side and user-side short-term experience replay area Random sampling One sample;

[0256] (6) Update using the minimum loss function based on elastic weight integration. The non-low-level parameters in the network are expressed as follows:

[0257] ;

[0258] ;

[0259] in, L Q This represents the loss function of the critic network. The commentator network represents the incremental learning of the m-th task scheduling mode. , , , For network parameters, S i and S i+1 N represents the environmental state. b F represents the number of samples in a batch. i Here, y represents the Fisher information matrix, ρ is the regularization coefficient, and y represents the Fisher information matrix. i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks in the ready queue with a service quality of 0. This represents the target critic network for incremental learning of the m-th task scheduling mode. This represents the corresponding target actor network; This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid;

[0260] (7) By maximizing Network output to update actor network The non-lower-level parameters are integrated using elastic weights to adjust the loss function:

[0261] ;

[0262] in, L A Let N represent the loss function of the actor network. b Indicates the number of samples in the batch;

[0263] The actor network represents the incremental learning of the m-th task scheduling mode. Indicates correspondence Critics network , , For network parameters, S i F represents the environmental state. i Here, ρ is the Fisher information matrix, and ρ is the regularization coefficient.

[0264] (8) Solve for the reward weight parameters by maximizing the cumulative reward and the degree of alignment with user preferences using the gradient descent method. Its expression is:

[0265] ;

[0266] Among them, L Re This represents the reward-weighted loss function. This indicates the update step size of the reward weight for user u. This represents the update amount of the reward weight for user u, and λ is the adjustment parameter for the two losses;

[0267] This represents the i-th sample randomly sampled from the experience replay area, r i The corresponding electricity price cost incentives, resource utilization incentives, and service quality incentives, This represents the electricity cost reward weight, resource utilization reward weight, and service quality reward weight for user u's m-th task scheduling mode.

[0268] Let m represent the maximum long-term cumulative reward for user u in the m-th task scheduling mode;

[0269] This represents the preference constraints for user u. , These are the preference constraint parameters for user u; Represents the user preference constraint for the m-th task scheduling mode for user u;

[0270] (9) With step length Soft update target network:

[0271]

[0272]

[0273] Repeat steps (1)-(9) above until convergence.

[0274] To optimize the parameters of the task scheduling model in the second stage, this invention constructs a user-side dynamic experience replay region, comprising a short-term experience replay region and a long-term experience replay region. This accumulates multi-granularity samples, from fine to coarse, lengthening the model fine-tuning timeline, slowing down the model's rate of change, and preventing catastrophic forgetting. Specifically, the short-term experience replay region accumulates short-term samples with small time spans to adapt to real-time changes; the long-term experience replay region accumulates long-term samples with larger time spans to prevent catastrophic forgetting.

[0275] Simultaneously, task scheduling strategies are rapidly adjusted on the user side by introducing a regularization term based on elastic weight integration into the optimization objective of the critic network. We encourage users to train new model parameters that are as close as possible to the factory-set model parameters to prevent catastrophic model amnesia.

[0276] This invention presents an adaptive task scheduling method for operating systems based on multi-objective reinforcement learning. It embeds deep reinforcement learning into the operating system kernel to achieve closed-loop optimization of dynamic resource allocation. A multi-objective reward function is designed to balance electricity cost, resource utilization, and service quality. A "user preference ranking" constraint index is designed, and a multi-objective reward weight optimization model based on user preferences is constructed to measure the degree of preference of different users for electricity cost, resource utilization, and service quality. Considering the urgency of periodic tasks in real-world scenarios, this invention reshapes the loss function of reinforcement learning using the service quality of periodic tasks, constraining the model to prioritize the execution of periodic tasks on schedule. This invention's reinforcement learning model optimization method based on incremental learning enables adaptive task scheduling from the initial operating system settings to scenarios with rapidly changing user preferences.

[0277] Figure 2 This is a block diagram of the adaptive task scheduling device provided in an embodiment of the present invention. Figure 2As shown in this embodiment, the adaptive task scheduling device applied to an operating system includes: a task scheduling model, which is configured with multiple task scheduling modes, including: an actor network, a critic network, a target actor network, and a target critic network. The parameters of the task scheduling model include: network parameters and reward weight parameters for electricity cost rewards, resource utilization rewards, and service quality rewards. The electricity cost reward is determined based on the average electricity cost required for the task scheduling model to execute task scheduling. The average electricity cost is the product of the average execution time of all tasks to be executed within a unit of time determined by the task scheduling model and the normalized peak-valley electricity price. The reward weight parameters are determined based on the user's preference for electricity cost, resource utilization, and service quality. The task scheduling model is pre-trained during the operating system's factory setup phase. After the parameters of the pre-trained task scheduling model are sent to the user-side operating system, the user-side operating system optimizes and trains the task scheduling model in the user-side operating system based on the current user's preference for electricity cost, resource utilization and service quality, so as to obtain a task scheduling model that matches the current user's preferences. According to the task scheduling mode selected by the current user, the task scheduling is performed using the task scheduling model that matches the current user's preferences.

[0278] The reward weight parameters of the task scheduling model include: electricity cost reward weight, resource utilization rate reward weight, and service quality reward weight. The relationship between these three reward weights is expressed as follows:

[0279] ;

[0280] Where, r t For immediate rewards, Cp t Cr is the electricity price cost bonus generated when the action is performed at time t. t Cq is the resource utilization reward at time t. t As a reward for service quality;

[0281] α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. α1+α2+α3=1, and α1, α2, and α3≥0.

[0282] The electricity cost incentive is a monotonically decreasing function with the average electricity cost required to execute task scheduling in the task scheduling model as its independent variable. The average electricity cost is determined based on the average execution time of all tasks that receive resource allocation per unit time, and the impact of the execution time of tasks that do not receive resource allocation on the average execution time is eliminated using a sign function. The electricity cost incentive is Cp. t The calculation expression is:

[0283] ;

[0284] in, It is a monotonically decreasing function. [0,1], e is a constant;

[0285] This represents the average electricity cost;

[0286] EP t N represents the normalized electricity price at time t. t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1. For symbolic functions, This represents the CPU resource allocation rate for each task in the ready queue at time t. Indicates the first i The CPU resource rate required for each ready task.

[0287] Resource utilization rate bonus Cr t Let S be the state space at time t. t and action space A t The environmental state S obtained by the action t+1 CPU load rate CL t+1 Wherein, the state space S at time t t Includes: CPU load rate CL at time t t Normalized electricity price EP at time t t and ready queue information RQ t Action space A at time t t Let t be the CPU resource allocation rate for each task in the ready queue. .

[0288] Service quality reward Cq t The calculation expression is:

[0289] ;

[0290] in, [0,1], e is a constant;

[0291] N t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1. Represents the time t. i The service quality of each ready task Represents the time t+1. i Service quality of each ready task;

[0292] When Cq t When =1, =1 indicates optimal service quality; when Cq t When =0, =0 indicates the worst service quality variation.

[0293] The task scheduling model employs a multi-objective reward weight optimization model based on user preferences to optimize the reward weight parameters. The multi-objective reward weight optimization model is as follows:

[0294] ;

[0295] Wherein, α1 is the electricity price cost incentive weight, α2 is the resource utilization rate incentive weight, and α3 is the service quality incentive weight. , These are mathematical symbols used to represent constraints. For sorting functions, This indicates user preference constraints.

[0296] The loss function for the electricity price cost incentive weight α1, the resource utilization incentive weight α2, and the service quality incentive weight α3 is as follows:

[0297] ;

[0298] Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses;

[0299] This refers to incentives for electricity cost reduction, resource utilization, and service quality. This indicates the weighting of electricity price cost incentives, resource utilization incentives, and service quality incentives.

[0300] This indicates maximizing long-term cumulative rewards;

[0301] This represents user preference constraints. , This represents the user preference constraint parameters.

[0302] The process of pre-training the task scheduling model during the operating system's factory setup includes: updating the critic network by minimizing the loss function; updating the actor network by maximizing the output of the critic network; and solving for the reward weights using gradient descent by maximizing the cumulative reward and the degree of user preference alignment.

[0303] The process of optimizing and training the task scheduling model in the user-side operating system includes: updating the non-lower-level parameters of the critic network by minimizing the loss function based on elastic weight integration; updating the non-lower-level parameters of the actor network by maximizing the output of the critic network and adjusting the loss function using elastic weight integration; and solving for the reward weights by maximizing the cumulative reward and the degree of user preference conformity using gradient descent.

[0304] Specifically, when optimizing and training the task scheduling model in the user-side operating system, the expression for minimizing the loss function based on elastic weight integration is:

[0305] ;

[0306] ;

[0307] in, L Q This represents the loss function of the critic network. The commentator network represents the incremental learning of the m-th task scheduling mode. , , , For network parameters, S i and S i+1 N represents the environmental state. b F represents the number of samples in a batch. i Here, ρ is the Fisher information matrix, and ρ is the regularization coefficient.

[0308] y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; This represents the target critic network for incremental learning of the m-th task scheduling mode. This represents the corresponding target actor network;

[0309] This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

[0310] The specific technical details and expected technical effects of the adaptive task scheduling device provided in this embodiment of the invention can be understood by referring to the technical details of the above-described adaptive task scheduling method for operating systems, and will not be repeated here.

[0311] The present invention also provides a computer device, including: a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the above-described operating system adaptive task scheduling method.

[0312] This invention also provides a machine-readable storage medium storing computer program instructions, which, when executed by a processor, implement the aforementioned operating system adaptive task scheduling method.

[0313] The present invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described operating system adaptive task scheduling method.

[0314] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0315] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0316] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0317] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0318] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. An adaptive task scheduling method for an operating system, characterized in that, include: During the operating system factory setup phase, the task scheduling model embedded in the operating system is pre-trained to obtain the parameters of the task scheduling model. The task scheduling model is configured with multiple task scheduling modes. The parameters of the task scheduling model include: electricity cost reward, resource utilization reward, service quality reward and reward weight parameters. Among them, the electricity cost reward is determined based on the average electricity cost required for the task scheduling model to execute task scheduling. The average electricity cost is the product of the average execution time of all tasks to be executed within a unit of time determined by the task scheduling model and the normalized peak-valley electricity price. The parameters of the pre-trained task scheduling model are sent to the user-side operating system. Based on the current user's preference for electricity cost, resource utilization and service quality, the parameters of the task scheduling model in the user-side operating system are optimized to obtain a task scheduling model that matches the current user's preferences. In the user-side operating system, task scheduling is performed based on the task scheduling mode selected by the current user, using a task scheduling model that matches the current user's preferences. The optimization of the task scheduling model parameters in the user-side operating system based on current user preferences for electricity costs, resource utilization, and service quality includes: A multi-objective reward weight optimization model based on user preferences is constructed. The multi-objective reward weight optimization model minimizes the sum of weighted electricity cost reward, resource utilization reward, and service quality reward. The user preference constraint of the multi-objective reward weight optimization model is that the order of the weights of electricity cost reward, resource utilization reward, and service quality reward is consistent with the user's preference order for electricity cost, resource utilization, and service quality. The reward weight parameters of the task scheduling model are solved using a multi-objective reward weight optimization model based on user preferences. The task scheduling model includes: an actor network, a critic network, a target actor network, and a target critic network. Optimization of the parameters of the task scheduling model includes: freezing the underlying parameters of the task scheduling model; selecting an action in the environment, executing the action to obtain immediate rewards and environmental states, and storing them in a short-term or long-term experience replay area; randomly sampling samples from the short-term or long-term experience replay area; updating the non-underlying parameters of the critic network using a loss function based on elastic weight integration; updating the non-underlying parameters of the actor network by maximizing the output of the critic network, and adjusting the loss function using elastic weight integration; solving for the reward weight parameters using gradient descent by maximizing cumulative rewards and user preference levels; and updating the target actor network and the target critic network according to the network update step size.

2. The operating system adaptive task scheduling method according to claim 1, characterized in that, The expression for the multi-objective reward weight optimization model based on user preferences is: ； Where a is the electricity price cost reward, b is the resource utilization rate reward, c is the service quality reward, α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. These are mathematical symbols used to represent constraints. For sorting functions, User preferences regarding electricity costs, resource utilization, and service quality; The order of the three rewards—electricity cost incentive weight, resource utilization incentive weight, and service quality incentive weight—is consistent with the user's preference order for electricity cost, resource utilization, and service quality.

3. The operating system adaptive task scheduling method according to claim 2, characterized in that, The loss function for the reward weight parameters is: ； Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses; This refers to incentives for electricity cost reduction, resource utilization, and service quality. This indicates the weighting of electricity price cost incentives, resource utilization incentives, and service quality incentives. This indicates maximizing long-term cumulative rewards; , Indicates user preference constraint parameters, This represents the user preference constraint, corresponding to the user preference constraint of the multi-objective reward weight optimization model.

4. The operating system adaptive task scheduling method according to claim 1, characterized in that, The relationship between electricity price cost incentives, resource utilization incentives, and service quality incentives is expressed as follows: ； Where, r t For immediate rewards, Cp t Cr is the electricity price cost bonus generated when the action is performed at time t. t Cq is the resource utilization reward at time t. t As a reward for service quality; α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. α1+α2+α3=1, and α1, α2, and α3≥0.

5. The operating system adaptive task scheduling method according to claim 1, characterized in that, The electricity price cost reward is a monotonically decreasing function with the average electricity price cost required for the task scheduling model to execute task scheduling as the independent variable; The average electricity cost is determined based on the average execution time of all tasks that have obtained resource allocation per unit time, and the impact of the execution time of tasks that have not obtained resource allocation on the average execution time is eliminated by a sign function.

6. The operating system adaptive task scheduling method according to claim 5, characterized in that, The calculation formula for the electricity price cost incentive is as follows: ； in, The electricity price cost incentive is represented by a monotonically decreasing function. [0,1], e is a constant; This represents the average electricity cost; EP t N represents the normalized electricity price at time t. t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1. For symbolic functions, This represents the CPU resource allocation rate for each task in the ready queue at time t. Indicates the first i The CPU resource rate required for each ready task.

7. The operating system adaptive task scheduling method according to claim 4, characterized in that, Resource utilization rate bonus Cr t Let S be the state space at time t. t and action space A t The environmental state S obtained by the action t+1 CPU load rate CL t+1 ; Wherein, the state space S at time t t Includes: CPU load rate CL at time t t Normalized electricity price EP at time t t and ready queue information RQ t ; Action space A at time t t Let t be the CPU resource allocation rate for each task in the ready queue. .

8. The operating system adaptive task scheduling method according to claim 4, characterized in that, Service quality reward Cq t The calculation expression is: ； ； in, [0,1], e is a constant; N t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1. Represents the time t. i The service quality of each ready task Represents the time t+1. i Service quality of each ready task; when If the service quality does not improve after implementing adaptive task scheduling, the service reward will be 0. when If the service quality improves after implementing adaptive task scheduling, then the service reward is greater than 0.

9. The operating system adaptive task scheduling method according to claim 4, characterized in that, The parameters of the task scheduling model also include: actor network parameters, critic network parameters, target actor network parameters, and target critic network parameters.

10. The operating system adaptive task scheduling method according to claim 9, characterized in that, The expression for the loss function of the commentator network is: ； ； in, L Q This represents the loss function of the critic network. This refers to the network of critics. , , S represents the network parameters. i and S i+1 Indicates the environmental state, A i Indicates an action, N b Indicates the number of samples in the batch; y i For the loss function parameters, r i Indicates immediate reward, γ represents discount factor; Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; Indicates a network of target commentators. This represents the corresponding target actor network; This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

11. The operating system adaptive task scheduling method according to claim 9, characterized in that, The task scheduling model is pre-trained during the operating system's factory setup phase, including: Initialize the actor network, critic network, target actor network, and target critic network; initialize the reward weights, the experience replay area for the corresponding task scheduling mode, the network update step size, and the reward weight update step size. Select an action in the environment, perform the action to receive an immediate reward and environmental status, and save it to the experience replay area; Randomly sample from the experience playback area; Update the critic network by minimizing the loss function; Update the actor network by maximizing the output of the critic network; By maximizing cumulative rewards and user preference levels, the reward weights are solved using gradient descent. Update the target actor network and target critic network according to the network update step size.

12. The operating system adaptive task scheduling method according to claim 11, characterized in that, The expression for updating the critic network by minimizing the loss function is: ； ； in, L Q This represents the loss function of the critic network. This represents the commentator network for the m-th task scheduling mode. , , Represents network parameters, A i Indicates an action, S i and S i+1 N represents the environmental state. b Indicates the number of samples in the batch; y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; Describes the target commentator network for the m-th task scheduling mode. This represents the corresponding target actor network; This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

13. The operating system adaptive task scheduling method according to claim 11, characterized in that, The expression for updating the actor network by maximizing the output of the critic network is: ； in, L A Let N represent the loss function of the actor network. b Indicates the number of samples in the batch; Represents the actor network for the m-th task scheduling mode; Indicates correspondence A network of critics.

14. The operating system adaptive task scheduling method according to claim 9, characterized in that, Optimize the parameters of the task scheduling model in the user-side operating system, including: Before freezing the underlying parameters of the task scheduling model, initialize the task scheduling mode of the task scheduling model, initialize the reward weight, the long-term experience replay area and short-term experience replay area of ​​the corresponding task scheduling mode, the network update step size and the reward weight update step size, and initialize the experience accumulation start time.

15. The operating system adaptive task scheduling method according to claim 14, characterized in that, Select an action within the environment, execute the action to receive an immediate reward and environmental status, and store them in either a short-term or long-term experience replay area, including: If the difference between the current time t and the empirical accumulation start time t0 is not the empirical time interval T L Multiples of this value will store immediate rewards and environmental states in the short-term experience replay area. If the difference between the current time t and the empirical accumulation start time t0 is the empirical time interval T L Multiples of this value will store immediate rewards and environmental states in the long-term experience replay area.

16. The operating system adaptive task scheduling method according to claim 14, characterized in that, Random samples are taken from either the short-term or long-term experience replay region, including: If the sample size in both the current short-term experience replay area and the long-term experience replay area is greater than half of the batch processing sample size, samples are randomly sampled from either the short-term experience replay area or the long-term experience replay area. If the sample size in the current short-term or long-term experience replay area is less than half the number of samples in the batch, select an action again in the environment.

17. The operating system adaptive task scheduling method according to claim 14, characterized in that, The expression for minimizing the loss function based on elastic weight integration is: ； ； in, L Q This represents the loss function of the critic network. The commentator network represents the incremental learning of the m-th task scheduling mode. , , , For network parameters, A i Indicates an action, S i and S i+1 N represents the environmental state. b F represents the number of samples in a batch. i Here, ρ is the Fisher information matrix, and ρ is the regularization coefficient. y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; This represents the target critic network for incremental learning of the m-th task scheduling mode. This represents the corresponding target actor network; This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

18. The operating system adaptive task scheduling method according to claim 14, characterized in that, The expression for adjusting the loss function using elastic weight integration is: ； in, L A Let N represent the loss function of the actor network. b Indicates the number of samples in the batch; The actor network represents the incremental learning of the m-th task scheduling mode. Indicates correspondence Critics network , , For network parameters, S i F represents the environmental state. i Let ρ be the Fisher information matrix, and ρ be the regularization coefficient.

19. The operating system adaptive task scheduling method according to claim 10, characterized in that, In the user-side operating system, task scheduling is performed based on the task scheduling mode selected by the current user, using a task scheduling model that matches the current user's preferences. This includes: Based on the task scheduling mode selected by the user, the current computer state is input into the actor network of the task scheduling model to obtain the action to be executed. Resources are allocated to each task in the ready queue based on the action to be performed, thereby achieving dynamic task scheduling.

20. An adaptive task scheduling device, applied in an operating system, characterized in that, The device includes: a task scheduling model; The task scheduling model is configured with multiple task scheduling modes. The parameters of the task scheduling model include: electricity cost reward, resource utilization reward, service quality reward, and reward weight parameters. Among them, the electricity cost reward is determined based on the average electricity cost required for the task scheduling model to execute task scheduling. The average electricity cost is the product of the average execution time of all tasks to be executed within a unit of time determined by the task scheduling model and the normalized peak-valley electricity price. The task scheduling model is pre-trained during the operating system's factory setup phase; After the parameters of the pre-trained task scheduling model are sent to the user-side operating system, the user-side operating system optimizes the parameters of the task scheduling model in the user-side operating system based on the current user's preference for electricity cost, resource utilization and service quality, so as to obtain a task scheduling model that matches the current user's preference. According to the task scheduling mode selected by the current user, the task scheduling is performed using the task scheduling model that matches the current user's preference. The user-side operating system optimizes the parameters of the task scheduling model based on the current user's preferences for electricity cost, resource utilization, and service quality, including: A multi-objective reward weight optimization model based on user preferences is constructed. The multi-objective reward weight optimization model minimizes the sum of weighted electricity cost reward, resource utilization reward, and service quality reward. The user preference constraint of the multi-objective reward weight optimization model is that the order of the weights of electricity cost reward, resource utilization reward, and service quality reward is consistent with the user's preference order for electricity cost, resource utilization, and service quality. The reward weight parameters of the task scheduling model are solved using a multi-objective reward weight optimization model based on user preferences. The task scheduling model includes: an actor network, a critic network, a target actor network, and a target critic network. Optimization of the parameters of the task scheduling model includes: freezing the underlying parameters of the task scheduling model; selecting an action in the environment, executing the action to obtain immediate rewards and environmental states, and storing them in a short-term or long-term experience replay area; randomly sampling samples from the short-term or long-term experience replay area; updating the non-underlying parameters of the critic network using a loss function based on elastic weight integration; updating the non-underlying parameters of the actor network by maximizing the output of the critic network, and adjusting the loss function using elastic weight integration; solving for the reward weight parameters using gradient descent by maximizing cumulative rewards and user preference levels; and updating the target actor network and the target critic network according to the network update step size.

21. The adaptive task scheduling device according to claim 20, characterized in that, The relationship between electricity price cost incentives, resource utilization incentives, and service quality incentives is expressed as follows: ； Where, r t For immediate rewards, Cp t Cr is the electricity price cost bonus generated when the action is performed at time t. t Cq is the resource utilization reward at time t. t As a reward for service quality; α1 is the weight of the electricity price cost reward, α2 is the weight of the resource utilization rate reward, and α3 is the weight of the service quality reward. α1+α2+α3=1, and α1, α2, and α3≥0.

22. The adaptive task scheduling device according to claim 20, characterized in that, The electricity price cost reward is a monotonically decreasing function with the average electricity price cost required for the task scheduling model to execute task scheduling as the independent variable; The average electricity cost is determined based on the average execution time of all tasks that have obtained resource allocation per unit time, and the impact of the execution time of tasks that have not obtained resource allocation on the average execution time is eliminated by a sign function.

23. The adaptive task scheduling device according to claim 22, characterized in that, The formula for calculating the electricity price cost incentive is as follows: ； in, The electricity price cost incentive is represented by a monotonically decreasing function. [0,1], e is a constant; This represents the average electricity cost; EP t N represents the normalized electricity price at time t. t Let N be the number of remaining tasks in the ready queue at time t. t+1 This represents the number of remaining tasks in the ready queue at time t+1. For symbolic functions, This represents the CPU resource allocation rate for each task in the ready queue at time t. Indicates the first i The CPU resource rate required for each ready task.

24. The adaptive task scheduling device according to claim 21, characterized in that, The task scheduling model uses a multi-objective reward weight optimization model based on user preferences to optimize the reward weight parameters; The multi-objective reward weight optimization model is as follows: ； Wherein, α1 is the electricity price cost incentive weight, α2 is the resource utilization rate incentive weight, and α3 is the service quality incentive weight. , These are mathematical symbols used to represent constraints. For sorting functions, This indicates user preference constraints.

25. The adaptive task scheduling device according to claim 21, characterized in that, The loss functions for the electricity price cost incentive weight, resource utilization incentive weight, and service quality incentive weight are as follows: ； Among them, L Re This represents the reward-weighted loss function. This indicates the step size for updating reward weights. This represents the update amount of the reward weights, and λ is the adjustment parameter for the two losses; This refers to incentives for electricity cost reduction, resource utilization, and service quality. This indicates the weighting of electricity price cost incentives, resource utilization incentives, and service quality incentives. This indicates maximizing long-term cumulative rewards; This represents user preference constraints. , This represents the user preference constraint parameters.

26. The adaptive task scheduling device according to claim 24, characterized in that, The parameters of the task scheduling model also include: actor network parameters, critic network parameters, target actor network parameters, and target critic network parameters; The task scheduling model is pre-trained during the operating system's factory setup phase, including: Update the critic network by minimizing the loss function; Update the actor network by maximizing the output of the critic network; By maximizing the cumulative reward and the degree of alignment with user preferences, the reward weights are solved using the gradient descent method.

27. The adaptive task scheduling device according to claim 26, characterized in that, The expression for minimizing the loss function based on elastic weight integration is: ； ； in, L Q This represents the loss function of the critic network. The commentator network represents the incremental learning of the m-th task scheduling mode. , , , For network parameters, A i Indicates an action, S i and S i+1 N represents the environmental state. b F represents the number of samples in a batch. i Here, ρ is the Fisher information matrix, and ρ is the regularization coefficient. y i For the loss function parameters, r i Indicates immediate reward, γ represents the discount factor. Indicates the state of environment S i Next, execute action A i Afterwards, the environmental state S i+1 The number of periodic tasks with a service quality of 0 in the ready queue; This represents the target critic network for incremental learning of the m-th task scheduling mode. This represents the corresponding target actor network; This indicates a periodic reward, given when action A is performed. i Afterwards, the environmental state S i+1 There are no expired periodic tasks in the ready queue. The periodic reward is 1, and the immediate reward is r. i Still valid; if action A is performed i Afterwards, the environmental state S i+1 There are still expired periodic tasks in the ready queue, with a periodic reward of 0 and an immediate reward of r. i invalid.

28. A computer device, characterized in that, include: Memory, which stores computer programs; A processor for executing the computer program to implement the operating system adaptive task scheduling method according to any one of claims 1-19.

29. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the operating system adaptive task scheduling method according to any one of claims 1-19.

30. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the operating system adaptive task scheduling method according to any one of claims 1-19.

Images