Two-stage scheduling method and device for data center, equipment and medium

Abstract

The invention discloses a data center-oriented two-stage scheduling method, device, equipment and medium, and belongs to the field of data center task scheduling, and the method comprises the steps: firstly obtaining a feasible starting time slot set through constraint screening, then selecting a starting time slot from the starting time slot set based on a time selection agent trained by a deep Q network, and carrying out the task scheduling of the data center. The method comprises the following steps: starting a starting time slot, determining a task execution window according to the starting time slot, determining a feasible server set based on the task execution window, and then selecting a target server from the feasible server set based on a near-end strategy optimization training opportunity selection agent so as to complete task scheduling; therefore, by implementing the method and the device, the problem that the accuracy of the scheduling result is difficult to improve while the task scheduling efficiency of the data center is guaranteed in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of task scheduling in data centers, and more particularly to a two-stage scheduling method, apparatus, device, and medium for data centers. Background Technology

[0002] In data center task scheduling, it is necessary to reasonably allocate user-submitted computing tasks to physical or virtual computing resources, and optimize the overall operation indicators of the data center while meeting service quality requirements such as task deadlines and resource needs.

[0003] In existing technologies, mainstream scheduling methods, such as traditional heuristic algorithms or early optimization models, mostly focus on optimizing resource utilization or task completion time, while generally ignoring the dynamic changes in electricity costs under time-of-use pricing mechanisms. Furthermore, when facing large-scale, multi-constraint scheduling problems, scheduling methods based on exact solutions or simple rules are computationally complex or lack adaptability, making it difficult to achieve real-time, efficient decision-making in dynamic environments. Because existing methods fail to effectively coordinate the optimization of time and resource dimensions, they suffer from problems such as being unable to significantly reduce electricity costs while ensuring service quality, insufficient scheduling efficiency and adaptability in complex dynamic environments, and difficulty in handling the combinatorial explosion challenges of large-scale problems. Summary of the Invention

[0004] This invention provides a two-stage scheduling method, apparatus, device, and medium for data centers, which can solve the problem in the prior art of improving the accuracy of scheduling results while ensuring the efficiency of task scheduling in data centers.

[0005] In a first aspect, embodiments of the present invention provide a two-stage scheduling method for data centers, comprising: Obtain the task information, currently available resource information, and current time-of-use electricity price information corresponding to the currently scheduled task in the data center; wherein, the task information corresponding to the currently scheduled task includes the task processing time, resource requirements, and deadline; Based on the task information and available resource information corresponding to the current task to be scheduled, exclude time slots that violate timing constraints or resource constraints from each start time slot to obtain a set of feasible start time slots; The current status information is determined based on the task information corresponding to the currently scheduled task, the current available resource information, and the current time-of-use electricity price information. The current status information is then input into the timing agent, which selects from the set of feasible start time slots to obtain the final start time slot. The task execution window is then determined based on the final start time slot. The timing agent is trained based on a first preset deep Q-network. A set of feasible servers is determined based on the task execution window, and a target server is obtained by selecting from the set of feasible servers by the timing agent, and then the current task to be scheduled is scheduled to the target server; wherein, the timing agent is obtained by training based on the near-end policy optimization algorithm.

[0006] This application's embodiments achieve pre-decision screening of the feasible space by excluding time slots that violate constraints, effectively reducing ineffective search and computational overhead. Secondly, a two-stage decision-making mechanism is adopted: first, a timing agent trained on a deep Q-network selects a start time slot from the set of feasible time slots; then, a timing agent trained on near-end policy optimization selects a target server from the set of feasible servers. This decomposes the high-dimensional, complex joint scheduling problem into two relatively independent sub-problems: timing decision-making and resource allocation. This not only significantly reduces the dimensionality of the action space and the learning difficulty but also allows the model to optimize for electricity price sensitivity and resource balance separately, thereby significantly reducing the overall power cost of the data center while ensuring task service quality.

[0007] As a preferred example of the first aspect, the step of excluding time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, to obtain a set of feasible start time slots, includes: Iterate through each start time slot, and obtain the first time based on the start time slot of the current iteration and the task processing time. If the first time is greater than the deadline, exclude the start time slot of the current iteration. After the iteration is completed, form the remaining start time slots into a first start time slot set. Traverse each start time slot in the first start time slot set. If the start time slot being traversed and the currently available resource information do not meet the first resource requirements, then exclude the start time slot being traversed. After the traversal is completed, the remaining start time slots are combined into the feasible start time slot set.

[0008] In this preferred example, through a hierarchical traversal and conditional exclusion mechanism, candidate time slots that meet the timing constraints are first selected based on the task processing time and deadline. Then, further filtering is performed based on resource availability, gradually converging to a set of time slots that simultaneously satisfy both time and resource feasibility. This process effectively avoids unnecessary calculations and evaluations on invalid or infeasible time slots, significantly reduces the size of the decision space, and improves the real-time performance and response efficiency of the scheduling system.

[0009] As a preferred example of the first aspect, the step of inputting the current state information into the timing agent and selecting from the set of feasible start time slots to obtain the final start time slot includes: The current state information is input into the timing agent to perform state-action value evaluation, and the evaluation results corresponding to each start time slot in the set of feasible start time slots are obtained. The final start time slot is determined based on the evaluation results corresponding to each start time slot.

[0010] In this preferred example, by having a timing agent evaluate the state-action value of each candidate time slot in the feasible time slot set and determine the final start time slot based on the evaluation results, this method introduces learning-based intelligent decision-making within a feasible range that strictly satisfies timing and resource constraints. This process enables the system to go beyond simple rule-based selection, comprehensively considering multi-dimensional state information such as time-of-use electricity price fluctuations and resource supply and demand dynamics through a value evaluation model, thereby automatically selecting the time slot that is more conducive to long-term cost optimization from the feasible solutions.

[0011] As a preferred example of the first aspect, determining the task execution window based on the final start time slot includes: The start time slot is determined based on the final start time slot, and the end time slot is determined based on the final start time slot and the task processing duration corresponding to the currently scheduled task; the task execution window is determined based on the start time slot and the end time slot.

[0012] In this preferred example, by clearly defining the start and end time slots of the task based on the determined start time slot and task processing duration, the task execution window is determined. This method ensures that the arrangement of tasks in the time dimension has clear boundaries and predictability.

[0013] As a preferred example of the first aspect, determining the set of feasible servers based on the task execution window includes: Traverse each server in the available resource information. For the currently traversed server, determine whether the available resource quantity of the currently traversed server in all resource dimensions is not less than the resource requirement corresponding to the currently scheduled task in each time slot included in the task execution window. If any resource dimension in any time slot does not meet the resource requirement corresponding to the currently scheduled task, then exclude the server. After the traversal is completed, form the feasible server set with the servers that have not been excluded.

[0014] In this preferred example, by performing continuous consistency checks on all servers across all time slots and resource dimensions throughout the entire task execution window, and strictly excluding servers that cannot meet the requirements at any time or in any resource dimension, this method can construct a completely reliable set of feasible servers. This pre-screening mechanism ensures that subsequent scheduling decisions are based solely on truly feasible candidate targets, thereby completely avoiding the risk of scheduling failures and task interruptions caused by temporary resource shortages or local conflicts.

[0015] As a preferred example of the first aspect, the step of selecting a target server from the set of feasible servers by an opportunistic agent includes: Obtain the status information corresponding to each server in the set of feasible servers; The status information is input into the timing agent to obtain the comprehensive score for each server, and the target server is selected from the set of feasible servers based on the comprehensive score for each server.

[0016] In this preferred example, by utilizing an opportunistic intelligent agent to perform multi-dimensional state perception and comprehensive scoring of each candidate server in the feasible server set, and selecting the target server based on the scoring results, the process achieves intelligent and refined decision-making in server allocation.

[0017] As a preferred example of the first aspect, after scheduling the currently scheduled task to the target server, the method further includes: Based on the execution status of the current scheduled tasks, obtain the immediate reward and the status information for the next round, and store the current status information, the allocation result of the current scheduled tasks, the immediate reward, and the status information for the next round into a preset priority experience replay pool. Based on a preset interval, training samples are obtained from a preset priority experience replay pool, and the timing agent and the opportunity agent are updated according to the training samples.

[0018] In this preferred example, by storing the execution status of task scheduling in a priority experience replay pool and sampling training samples from it at preset intervals to continuously update the timing and opportunity selection agents, the system constructs a closed-loop, self-feedback learning mechanism. This mechanism enables the agent to continuously extract effective information from historical scheduling experience and continuously optimize its decision-making strategy based on actual execution results.

[0019] Secondly, the present invention provides a two-stage scheduling device for data centers, comprising: a data acquisition module, a first processing module, a second processing module, and a scheduling module; The data acquisition module is used to acquire task information, currently available resource information, and current time-of-use electricity price information corresponding to the currently scheduled task in the data center; wherein, the task information corresponding to the currently scheduled task includes task processing time, resource requirements, and deadline. The first processing module is used to exclude time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, so as to obtain a set of feasible start time slots; The second processing module is used to determine the current status information based on the task information, available resource information, and time-of-use electricity price information corresponding to the current task to be scheduled, and input the current status information into the timing agent to select from the set of feasible start time slots to obtain the final start time slot, and then determine the task execution window based on the final start time slot; wherein, the timing agent is obtained by training based on a first preset deep Q network; The scheduling module is used to determine a set of feasible servers based on the task execution window, and select a target server from the set of feasible servers based on the timing agent, and then schedule the current task to be scheduled to the target server; wherein, the timing agent is obtained by training based on the near-end policy optimization algorithm.

[0020] As a preferred example of the second aspect, the step of excluding time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, to obtain a set of feasible start time slots, includes: Iterate through each start time slot, and obtain the first time based on the start time slot of the current iteration and the task processing time. If the first time is greater than the deadline, exclude the start time slot of the current iteration. After the iteration is completed, form the remaining start time slots into a first start time slot set. Traverse each start time slot in the first start time slot set. If the start time slot being traversed and the currently available resource information do not meet the first resource requirements, then exclude the start time slot being traversed. After the traversal is completed, the remaining start time slots are combined into the feasible start time slot set.

[0021] As a preferred example of the second aspect, the step of inputting the current state information into the timing agent and selecting from the set of feasible start time slots to obtain the final start time slot includes: The current state information is input into the timing agent to perform state-action value evaluation, and the evaluation results corresponding to each start time slot in the set of feasible start time slots are obtained. The final start time slot is determined based on the evaluation results corresponding to each start time slot.

[0022] As a preferred example of the second aspect, determining the task execution window based on the final start time slot includes: The start time slot is determined based on the final start time slot, and the end time slot is determined based on the final start time slot and the task processing duration corresponding to the currently scheduled task; the task execution window is determined based on the start time slot and the end time slot.

[0023] As a preferred example of the second aspect, determining the set of feasible servers based on the task execution window includes: Traverse each server in the available resource information. For the currently traversed server, determine whether the available resource quantity of the currently traversed server in all resource dimensions is not less than the resource requirement corresponding to the currently scheduled task in each time slot included in the task execution window. If any resource dimension in any time slot does not meet the resource requirement corresponding to the currently scheduled task, then exclude the server. After the traversal is completed, form the feasible server set with the servers that have not been excluded.

[0024] As a preferred example of the second aspect, the step of selecting a target server from the set of feasible servers by an opportunistic agent includes: Obtain the status information corresponding to each server in the set of feasible servers; The status information is input into the timing agent to obtain the comprehensive score for each server, and the target server is selected from the set of feasible servers based on the comprehensive score for each server.

[0025] As a preferred example of the second aspect, after scheduling the currently scheduled task to the target server, the method further includes: Based on the execution status of the current scheduled tasks, obtain the immediate reward and the status information for the next round, and store the current status information, the allocation result of the current scheduled tasks, the immediate reward, and the status information for the next round into a preset priority experience replay pool. Based on a preset interval, training samples are obtained from a preset priority experience replay pool, and the timing agent and the opportunity agent are updated according to the training samples.

[0026] In summary, the embodiments of this application achieve screening of the feasible space before decision-making by excluding time slots that violate constraints, effectively reducing the overhead of ineffective search and computation. Secondly, a two-stage decision-making mechanism is adopted. First, a timing agent trained based on a deep Q-network selects a start time slot from the set of feasible time slots. Then, a timing agent trained based on near-end policy optimization selects a target server from the set of feasible servers. This decomposes the high-dimensional and complex joint scheduling problem into two relatively independent sub-problems: timing decision-making and resource allocation. This not only significantly reduces the dimensionality of the action space and the learning difficulty, but also enables the model to optimize for electricity price sensitivity and resource balance respectively. Thus, while ensuring the quality of task service, it significantly reduces the overall power cost of the data center.

[0027] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the two-stage scheduling method for data centers as described in the present invention.

[0028] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform steps of the two-stage scheduling method for data centers of the present invention. Attached Figure Description

[0029] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 A flowchart illustrating an embodiment of a two-stage scheduling method for data centers provided by the present invention; Figure 2 This is a module structure diagram of one embodiment of a two-stage scheduling device for data centers provided by the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0033] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0034] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0035] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0036] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0037] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0038] Example 1 See Figure 1 To address the problem in existing technologies of simultaneously ensuring task scheduling efficiency and improving scheduling accuracy in data centers, an embodiment of the present invention provides a two-stage scheduling method for data centers, comprising: S1. Obtain the task information, currently available resource information, and current time-of-use electricity price information corresponding to the currently scheduled task in the data center; wherein, the task information corresponding to the currently scheduled task includes the task processing time, resource requirements, and deadline. S2. Based on the task information and available resource information corresponding to the current task to be scheduled, exclude time slots that violate timing constraints or resource constraints from each start time slot to obtain a set of feasible start time slots. As a preferred implementation, the step of excluding time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, to obtain a set of feasible start time slots, includes: Iterate through each start time slot, and obtain the first time based on the start time slot of the current iteration and the task processing time. If the first time is greater than the deadline, exclude the start time slot of the current iteration. After the iteration is completed, form the remaining start time slots into a first start time slot set. Traverse each start time slot in the first start time slot set. If the start time slot being traversed and the currently available resource information do not meet the first resource requirements, then exclude the start time slot being traversed. After the traversal is completed, the remaining start time slots are combined into the feasible start time slot set.

[0039] It should be noted that, in order to ensure that the timing decision takes into account both electricity price and feasibility, this application divides a day into several discrete time slots, numbered as follows: arrive , of which The time-of-use electricity price corresponding to each time slot is denoted as follows: (Unit electricity price / per time slot); the processing time of the task is recorded as... (Once started, continuous usage is required) (each time slot); the deadline for the task is denoted as... (This indicates that the task must be completed before this time slot). If a combination of a start time slot and processing duration would cause the task's end time slot to exceed the deadline, then that start time slot is temporally infeasible. To avoid selecting a start time slot that meets the deadline but suffers from resource shortages in some time slots, the system estimates the available resources within the future window, denoted as... ,in If it is estimated that there are insufficient resources in any time slot within the window ( If so, the corresponding start time slot is excluded.

[0040] Specifically, the process for determining resource constraints can be as follows: Record the task in the start time slot Start and run continuously If there are multiple time slots, then the time window for the task is a closed interval. Within this window, for each server Each resource examine Are all time slots not less than If any time-related time slot exists With any resource Make Then the server It was deemed infeasible and excluded.

[0041] S3. Determine the current state information based on the task information, available resource information, and time-of-use electricity price information corresponding to the current task to be scheduled, and input the current state information into the timing agent. Select from the set of feasible start time slots to obtain the final start time slot, and then determine the task execution window based on the final start time slot; wherein, the timing agent is obtained by training based on a first preset deep Q network; As a preferred implementation, the step of inputting the current state information into the timing agent and selecting from the set of feasible start time slots to obtain the final start time slot includes: The current state information is input into the timing agent to perform state-action value evaluation, and the evaluation results corresponding to each start time slot in the set of feasible start time slots are obtained. The final start time slot is determined based on the evaluation results corresponding to each start time slot.

[0042] As a preferred embodiment, determining the task execution window based on the final start time slot includes: The start time slot is determined based on the final start time slot, and the end time slot is determined based on the final start time slot and the task processing duration corresponding to the currently scheduled task; the task execution window is determined based on the start time slot and the end time slot.

[0043] It should be noted that in this embodiment, the cost trend is roughly assessed for each candidate start time slot by combining its covered time window and corresponding electricity price. The deep Q network gradually forms a preference for different electricity price levels during training, making it more inclined to schedule task start-up in low electricity price ranges and more restrained in high electricity price ranges.

[0044] S4. Determine a set of feasible servers based on the task execution window, and select a target server from the set of feasible servers based on the timing agent, and then schedule the current task to be scheduled to the target server; wherein, the timing agent is obtained by training based on the near-end policy optimization algorithm.

[0045] As a preferred embodiment, determining the set of feasible servers based on the task execution window includes: Traverse each server in the available resource information. For the currently traversed server, determine whether the available resource quantity of the currently traversed server in all resource dimensions is not less than the resource requirement corresponding to the currently scheduled task in each time slot included in the task execution window. If any resource dimension in any time slot does not meet the resource requirement corresponding to the currently scheduled task, then exclude the server. After the traversal is completed, form the feasible server set with the servers that have not been excluded.

[0046] As a preferred implementation, the step of selecting a target server from the set of feasible servers based on the opportunistic intelligent agent includes: Obtain the status information corresponding to each server in the set of feasible servers; The status information is input into the timing agent to obtain the comprehensive score for each server, and the target server is selected from the set of feasible servers based on the comprehensive score for each server.

[0047] In a preferred embodiment, after scheduling the currently scheduled task to the target server, the method further includes: Based on the execution status of the current scheduled tasks, obtain the immediate reward and the status information for the next round, and store the current status information, the allocation result of the current scheduled tasks, the immediate reward, and the status information for the next round into a preset priority experience replay pool. Based on a preset interval, training samples are obtained from a preset priority experience replay pool, and the timing agent and the opportunity agent are updated according to the training samples.

[0048] Specifically, to ensure server selection balances system equilibrium and stability, this embodiment statistically analyzes the average utilization rate of each server within a window and applies appropriate rewards or penalties based on a preset target utilization rate range. The average utilization rate is denoted as... Target utilization rate .when near Positive incentives are given when things go wrong, and suppression is given when things deviate, thereby avoiding long-term overload or long-term idleness of a single server.

[0049] Record server In the time slot The power consumption is (Power consumption per time slot). When the server is in sleep mode and under no load, power consumption is considered... When the server is running, the power consumption is determined by the static power consumption ( With dynamic power consumption It consists of two parts: Record the total system power consumption before deploying this task as follows: (“ "Before deployment; subscript" "" is an abbreviation for "total," representing the total system power consumption. After deploying this task, the total system power consumption is... The expression is as follows: Within the task's time window, the incremental electricity charge is defined (with a single time slot as the settlement granularity): in, For the mission start time slot, The duration of continuous processing (unit: time slot); if To express power (kW), the above formula should be rewritten as: in, The duration of a single time slot (in hours).

[0050] Specifically, the instantaneous reward expression for the opportunistic agent can be: in, This is a failure indicator (taken as 1 if the server phase has no action to take within the allowed attempts). >0 represents the failure cost coefficient.

[0051] In summary, the embodiments of this application achieve screening of the feasible space before decision-making by excluding time slots that violate constraints, effectively reducing the overhead of ineffective search and computation. Secondly, a two-stage decision-making mechanism is adopted. First, a timing agent trained based on a deep Q-network selects a start time slot from the set of feasible time slots. Then, a timing agent trained based on near-end policy optimization selects a target server from the set of feasible servers. This decomposes the high-dimensional and complex joint scheduling problem into two relatively independent sub-problems: timing decision-making and resource allocation. This not only significantly reduces the dimensionality of the action space and the learning difficulty, but also enables the model to optimize for electricity price sensitivity and resource balance respectively. Thus, while ensuring the quality of task service, it significantly reduces the overall power cost of the data center.

[0052] Example 2 like Figure 2 As shown, based on the above method embodiments, corresponding device embodiments are provided; An embodiment of the present invention provides a two-stage scheduling device for data centers, comprising: a data acquisition module 21, a first processing module 22, a second processing module 23, and a scheduling module 24; Data acquisition module 21 is used to acquire task information, currently available resource information and current time-of-use electricity price information corresponding to the currently scheduled task in the data center; wherein, the task information corresponding to the currently scheduled task includes task processing time, resource requirements and deadline; The first processing module 22 is used to exclude time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the current task to be scheduled and the current available resource information, so as to obtain a set of feasible start time slots. The second processing module 23 is used to determine the current status information based on the task information, available resource information, and time-of-use electricity price information corresponding to the current task to be scheduled, and input the current status information into the timing agent to select from the set of feasible start time slots to obtain the final start time slot, and then determine the task execution window based on the final start time slot; wherein, the timing agent is obtained by training based on a first preset deep Q network; The scheduling module 24 is used to determine a set of feasible servers based on the task execution window, and select a target server from the set of feasible servers based on the timing agent, and then schedule the current task to be scheduled to the target server; wherein the timing agent is obtained by training based on the near-end policy optimization algorithm.

[0053] As a preferred implementation, the step of excluding time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, to obtain a set of feasible start time slots, includes: Iterate through each start time slot, and obtain the first time based on the start time slot of the current iteration and the task processing time. If the first time is greater than the deadline, exclude the start time slot of the current iteration. After the iteration is completed, form the remaining start time slots into a first start time slot set. Traverse each start time slot in the first start time slot set. If the start time slot being traversed and the currently available resource information do not meet the first resource requirements, then exclude the start time slot being traversed. After the traversal is completed, the remaining start time slots are combined into the feasible start time slot set.

[0054] As a preferred implementation, the step of inputting the current state information into the timing agent and selecting from the set of feasible start time slots to obtain the final start time slot includes: The current state information is input into the timing agent to perform state-action value evaluation, and the evaluation results corresponding to each start time slot in the set of feasible start time slots are obtained. The final start time slot is determined based on the evaluation results corresponding to each start time slot.

[0055] As a preferred embodiment, determining the task execution window based on the final start time slot includes: The start time slot is determined based on the final start time slot, and the end time slot is determined based on the final start time slot and the task processing duration corresponding to the currently scheduled task; the task execution window is determined based on the start time slot and the end time slot.

[0056] As a preferred embodiment, determining the set of feasible servers based on the task execution window includes: Traverse each server in the available resource information. For the currently traversed server, determine whether the available resource quantity of the currently traversed server in all resource dimensions is not less than the resource requirement corresponding to the currently scheduled task in each time slot included in the task execution window. If any resource dimension in any time slot does not meet the resource requirement corresponding to the currently scheduled task, then exclude the server. After the traversal is completed, form the feasible server set with the servers that have not been excluded.

[0057] As a preferred implementation, the step of selecting a target server from the set of feasible servers based on the opportunistic intelligent agent includes: Obtain the status information corresponding to each server in the set of feasible servers; The status information is input into the timing agent to obtain the comprehensive score for each server, and the target server is selected from the set of feasible servers based on the comprehensive score for each server.

[0058] In a preferred embodiment, after scheduling the currently scheduled task to the target server, the method further includes: Based on the execution status of the current scheduled tasks, obtain the immediate reward and the status information for the next round, and store the current status information, the allocation result of the current scheduled tasks, the immediate reward, and the status information for the next round into a preset priority experience replay pool. Based on a preset interval, training samples are obtained from a preset priority experience replay pool, and the timing agent and the opportunity agent are updated according to the training samples.

[0059] For more detailed steps and working principles of this embodiment, please refer to the relevant description in Embodiment 1, but not limited to these descriptions.

[0060] In summary, the embodiments of this application achieve screening of the feasible space before decision-making by excluding time slots that violate constraints, effectively reducing the overhead of ineffective search and computation. Secondly, a two-stage decision-making mechanism is adopted. First, a timing agent trained based on a deep Q-network selects a start time slot from the set of feasible time slots. Then, a timing agent trained based on near-end policy optimization selects a target server from the set of feasible servers. This decomposes the high-dimensional and complex joint scheduling problem into two relatively independent sub-problems: timing decision-making and resource allocation. This not only significantly reduces the dimensionality of the action space and the learning difficulty, but also enables the model to optimize for electricity price sensitivity and resource balance respectively. Thus, while ensuring the quality of task service, it significantly reduces the overall power cost of the data center.

[0061] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the two-stage scheduling method for data centers provided by any of the above-described method embodiments of the present invention.

[0062] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0063] Example 3 Based on the above embodiments of the two-stage scheduling method for data centers, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the two-stage scheduling method for data centers according to any embodiment of the present invention.

[0064] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0065] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0066] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0067] Example 4 Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the two-stage scheduling method for data centers described in any of the above-described method embodiments of the present invention.

[0068] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0069] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A two-stage scheduling method for data centers, characterized in that, include: Obtain the task information, currently available resource information, and current time-of-use electricity price information corresponding to the currently scheduled task in the data center; wherein, the task information corresponding to the currently scheduled task includes the task processing time, resource requirements, and deadline; Based on the task information and available resource information corresponding to the current task to be scheduled, exclude time slots that violate timing constraints or resource constraints from each start time slot to obtain a set of feasible start time slots; The current status information is determined based on the task information corresponding to the currently scheduled task, the current available resource information, and the current time-of-use electricity price information. The current status information is then input into the timing agent, which selects from the set of feasible start time slots to obtain the final start time slot. The task execution window is then determined based on the final start time slot. The timing agent is trained based on a first preset deep Q-network. A set of feasible servers is determined based on the task execution window, and a target server is obtained by selecting from the set of feasible servers by the timing agent, and then the current task to be scheduled is scheduled to the target server; wherein, the timing agent is obtained by training based on the near-end policy optimization algorithm.

2. The two-stage scheduling method for data centers as described in claim 1, characterized in that, The step involves excluding time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, to obtain a set of feasible start time slots, including: Iterate through each start time slot, and obtain the first time based on the start time slot of the current iteration and the task processing time. If the first time is greater than the deadline, exclude the start time slot of the current iteration. After the iteration is completed, form the remaining start time slots into a first start time slot set. Traverse each start time slot in the first start time slot set. If the start time slot being traversed and the currently available resource information do not meet the first resource requirements, then exclude the start time slot being traversed. After the traversal is completed, the remaining start time slots are combined into the feasible start time slot set.

3. The two-stage scheduling method for data centers as described in claim 1, characterized in that, The step of inputting the current state information into the timing agent and selecting from the set of feasible start time slots to obtain the final start time slot includes: The current state information is input into the timing agent to perform state-action value evaluation, and the evaluation results corresponding to each start time slot in the set of feasible start time slots are obtained. The final start time slot is determined based on the evaluation results corresponding to each start time slot.

4. The two-stage scheduling method for data centers as described in claim 1, characterized in that, The step of determining the task execution window based on the final start time slot includes: The start time slot is determined based on the final start time slot, and the end time slot is determined based on the final start time slot and the task processing duration corresponding to the currently scheduled task; the task execution window is determined based on the start time slot and the end time slot.

5. A two-stage scheduling method for data centers as described in claim 1, characterized in that, The step of determining the set of feasible servers based on the task execution window includes: Traverse each server in the available resource information. For the currently traversed server, determine whether the available resource quantity of the currently traversed server in all resource dimensions is not less than the resource requirement corresponding to the currently scheduled task in each time slot included in the task execution window. If any resource dimension in any time slot does not meet the resource requirement corresponding to the currently scheduled task, then exclude the server. After the traversal is completed, form the feasible server set with the servers that have not been excluded.

6. The two-stage scheduling method for data centers as described in claim 1, characterized in that, The step of selecting a target server from the set of feasible servers by an opportunistic agent includes: Obtain the status information corresponding to each server in the set of feasible servers; The status information is input into the timing agent to obtain the comprehensive score for each server, and the target server is selected from the set of feasible servers based on the comprehensive score for each server.

7. A two-stage scheduling method for data centers as described in any one of claims 1-6, characterized in that, After scheduling the currently scheduled task to the target server, the process further includes: Based on the execution status of the current scheduled tasks, obtain the immediate reward and the status information for the next round, and store the current status information, the allocation result of the current scheduled tasks, the immediate reward, and the status information for the next round into a preset priority experience replay pool. Based on a preset interval, training samples are obtained from a preset priority experience replay pool, and the timing agent and the opportunity agent are updated according to the training samples.

8. A two-stage scheduling device for data centers, characterized in that, include: The system comprises a data acquisition module, a first processing module, a second processing module, and a scheduling module. The data acquisition module is used to acquire task information, currently available resource information, and current time-of-use electricity price information corresponding to the currently scheduled task in the data center; wherein, the task information corresponding to the currently scheduled task includes task processing time, resource requirements, and deadline. The first processing module is used to exclude time slots that violate timing constraints or resource constraints from each start time slot based on the task information corresponding to the currently scheduled task and the currently available resource information, so as to obtain a set of feasible start time slots; The second processing module is used to determine the current status information based on the task information, available resource information, and time-of-use electricity price information corresponding to the current task to be scheduled, and input the current status information into the timing agent to select from the set of feasible start time slots to obtain the final start time slot, and then determine the task execution window based on the final start time slot; wherein, the timing agent is obtained by training based on a first preset deep Q network; The scheduling module is used to determine a set of feasible servers based on the task execution window, and select a target server from the set of feasible servers based on the timing agent, and then schedule the current task to be scheduled to the target server; wherein, the timing agent is obtained by training based on the near-end policy optimization algorithm.

9. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the two-stage scheduling method for data centers as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the two-stage scheduling method for data centers as described in any one of claims 1-7.

Images