Task scheduling method and system for data center

Abstract

The invention discloses a task scheduling method and system for a data center, and the method comprises the steps: obtaining a plurality of data centers connected with a local data center based on the local data center where a to-be-scheduled task is located, and forming a candidate data center set; based on the detachability of the to-be-scheduled task, taking the minimum value of the sum of the carbon emission consumed by transmitting the to-be-scheduled task to the candidate data center set and the carbon emission consumed by processing the to-be-scheduled task in the candidate data center set as a target, and constructing a target function of a task scheduling model; and solving the task scheduling model by adopting an improved NSGA-II algorithm to obtain a scheduling scheme of the to-be-scheduled task, and scheduling the to-be-scheduled task according to the scheduling scheme. By adopting the embodiment of the invention, carbon emission can be reduced, and the emission reduction effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of task scheduling, in particular to a task scheduling method and system of a data center. BACKGROUND

[0002] At present, in the task scheduling scheme, after obtaining a to-be-scheduled task, the computing resources of the to-be-scheduled task are calculated, a target candidate data center capable of processing the to-be-scheduled task is selected according to the computing resources, and the to-be-scheduled task is sent to the target candidate data center. However, this scheme only focuses on whether the candidate data center can process the to-be-scheduled task, but different candidate data centers have different power sources (for example, thermal power, wind power, solar power, etc.), which makes the carbon emissions consumed by the candidate data centers when processing tasks differ significantly, ultimately resulting in poor emission reduction effect of the task scheduling scheme. SUMMARY

[0003] The present application provides a task scheduling method and system of a data center to solve the problem that the prior art only focuses on whether the candidate data center can process the to-be-scheduled task, and the emission reduction effect of the task scheduling scheme is poor.

[0004] To achieve the above-mentioned purpose, the embodiments of the present application provide a task scheduling method of a data center, comprising: Based on a local data center where a to-be-scheduled task is located, a plurality of data centers connected with the local data center are obtained to form a candidate data center set; Based on the detachability of the to-be-scheduled task, a minimum value of the sum of the carbon emissions consumed by transmitting the to-be-scheduled task to the candidate data center set and the carbon emissions consumed by processing the to-be-scheduled task in the candidate data center set is taken as the target to construct a target function of a task scheduling model; An improved NSGA-II algorithm is used to solve the task scheduling model to obtain a scheduling scheme of the to-be-scheduled task, and the to-be-scheduled task is scheduled according to the scheduling scheme; The improved NSGA-II algorithm includes: generating individual encoding of the discrete layer and the continuous layer by using a double-layer encoding rule, and using different crossover and mutation operations for the discrete layer and the continuous layer; the double-layer encoding rule includes: assigning a path ID to a subtask of the to-be-scheduled task in the discrete layer, and assigning a subtask resource of the to-be-scheduled task to the candidate data center set in the continuous layer.

[0005] As an improvement of the above-mentioned scheme, the improved NSGA-II algorithm includes: Based on the double-layer encoding rule, individual encoding of the discrete layer and the continuous layer is generated; The initial population is generated by mixing a greedy strategy, a random strategy and a perturbation strategy; wherein the perturbation strategy is to add Gaussian perturbation to the individual obtained by the greedy strategy; The individuals in the initial population are subjected to non-dominated sorting and elitist strategy to generate the next generation population; The individuals in the discrete layer of the next generation population are subjected to recombination crossover, single-point mutation and 2-opt mutation, and the individuals in the continuous layer of the initial population are subjected to SBX crossover and Gaussian mutation; The next generation population is taken as a new initial population, and the step of subjecting the individuals in the initial population to non-dominated sorting and elitist strategy to generate the next generation population is returned until a preset iteration stop condition is met.

[0006] As an improvement of the above scheme, after the individuals in the initial population are subjected to non-dominated sorting and elitist strategy to generate the next generation population, before the individuals in the discrete layer of the next generation population are subjected to recombination crossover, single-point mutation and 2-opt mutation, and the individuals in the continuous layer of the initial population are subjected to SBX crossover and Gaussian mutation, the method further comprises: The individual in the next generation population with a congestion degree greater than a preset congestion degree threshold and in the continuous layer is added with a perturbation by combining the starfish algorithm and the jellyfish algorithm, and the size of the perturbation is the size of a step, and the size of the step is exponentially annealed.

[0007] As an improvement of the above scheme, during the process of solving the task scheduling model by using the improved NSGA-II algorithm, the simplex projection is performed on the individual in the continuous layer after the individual in the continuous layer is changed.

[0008] As an improvement of the above scheme, before the task scheduling model is solved by using the improved NSGA-II algorithm, the method further comprises: The constraint conditions of the task scheduling model are constructed; wherein the constraint conditions include: task integrity constraint, resource capacity constraint, total time limit constraint and path feasibility constraint.

[0009] As an improvement of the above scheme, the constraint conditions further include: single machine computing amount threshold constraint, cache capacity constraint and link capacity constraint.

[0010] As an improvement of the above scheme, the scheduling scheme includes a transmission path; The method further comprises: If the transmission path exists a coincident transmission path, the non-local data center in the coincident transmission path is sequentially taken as a first data center; For each of the first data centers, the last non-local data center of the first data center on the coincident transmission path is taken as a second data center; A data staging instruction is sent to the second data center, and the data staging instruction is used to inform the second data center to stage target computing data of the first data center; wherein the target computing data is target computing data required for executing a subtask scheduled to the first data center; In response to a secondary data request sent by the first data center, a data forwarding instruction is sent to the second data center, and the data forwarding instruction is used to instruct the second data center to send target computing data to the first data center.

[0011] As an improvement of the above scheme, the method further comprises: In a case where a data amount of the target computing data is greater than an available storage space of the second data center, a data center connected with the second data center and having a distance less than a preset distance from the second data center is obtained to form a backup data center set; A data amount difference between the target computing data and the available storage space is obtained; One or more target backup data centers are determined in the backup data center set according to the data amount difference; wherein a sum of available storage spaces of the one or more target backup data centers is greater than or equal to the data amount difference; A data transfer instruction is sent to the second data center and the one or more target backup data centers, and the data transfer instruction is used to instruct the second data center to transfer the target computing data to the one or more target backup data centers.

[0012] To achieve the above object, the embodiments of the present application further provide a task scheduling system of a data center, comprising: An obtaining module is configured to obtain a plurality of data centers connected with a local data center where a to-be-scheduled task is located, to form a candidate data center set; A first constructing module is configured to construct a target function of a task scheduling model based on a detachability of the to-be-scheduled task, and taking a minimum value of a sum of carbon emissions consumed by transferring the to-be-scheduled task to the candidate data center set and carbon emissions consumed by processing the to-be-scheduled task in the candidate data center set as a target; A scheduling module is configured to solve the task scheduling model by using an improved NSGA-II algorithm to obtain a scheduling scheme of the to-be-scheduled task, and schedule the to-be-scheduled task according to the scheduling scheme; The improved NSGA-II algorithm comprises the following steps: generating individual coding of the discrete layer and the continuous layer by adopting double-layer coding rules, and different crossover and mutation operations are adopted for the discrete layer and the continuous layer; the double-layer coding rules comprise: assigning a path ID to a subtask of the to-be-scheduled task in the discrete layer, and assigning a subtask resource of the to-be-scheduled task to the candidate data center set in the continuous layer.

[0013] As an improvement of the above scheme, the improved NSGA-II algorithm comprises: generating individual coding of the discrete layer and the continuous layer based on double-layer coding rules; generating an initial population in a mixed manner of a greedy strategy, a random strategy and a perturbation strategy; wherein the perturbation strategy is to add Gaussian perturbation to the individual obtained by the greedy strategy; performing non-dominated sorting and elitist reservation strategy on the individual in the initial population to generate a next generation population; adopting recombination crossover, single-point mutation and 2-opt mutation for the individual in the discrete layer in the next generation population, and adopting SBX crossover and Gaussian mutation for the individual in the continuous layer in the initial population; taking the next generation population as a new initial population, returning to the step of performing non-dominated sorting and elitist reservation strategy on the individual in the initial population to generate a next generation population, until a preset iteration stop condition is met.

[0014] Compared with the prior art, the task scheduling and system of the data center provided in the embodiments of the present application can obtain a plurality of data centers connected with a local data center where a to-be-scheduled task is located to form a candidate data center set based on the to-be-scheduled task; based on the detachability of the to-be-scheduled task, and taking the minimum value of the sum of the carbon emission consumed by transmitting the to-be-scheduled task to the candidate data center set and the carbon emission consumed by processing the to-be-scheduled task in the candidate data center set as the target, a target function of a task scheduling model is constructed; an improved NSGA-II algorithm (non-dominated sorting genetic algorithm II) is used to solve the task scheduling model to obtain a scheduling scheme of the to-be-scheduled task, and the to-be-scheduled task is scheduled according to the scheduling scheme. As can be seen, the embodiments of the present application fully consider the difference between different power sources, not only take into account the carbon emission generated by processing the task, but also take into account the carbon emission consumed by the data transmission process of transmitting the task, so that the accounting of carbon emission is more accurate, and finally the minimum carbon emission is taken as the target to realize the scheduling of the task, so as to reduce carbon emission and improve the emission reduction effect. In addition, the embodiments of the present application adopt the method of "path ID (discrete layer) + subtask resource (continuous layer)", which is fully adapted to multi-path selection and the detachability of the task, can comprehensively describe the scheduling scheme, and improves the diversity of the solution by adopting different crossover and mutation operations for the discrete layer and the continuous layer. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flowchart of a task scheduling method of a data center provided by an embodiment of the present application; Figure 2 is a structural block diagram of a task scheduling system of a data center provided by an embodiment of the present application. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the scope of protection of the present application.

[0017] In the description of the present application, the size of the serial number of each process does not mean the execution order. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0018] In the description of the present application, the term "exemplary" or "for example" is used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the term "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0019] In the description of the application, the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices. The term "based on" is "at least partially based on". The term "according to" is "at least partially according to". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The term "and / or" means at least one of the connected objects, for example, A and / or B means three cases including A alone, B alone, and A and B both. Unless otherwise stated, the term "a plurality of" means two or more.

[0020] Referring to Figure 1 , Figure 1 is a flowchart of a data center task scheduling method provided by an embodiment of the application. The data center task scheduling method comprises: S1, based on a local data center where a to-be-scheduled task is located, obtaining a plurality of data centers connected with the local data center to form a candidate data center set; It is worth noting that the to-be-scheduled task includes at least one of a computing task, a storage task, and a resource management task. Optionally, the to-be-scheduled task can be sent to the local data center by other computer devices, or can be input into the local data center by technical personnel.

[0021] Connection refers to communication connection. Through the connection relationship, a plurality of candidate data centers directly / indirectly connected with the local data center where the to-be-scheduled task is located are obtained to form a candidate data center set. The candidate data centers in the candidate data center set have a connection relationship with the local data center.

[0022] S2, based on the detachability of the to-be-scheduled task, and taking the minimum value of the sum of the carbon emissions consumed by transmitting the to-be-scheduled task to the candidate data center set and the carbon emissions consumed by processing the to-be-scheduled task in the candidate data center set as the target, constructing a target function of a task scheduling model; S3, solve the task scheduling model by using the improved NSGA-II algorithm to obtain a scheduling scheme of the to-be-scheduled task, and schedule the to-be-scheduled task according to the scheduling scheme; wherein the improved NSGA-II algorithm comprises: generating individual encoding of the discrete layer and the continuous layer by using a double-layer encoding rule, and using different crossover and mutation operations for the discrete layer and the continuous layer; the double-layer encoding rule comprises: assigning a path ID to a subtask of the to-be-scheduled task in the discrete layer, and assigning a subtask resource of the to-be-scheduled task to the candidate data center set in the continuous layer.

[0023] It is worth noting that when the improved NSGA-II algorithm is used to solve the task scheduling model, the task scheduling model is decomposed into two solving models: one is to calculate the carbon emissions consumed by transmitting the to-be-scheduled task to the candidate data center set; the other is to calculate the carbon emissions consumed by processing the to-be-scheduled task in the candidate data center set.

[0024] The embodiments of the present application fully consider the differences between different sources of electric energy, not only take into account the carbon emissions generated by processing tasks, but also take into account the carbon emissions consumed by the data transmission process of transmitting tasks, so that the value of carbon emissions is more accurate, and finally the minimum carbon emissions is taken as the target to realize the scheduling of tasks, so as to reduce carbon emissions and improve the effect of emission reduction.

[0025] In addition, the embodiments of the present application use the method of "path ID (discrete layer) + subtask resource (continuous layer)", which is fully adapted to multi-path selection and the detachability of tasks, can comprehensively describe the scheduling scheme, and improves the diversity of solutions by using different crossover and mutation operations for the discrete layer and the continuous layer.

[0026] In an optional embodiment, the improved NSGA-II algorithm comprises: S41, generating individual encoding of the discrete layer and the continuous layer based on the double-layer encoding rule; The embodiments of the present application use a double-layer encoding rule: Discrete layer: assign a path ID to a subtask of the to-be-scheduled task. Specifically, select a path ID in the pre-generated path pool for each subtask. The path pool can be constructed once or periodically.

[0027] Wherein, the path pool can use path cost for K shortest path search (for example, Yen algorithm or Eppstein algorithm), K is 3-5, and here the cost is only used for search heuristic. Then, the obtained shortest path is covered and de-duplicated, and paths with different prefixes / edge sets are preferentially retained to increase diversity.

[0028] Continuous layer: assign a subtask resource of the to-be-scheduled task to the candidate data center set . represents the data amount proportion of the subtask allocated to the jth candidate data center in the gen th iteration.

[0029] The data amount proportions of the subtasks allocated to each candidate data center in the gen th iteration form a resource vector , where the superscript T represents transposition, I represents the total number of candidate data centers, and the vector is simplex-projected so that the vector satisfies

[0030] S42, an initial population is generated in a mixed manner of a greedy strategy, a random strategy, and a perturbation strategy; the perturbation strategy is to add Gaussian perturbation to an individual obtained by the greedy strategy; Embodiments of the present application use a mixed initial population of three types of individuals to improve the proportion of early feasible solutions and take into account diversity: (i) The greedy strategy: allocation according to the principle of minimum path cost. This can avoid wasting computing resources due to a large number of infeasible solutions at the beginning of the algorithm and accelerate the convergence speed of the algorithm. The path cost is determined by the carbon emissions consumed by the path.

[0031] (ii) The random strategy: each subtask is sampled from Dirichlet(1) and a path ID is randomly selected; Dirichlet(1) refers to Dirichlet distribution with a parameter of 1. This can increase the diversity of the initial population.

[0032] (iii) The perturbation strategy: Gaussian perturbation is applied on the basis of (i). This not only ensures the quality of the solution, but also introduces new solution characteristics through perturbation, balancing the quality and exploration ability of the initial population.

[0033] S43, non-dominated sorting and elitist reservation strategies are performed on the individuals in the initial population to generate a next-generation population; Specifically, the solution set is non-dominantly sorted based on the rule of feasibility priority, and the diversity is maintained by crowding degree. When complete parallelism occurs, the hypervolume index (HV) is used as a sequence breaking criterion (only effective when parallelism occurs, and does not enter the regular sorting), and the solution with a larger hypervolume index is preferentially selected.

[0034] The elitist reservation strategy only saves feasible solutions; if the preset population capacity is exceeded, the solution with the smallest hypervolume index is preferentially removed.

[0035] S44, recombination crossover, single-point mutation, and 2-opt mutation are used for the individuals in the discrete layer in the next-generation population, and SBX crossover and Gaussian mutation are used for the individuals in the continuous layer in the initial population. ​​​​In the embodiments of the present application, the paths of the discrete layer are modified by recombination crossover, single-point mutation and 2-opt mutation to improve the diversity of solutions. The resource of the subtask of the continuous layer is adjusted by Gaussian mutation to ensure that the resource is within a feasible range.

[0036] Specifically, the recombination crossover: preferentially inherits the common prefix / edge set of the parent; and inserts a feasible edge to the conflict position according to the minimum incremental path cost. The single-point mutation: replaces the path ID of a certain subtask with another path in the path pool according to a first probability of the current iteration, for example, when the first probability of the current iteration is greater than a preset single-point mutation probability, the single-point mutation is performed. The 2-opt (2-optimization) mutation: performs a single edge exchange on the selected path according to a second probability of the current iteration to shorten the path or bypass a hot link while maintaining connectivity and feasibility. For example, when the second probability of the current iteration is greater than a preset 2-opt mutation probability, the 2-opt mutation is performed.

[0037] The SBX crossover is a simulation of binary crossover. The Gaussian mutation: applies independent and identically distributed noise to the individual after SBX crossover.

[0038] S45, returns to the step of performing non-dominated sorting and elitist retention strategy on the individuals in the initial population to generate a next generation population with the next generation population as a new initial population, until a preset iteration stop condition is met.

[0039] In the embodiments of the present application, the next generation population (i.e. offspring) is taken as a new initial population (and parent), and the step S43 is re-executed until a preset iteration stop condition is met, and the iteration is stopped, and a scheduling scheme is obtained.

[0040] In an optional embodiment, after the step of performing non-dominated sorting and elitist retention strategy on the individuals in the initial population to generate a next generation population, before the step of using recombination crossover, single-point mutation and 2-opt mutation on the individuals of the discrete layer in the next generation population, and using SBX crossover and Gaussian mutation on the individuals of the continuous layer in the initial population, the method further comprises: In combination with the starfish algorithm and the jellyfish algorithm, a disturbance is added to the individual of the continuous layer with a crowdedness greater than a preset crowdedness threshold in the next generation population, and the size of the disturbance is the size of a step, and the size of the step is exponentially annealed.

[0041] In the embodiments of the present application, a disturbance is added to the individual of the continuous layer with a crowdedness greater than a preset crowdedness threshold in the next generation population, for example, a disturbance is added to the first 20% individuals with a larger crowdedness in the offspring in the continuous layer: ; in the formula, assigns the first gen iteration after adding the disturbance to the first a data amount proportion of a subtask of a candidate data center, allocating to the first gen iteration a data amount proportion of a subtask of a candidate data center, a step length of the first gen iteration, a zero-mean random direction (a spherical surface symmetric Gaussian or Levy flight step length) is taken; The size of the step length is exponentially annealed: ; wherein, is a preset maximum step length, is a natural exponential function, is a preset constant, gen represents a current iteration number (also a population algebra), N gen represents a total iteration number.

[0042] Embodiments of the present application significantly improve the feasibility and convergence performance of the solution by locally strengthening and optimizing the continuous layer through perturbation.

[0043] In an optional embodiment, in the process of solving the task scheduling model by using the improved NSGA-II algorithm, when the individual of the continuous layer changes, the individual of the changed continuous layer is subjected to simplex projection.

[0044] In order to ensure that the solution of the continuous layer is feasible, when the individual of the continuous layer changes, the individual of the changed continuous layer is subjected to simplex projection, so that the resource of the subtask satisfies , For example: In the process of generating the initial population in a mixed manner of the greedy strategy, the random strategy and the perturbation strategy, when the initial population is obtained by using the greedy strategy and the perturbation strategy, the individual of the continuous layer in the initial population is subjected to simplex projection.

[0045] After the SBX crossover is used, the individual subjected to the SBX crossover is subjected to simplex projection.

[0046] After the Gaussian mutation is used, the individual subjected to the Gaussian mutation is subjected to simplex projection.

[0047] In an optional embodiment, the preset iteration stopping condition includes any one of the following: The total hypervolume change amount of a preset number of iterations is less than a preset hypervolume change amount threshold; The total reverse generation distance change amount of a preset number of iterations is less than a preset reverse generation distance change amount threshold.

[0048] Specifically, the preset iteration stopping condition is: In the formula, gen is the current iteration number, and HV is... gen-j+1 HV represents the hypervolume in the (gen-j+1)th iteration. gen-j Represents the hypervolume in the gen-j-th iteration; IGD gen-j+1 Represents the reverse generation distance in the (gen-j+1)th iteration, IGD gen-j This represents the reverse generation distance in the gen-j-th iteration.

[0049] In an optional embodiment, the method further includes: During the solution process, constraints are handled according to the Deb constraint rule and the ε-constraint method.

[0050] Deb constraint rule: When selecting and ranking, "feasibility takes precedence," meaning any feasible solution is preferred over an infeasible one; if neither is feasible, then the total violation is compared. . This represents the degree of violation of the z-th type of constraint. This indicates the total number of constraint categories. For example, constraint categories include: Resource Capacity Constraint: A violation occurs if the load allocated to a data center exceeds its available capacity; Time Limit Constraint: A violation occurs if a subtask cannot be completed within a given deadline; Path Feasibility Constraint: A violation occurs if the scheduling path exceeds the maximum transmission distance or the link bandwidth is insufficient to handle the allocated data volume. Each constraint category corresponds to one... Its value reflects the degree of violation. This represents the violation threshold for the z-th type of constraint.

[0051] - Constraint method: Set a dynamic violation threshold for each type of constraint. It is gradually tightened every 20 generations for order comparison during sorting (without mixing with the target). ,0< <1. This indicates the minimum level of violation. This indicates the preset value.

[0052] Furthermore, during the solution process, constraints are handled according to the repairer, in three steps: Link bandwidth: Calculate the overload of each link, prioritizing those that result in the highest "bandwidth usage × carbon intensity". Yes, migrate to a path with lower carbon emissions for the same task; among which, Indicates candidate data centers, This represents a subtask of the task to be scheduled; or in a successive layer... Migrate to candidate data centers with greater available resources. Indicates allocation to the first The first candidate data center The percentage of data volume for each sub-task.

[0053] Compute / Storage: If candidate data centers If the limit is exceeded, then sort the items column by column according to path cost from highest to lowest. They moved to candidate data centers with larger available resources.

[0054] Time limit: If a single task times out, it will be replaced with a faster path first; if this is still not satisfied, the resource vector will be projected with weights enhanced towards the "fast center".

[0055] The embodiments of this application employ the above-described constraint processing to ensure that the individual is always within the feasible region, significantly improving the feasibility and convergence performance of the solution.

[0056] In an optional embodiment, the objective function is expressed as: in, Represents the set of candidate data centers. Represents the first in the candidate data center set One candidate data center This represents the set of all subtasks in the task to be scheduled. This indicates a subtask assigned to a candidate data center; Indicates the first Each subtask is transferred from the local data center to the... The electrical energy consumed by each candidate data center ; This indicates the amount of data for the task to be scheduled. Indicates allocation to the first The first candidate data center The percentage of data volume for each sub-task Indicates the local data center to the The transmission distance of each candidate data center This represents the energy consumption per unit of data volume per unit distance. Indicates the first Network energy efficiency factor of candidate data centers; Indicates the first The candidate data center processing The electrical energy consumed by each sub-task ; Indicates the first Power usage efficiency of candidate data centers Indicates the first The computing density of each candidate data center; Indicates the first Energy efficiency of IT equipment in candidate data centers; express Time of the first Carbon intensity of candidate data centers ; This refers to the first term used in calculations or transmissions. Energy-like Indicates the first Carbon intensity of energy sources express Time of the first The first candidate data center consumed the first The proportion of energy sources.

[0057] Specifically, , Assigned to the The first candidate data center The computational cost of each subtask; Indicates the first Effective computational cost of candidate data centers / number of candidate data centers Power consumption of IT equipment in candidate data centers, effective computing power: for the first Benchmark tests were conducted on server clusters in candidate data centers, and the total number of floating-point operations performed during the test was recorded. IT equipment power consumption: Only the energy consumption of IT equipment (excluding cooling, lighting, etc.) was measured, and smart meters were used to record power consumption during the test.

[0058] Indicates the first The network energy efficiency factors for the candidate data centers are as follows: from the local data center to the... Theoretical minimum transmission power consumption of candidate data centers / local data center to the first The actual transmission energy consumption of each candidate data center. Actual transmission energy consumption refers to the energy consumption of network devices (routers, switches, etc.) when transmitting data (measuring the energy consumption of routers / switches when transmitting data). The theoretical minimum transmission energy consumption is obtained based on the Shannon limit. ; Indicates the local data center to the Minimum data transfer volume for each candidate data center Indicates the local data center to the The transmission distance of each candidate data center; Denotes the Boltzmann constant (taken as...) ), This indicates absolute temperature (usually taken as 300K). This indicates the coding efficiency of the candidate data center (usually taken as 0.5~1).

[0059] In an optional embodiment, before solving the task scheduling model using the improved NSGA-II algorithm, the method further includes: The constraints for constructing the task scheduling model include: task integrity constraints, resource capacity constraints, total time constraints, and path feasibility constraints. Specifically, task integrity constraints: ; To be allocated to the The first candidate data center The proportion of data volume for each sub-task; , Indicates allocation to the first The first candidate data center The amount of data for each subtask This represents the amount of data for a task to be scheduled. ; Resource capacity constraints: ; For the first Candidate data centers in Available storage of time, The time is the time when the task begins processing; : Indicates candidate data center Whether it is used (1 if selected, 0 otherwise). First candidate data center mode (does not split the scheduling task mode): Second candidate data center mode (split task scheduling mode): , .

[0060] Total time limit constraint: ; Indicates the first The sub-tasks in the first The processing time for each candidate data center ; , indicating the first Do individual subtasks use links? , Indicates link exist The bandwidth at any given time can be provided through real-time measurement or directly via the configuration interface. Indicates link The time delay, Indicates the first CPU frequencies of the candidate data centers Indicates allocation to the first The first candidate data center The computational cost of each subtask This is the cutoff time. The link refers to the transmission path.

[0061] Path feasibility constraints: ; Indicates the local center to the number The transmission distance of each candidate data center This represents the maximum path length. Q represents an infinite number, only when... Activated at that time.

[0062] In one optional embodiment, the constraints further include: a single-machine computation threshold constraint, a cache capacity constraint, and a link capacity constraint.

[0063] Specifically, the threshold constraint for single-machine computational load: , Indicates the first Minimum computational cost for each candidate data center Indicates the first Maximum computing power of each candidate data center; Cache capacity constraints: , This indicates that after the task is assigned, the first... The amount of data that a candidate data center needs to cache cannot exceed its local cache capacity. ; Link capacity constraints: , Indicates the length of a scheduling period. Indicates allocation to the first The first candidate data center The amount of data for each subtask , indicating the first Do individual subtasks use links? , Indicates link exist Bandwidth at any given moment.

[0064] In an optional embodiment, the method further includes: If there are overlapping transmission paths, then the non-local data centers in the overlapping transmission paths will be successively designated as the first data center. For each of the first data centers, the non-local data center preceding the first data center on the overlapping transmission path is designated as the second data center. A data storage instruction is sent to the second data center, which is used to notify the second data center to temporarily store the target computing data of the first data center; wherein, the target computing data is the target computing data required to execute the sub-task scheduled to the first data center; In response to a secondary data request sent by the first data center, a data forwarding instruction is sent to the second data center, the data forwarding instruction being used to instruct the second data center to send the target computing data to the first data center.

[0065] It is worth noting that during actual data transmission, data is transmitted along a transmission path. If the data required by one of the candidate data centers is lost, the local data center needs to retransmit it along the original transmission path, which consumes significant resources. Therefore, this application provides a data temporary storage method. During task scheduling, data is temporarily stored in a second data center. When the first data center needs to retrieve the data again due to various unforeseen circumstances, the data is forwarded through the second data center, shortening the data transmission path and reducing the energy consumption associated with data transmission.

[0066] For example, by solving the objective function, the local center is determined to transfer tasks to candidate data centers B and C. Specifically, the transmission path from local data center A to B is: A→O→P→Q→R→B; the transmission path from local data center A to C is: A→O→P→Q→R→C; where O, P, Q, and R are candidate data centers along the path, and they also belong to the candidate data center set.

[0067] There exists an overlapping transmission path: A→O→P→Q→R. The non-local data centers in this overlapping transmission path are sequentially designated as the first data center, and the non-local data center preceding the first data center in the overlapping transmission path is designated as the second data center. Non-local data centers are candidate data centers excluding the local data center.

[0068] For example, if R is designated as the first data center, then Q is the second data center. If Q is designated as the first data center, then P is the second data center. If P is designated as the first data center, then O is the second data center. The first and second data centers are relative concepts. Candidate data centers in overlapping transmission paths can temporarily store the data needed by the next candidate data center.

[0069] The data storage instruction is sent by the local data center before scheduling, or the data storage instruction is sent along with the data that needs to be sent during scheduling.

[0070] For example, since data sent from the local data center needs to be transmitted along a transmission path, when the data is transmitted to the second data center, the second data center will temporarily store a copy of the target computing data from the first data center. The second data center can store the target computing data in its own database.

[0071] Furthermore, after receiving the target computing data from the first data center, the second data center can detect the completeness of the target computing data from the first data center. If the target computing data is complete, a temporary storage operation is performed. If the target computing data is incomplete, a secondary data request is sent to the local data center. This secondary data request is used to request the second data center to resend the target computing data.

[0072] Furthermore, after completing the sub-task in the first data center, it sends a task completion notification to the local data center. Upon receiving the notification, the local data center sends a data deletion command to the second data center. The second data center then deletes the target computational data from the first data center according to the deletion command.

[0073] A secondary data request is used by the first data center to request target computation data from the local data center. For example, if the first data center detects that the target computation data is missing, it sends a secondary data request to the local data center. Alternatively, if the first data center detects that the target computation data is incorrect, it sends a secondary data request to the local data center.

[0074] By adopting the above embodiments, when sending data to candidate data centers, the target computing data of the first data center can be temporarily stored in the second data center. When the first data center needs to reacquire the target computing data due to various unforeseen circumstances, the second data center can send the target computing data to the first data center, thereby shortening the transmission path of the target computing data and reducing the energy consumption caused by data transmission.

[0075] In an optional embodiment, the method further includes: If the amount of data in the target computation data is greater than the available storage space of the second data center, then data centers connected to the second data center and located at a distance less than a preset distance from the second data center are acquired to form a set of backup data centers; Obtain the data volume difference between the target computation data and the available storage space; One or more target backup data centers are determined within the set of backup data centers based on the data volume difference; wherein the total available storage space of the one or more target backup data centers is greater than or equal to the data volume difference; Send a data transfer instruction to the second data center and the one or more target backup data centers, the data transfer instruction being used to instruct the second data center to transfer the target computing data to the one or more target backup data centers.

[0076] It is worth noting that the available storage space in the second data center may not be sufficient to hold the target computing data from the first data center. In this case, the second data center needs to transfer some of the target computing data. Therefore, this application provides a data transfer method that selects one or more target backup data centers to forward the target computing data that cannot be temporarily stored in the second data center to one or more target backup data centers for temporary storage, thereby ensuring data integrity.

[0077] For example, based on the time limit requirements of the task to be scheduled, the changes in available storage space in the second data center during the processable time period are statistically analyzed. If the available storage space in the second data center is consistently greater than the amount of target computation data during the processable time period, then the second data center is considered to be able to temporarily store the target computation data without transferring it. If, during the processable time period, the available storage space at the target moment is less than the amount of target computation data, then the second data center needs to transfer a portion of the target computation data to other candidate data centers.

[0078] Furthermore, the total duration of the target time within the processable time period is calculated. If the total duration is less than a preset duration threshold, the minimum available storage space in the second data center within the processable time period is taken. The difference between the minimum value and the computational amount of the target data is calculated to obtain the data volume difference. The data volume difference represents the amount of data that needs to be transferred from the second data center.

[0079] The system determines whether any candidate backup data center in the backup data center set has available storage space greater than the data volume difference. The calculation method for the available storage space of a backup data center is the same as that for the second data center, and will not be repeated here. If such a candidate data center exists, it is designated as one or more target backup data centers. If not, the backup data center set is sorted in descending order based on the minimum available storage space within the processable time period. In this sorted set, the top k candidate data centers are selected to form k target backup data centers, such that the sum of the minimum available storage space of these k target backup data centers within the processable time period is greater than the data volume difference. It should be noted that the descending order is based on the minimum available storage space of the backup data centers throughout the entire processable time period, ensuring that the available storage space of these k target backup data centers is always greater than the data volume difference at any given time within the processable time period.

[0080] The data transfer instruction includes the network addresses or numbers of the second data center and the target backup data center. Furthermore, the data transfer instruction also includes the data volume difference and the data reception capacity of the target backup data center. After receiving the data transfer instruction, the second data center can split the target computing data according to the instruction, so that the split target computing data includes first target computing data and second target computing data. The first target computing data is used by the second data center, and its data volume is equal to the available storage space of the second data center. The second target computing data is used by the target backup data center, and the sum of the data volumes of the second target computing data is the data volume difference. The second target computing data corresponds one-to-one with the target backup data center. The second data center retains the first target computing data and sends the second target computing data to their respective target backup data centers.

[0081] By adopting the above embodiments, when the amount of target computing data is greater than the available storage space of the second data center, a target backup data center can be further determined, and the target computing data can be transferred to the target backup data center to ensure data integrity.

[0082] See Figure 2 , Figure 2 This is a structural block diagram of a data center task scheduling system provided in an embodiment of this application. The data center task scheduling system includes: The acquisition module 11 is used to acquire multiple data centers connected to the local data center where the task to be scheduled is located, and form a candidate data center set; The first construction module 12 is used to construct an objective function for the task scheduling model based on the detachability of the task to be scheduled and with the goal of minimizing the sum of the carbon emissions consumed in transmitting the task to be scheduled to the candidate data center set and the carbon emissions consumed in processing the task to be scheduled in the candidate data center set. The scheduling module 13 is used to solve the task scheduling model using the improved NSGA-II algorithm to obtain the scheduling scheme of the task to be scheduled, and to schedule the task to be scheduled according to the scheduling scheme.

[0083] Optionally, the improved NSGA-II algorithm includes: Based on the two-layer coding rule, individual codes for discrete and continuous layers are generated; An initial population is generated using a combination of greedy, random, and perturbation strategies; wherein the perturbation strategy involves adding a Gaussian perturbation to the individuals obtained by the greedy strategy. The individuals in the initial population are subjected to non-dominated ranking and elite retention strategies to generate the next generation population; Individuals in the discrete layer of the next generation population are subjected to recombination crossover, single-point mutation, and 2-opt mutation, while individuals in the continuous layer of the initial population are subjected to SBX crossover and Gaussian mutation. The next generation population is used as a new initial population, and the process returns to the step of performing non-dominated sorting and elite retention strategies on the individuals in the initial population to generate the next generation population, until the preset iteration stopping condition is met.

[0084] Optionally, after performing non-dominated sorting and elite retention strategies on the individuals in the initial population to generate the next generation population, and before applying recombination crossover, single-point mutation, and 2-opt mutation to the individuals in the discrete layer of the next generation population, and applying SBX crossover and Gaussian mutation to the individuals in the continuous layer of the initial population, the method further includes: The starfish algorithm and jellyfish algorithm are combined to add perturbations to individuals in the next generation population whose crowding degree is greater than a preset crowding degree threshold and which are in continuous layers. The size of the perturbation is the size of the step size, which is annealed exponentially.

[0085] Optionally, during the process of solving the task scheduling model using the improved NSGA-II algorithm, when individuals in the continuous layer change, simplex projection is performed on the individuals in the changed continuous layer.

[0086] Optionally, the task scheduling system of the data center further includes: The second construction module is used to construct the constraints of the task scheduling model: wherein the constraints include: task integrity constraints, resource capacity constraints, total time constraints, and path feasibility constraints. Optionally, the constraints may also include: single-machine computation threshold constraints, cache capacity constraints, and link capacity constraints.

[0087] Optionally, the system further includes: a first backup module, used for: If there are overlapping transmission paths, then the non-local data centers in the overlapping transmission paths will be successively designated as the first data center. For each of the first data centers, the non-local data center preceding the first data center on the overlapping transmission path is designated as the second data center. A data storage instruction is sent to the second data center, which is used to notify the second data center to temporarily store the target computing data of the first data center; wherein, the target computing data is the target computing data required to execute the sub-task scheduled to the first data center; In response to a secondary data request sent by the first data center, a data forwarding instruction is sent to the second data center, which instructs the second data center to send the target computing data to the first data center.

[0088] Optionally, the system further includes: a second backup module, used for: If the amount of data in the target computation data is greater than the available storage space of the second data center, then data centers connected to the second data center and located at a distance less than a preset distance from the second data center are acquired to form a set of backup data centers; Obtain the data volume difference between the target computation data and the available storage space; One or more target backup data centers are determined within the set of backup data centers based on the data volume difference; wherein the total available storage space of the one or more target backup data centers is greater than or equal to the data volume difference; Send a data transfer instruction to the second data center and the one or more target backup data centers, the data transfer instruction being used to instruct the second data center to transfer the target computing data to the one or more target backup data centers.

[0089] It is worth noting that the working process of each module in the data center task scheduling system 10 described in the embodiments of the present invention can refer to the working process of the data center task scheduling method described in the above embodiments and achieve the same beneficial effects, and will not be repeated here.

[0090] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the data center task scheduling method as described in any of the above embodiments.

[0091] Furthermore, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the data center task scheduling method as described in any of the above embodiments.

[0092] Furthermore, this application provides a data center task scheduling device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above-described data center task scheduling method embodiments. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.

[0093] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the task scheduling device of the data center.

[0094] The task scheduling device of the data center may include, but is not limited to, processors and memory. Those skilled in the art will understand that the schematic diagram is merely an example of a task scheduling device for a data center and does not constitute a limitation on the task scheduling device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the task scheduling device of the data center may also include input / output devices, network access devices, buses, etc.

[0095] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor serves as the control center of the data center's task scheduling equipment, connecting various parts of the data center's task scheduling equipment via various interfaces and lines.

[0096] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the task scheduling device in the data center by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0097] If the modules / units integrated into the task scheduling equipment of the data center are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0098] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0099] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A method for task scheduling of a data center, the method comprising: The method comprises the following steps: obtaining a candidate data center set based on a local data center where a to-be-scheduled task is located and a plurality of data centers connected to the local data center; constructing a target function of a task scheduling model based on detachability of the to-be-scheduled task and a minimum value of a sum of carbon emissions consumed by transmitting the to-be-scheduled task to the candidate data center set and carbon emissions consumed by processing the to-be-scheduled task in the candidate data center set; solving the task scheduling model by using an improved NSGA-II algorithm to obtain a scheduling scheme of the to-be-scheduled task, and scheduling the to-be-scheduled task according to the scheduling scheme; wherein the improved NSGA-II algorithm comprises: generating individual encoding of the discrete layer and the continuous layer by using a double-layer encoding rule, and using different crossover and mutation operations for the discrete layer and the continuous layer; the double-layer encoding rule comprises: assigning a path ID to a subtask of the to-be-scheduled task in the discrete layer, and assigning a subtask resource of the to-be-scheduled task to the candidate data center set in the continuous layer.

2. The method of claim 1, wherein, The improved NSGA-II algorithm comprises: generating individual encoding of the discrete layer and the continuous layer based on a double-layer encoding rule; generating an initial population in a mixed manner of a greedy strategy, a random strategy and a perturbation strategy; wherein the perturbation strategy is to add Gaussian perturbation to the individual obtained by the greedy strategy; performing non-dominated sorting and elitist reservation strategy on the individuals in the initial population to generate a next generation population; applying recombination crossover, single-point mutation and 2-opt mutation to the individuals of the discrete layer in the next generation population, and applying SBX crossover and Gaussian mutation to the individuals of the continuous layer in the initial population; taking the next generation population as a new initial population, returning to the step of performing non-dominated sorting and elitist reservation strategy on the individuals in the initial population to generate a next generation population, and repeating until a preset iteration stop condition is met.

3. The method of claim 2, wherein, After the step of performing non-dominated sorting and elitist reservation strategy on the individuals in the initial population to generate a next generation population, and before the step of applying recombination crossover, single-point mutation and 2-opt mutation to the individuals of the discrete layer in the next generation population, and applying SBX crossover and Gaussian mutation to the individuals of the continuous layer in the initial population, the method further comprises: adding a perturbation to the individual of the continuous layer in the next generation population with a crowding degree greater than a preset crowding degree threshold by combining a sea star algorithm and a jellyfish algorithm, and the size of the perturbation is the size of a step, and the size of the step is exponentially annealed.

4. The method of claim 1, wherein, During the process of solving the task scheduling model by using the improved NSGA-II algorithm, simplex projection is performed on the individual of the continuous layer after the individual of the continuous layer is changed.

5. The method of claim 1, wherein, Before the step of solving the task scheduling model by using the improved NSGA-II algorithm, the method further comprises: constructing a constraint condition of the task scheduling model; wherein the constraint condition comprises: task integrity constraint, resource capacity constraint, total time limit constraint and path feasibility constraint.

6. The method of claim 5, wherein, The constraint condition further comprises: single-machine computing amount threshold constraint, cache capacity constraint and link capacity constraint.

7. The method of claim 1, wherein, The scheduling scheme includes a transmission path; The method further includes: If there is a coincident transmission path in the transmission path, sequentially taking a non-local data center in the coincident transmission path as a first data center; For each first data center, taking a last non-local data center of the first data center in the coincident transmission path as a second data center; Sending a data temporary storage instruction to the second data center, the data temporary storage instruction being used to inform the second data center to temporarily store target computing data of the first data center; wherein the target computing data is target computing data required for executing a subtask scheduled to the first data center; In response to a secondary data request sent by the first data center, sending a data forwarding instruction to the second data center, the data forwarding instruction being used to instruct the second data center to send the target computing data to the first data center.

8. The method of claim 7, wherein, The method further includes: In a case where a data amount of the target computing data is greater than an available storage space of the second data center, obtaining data centers connected with the second data center and having a distance less than a preset distance from the second data center, to form a backup data center set; Obtaining a data amount difference between the target computing data and the available storage space; Determining one or more target backup data centers in the backup data center set according to the data amount difference; wherein a sum of available storage spaces of the one or more target backup data centers is greater than or equal to the data amount difference; Sending a data transfer instruction to the second data center and the one or more target backup data centers, the data transfer instruction being used to instruct the second data center to transfer the target computing data to the one or more target backup data centers.

9. A task scheduling system of a data center, characterized by, It includes: An obtaining module is configured to obtain, based on a local data center where a to-be-scheduled task is located, a plurality of data centers connected with the local data center, to form a candidate data center set; A first constructing module is configured to, based on detachability of the to-be-scheduled task, construct a target function of a task scheduling model, with a minimum value of a sum of carbon emission consumed by transmitting the to-be-scheduled task to the candidate data center set and carbon emission consumed by processing the to-be-scheduled task in the candidate data center set as a target; A scheduling module is configured to solve the task scheduling model by using an improved NSGA-II algorithm, to obtain a scheduling scheme of the to-be-scheduled task, and to schedule the to-be-scheduled task according to the scheduling scheme; The improved NSGA-II algorithm includes: generating individual encoding of a discrete layer and a continuous layer by using a double-layer encoding rule, and using different crossover and mutation operations for the discrete layer and the continuous layer; the double-layer encoding rule includes: allocating a path ID of a subtask of the to-be-scheduled task to the discrete layer, and allocating a subtask resource of the to-be-scheduled task to the continuous layer.

10. The task scheduling system of a data center of claim 9, wherein, The improved NSGA-II algorithm includes: Generating individual encoding of a discrete layer and a continuous layer based on a double-layer encoding rule; The initial population is generated by mixing a greedy strategy, a random strategy and a perturbation strategy; the perturbation strategy is to add Gaussian perturbation on the individual obtained by the greedy strategy; The individuals in the initial population are subjected to non-dominated sorting and elitist strategy to generate the next generation population; The individuals in the discrete layer of the next generation population are subjected to recombination crossover, single-point mutation and 2-opt mutation, and the individuals in the continuous layer of the initial population are subjected to SBX crossover and Gaussian mutation; The next generation population is taken as a new initial population, and the step of subjecting the individuals in the initial population to non-dominated sorting and elitist strategy to generate the next generation population is returned until a preset iteration stop condition is met.

Images