Cross-cluster job scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a cross-cluster job scheduling method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of job scheduling. The scheduling method comprises the steps that a scheduling request of a target job is received, and the scheduling request comprises a scheduling strategy and resource requirements corresponding to a plurality of subtasks, the scheduling strategies comprise one or a combination of a load factor scheduling strategy, a network factor scheduling strategy, a cost factor scheduling strategy, a data factor scheduling strategy, a hardware affinity factor scheduling strategy and a position factor scheduling strategy; determining a target cluster corresponding to the plurality of subtasks according to the scheduling strategy and the resource demand; and scheduling the plurality of sub-tasks to the corresponding target clusters. The problem that diversified scheduling requirements and dynamic load changes are difficult to adapt in the prior art is solved at least. The method is suitable for job scheduling scenes.

Description

Technical Field

[0001] This invention relates to the field of job scheduling technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for scheduling cross-cluster jobs. Background Technology

[0002] With the rapid development of cloud computing, edge computing, and hybrid cloud architectures, modern computing environments exhibit the following characteristics: 1) Computing infrastructure has evolved from traditional homogeneous x86 clusters to heterogeneous environments encompassing various computing units such as ARM (Advanced RISC Machines), GPUs (Graphics Processing Units), TPUs (Tensor Processing Units), and FPGAs (Field Programmable Gate Arrays). 2) Enterprise computing workloads are typically distributed across clusters in multiple geographical locations, including public clouds, private clouds, and edge nodes. 3) Different business scenarios, such as AI (Artificial Intelligence) training, big data analytics, and real-time computing, place differentiated demands on resource scheduling. Multi-cluster management has become a development trend for computing power platforms. After a job is created on a computing power platform, the standard three elements of a job only include the job execution instructions, job data (scripts, datasets), and job configuration, without specifying the assigned cluster. Therefore, computing platforms need a cross-cluster job scheduler to solve the problem of cross-cluster job scheduling in the background. The scheduler can comprehensively consider the characteristics of heterogeneous clusters and schedule jobs to the optimal cluster for computation.

[0003] However, in related technologies, cross-cluster job schedulers generally only consider cluster resource metrics (such as basic CPU / memory metrics), resulting in a single decision-making dimension. They are mostly rule-based scheduling, lacking predictability; their scheduling modes are also limited, typically using load balancing algorithms, which cannot be dynamically optimized based on historical data and real-time metrics to adapt to dynamic load changes, making them ill-suited for diverse scheduling needs. In summary, existing cross-cluster job schedulers struggle to adapt to diverse scheduling requirements and dynamic load changes. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art by providing a scheduling method, apparatus, electronic device and computer-readable storage medium for cross-cluster operations. This method can realize flexible and effective scheduling of cross-cluster operations.

[0005] In a first aspect, the present invention provides a method for scheduling cross-cluster jobs, comprising: receiving a scheduling request for a target job, wherein the scheduling request includes a scheduling strategy and resource requirements corresponding to several subtasks, and the scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy; determining the target clusters corresponding to the several subtasks according to the scheduling strategy and resource requirements; and scheduling the several subtasks to their corresponding target clusters.

[0006] Preferably, the resource requirements include at least one of the following: load requirements, hardware requirements, and taint tolerance requirements. Based on the scheduling strategy and resource requirements, the target clusters corresponding to several sub-tasks are determined, specifically including: acquiring multi-dimensional state data of all clusters, wherein the multi-dimensional state data includes taint information, load state data, and hardware information; determining a set of candidate clusters corresponding to several sub-tasks based on the resource requirements and multi-dimensional state data, wherein candidate clusters refer to clusters whose load state data, hardware information, and taint information respectively meet the load requirements, hardware requirements, and taint tolerance requirements; evaluating the state score of each candidate cluster in the candidate cluster set based on the scheduling strategy and the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks; and determining the target clusters corresponding to several sub-tasks based on the state scores.

[0007] Preferably, the multidimensional state data also includes network state data and cost information. Based on the scheduling strategy and the multidimensional state data of the candidate cluster set corresponding to several sub-tasks, the state score of each candidate cluster in the candidate cluster set is evaluated. Specifically, this includes: determining the content of the scheduling strategy, wherein the scheduling strategy includes one of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, hardware affinity factor scheduling strategy, data factor scheduling strategy, location factor scheduling strategy, and comprehensive scheduling strategy; in response to the scheduling strategy being a comprehensive scheduling strategy, based on the load state data, network state data, and cost information, the computing power load score, network state score, and cost score of each candidate cluster in the candidate cluster set are evaluated respectively, and the computing power load score, network state score, and cost score of each candidate cluster are weighted and summed to obtain the state score of each candidate cluster.

[0008] Preferably, the multidimensional state data also includes mirror cache state data and geographic location information. After determining the content of the scheduling strategy, the cross-cluster job scheduling method further includes: responding to the scheduling strategy being a load factor scheduling strategy, evaluating the computing power load score of each candidate cluster in the candidate cluster set based on the load state data, and determining it as the state score of each candidate cluster; responding to the scheduling strategy being a network factor scheduling strategy, evaluating the network state score of each candidate cluster in the candidate cluster set based on the network state data, and determining it as the state score of each candidate cluster; responding to the scheduling strategy being a cost factor scheduling strategy, evaluating the network state score of each candidate cluster in the candidate cluster set based on cost information. The system calculates the cost score of each candidate cluster and uses it as the status score for each candidate cluster. In response to a hardware affinity factor scheduling strategy, it evaluates the hardware affinity score of each candidate cluster in the candidate cluster set based on hardware information and hardware requirements, and uses it as the status score for each candidate cluster. Similarly, in response to a data factor scheduling strategy, it evaluates the data score of each candidate cluster in the candidate cluster set based on mirrored cache status data, and uses it as the status score for each candidate cluster. Finally, in response to a location factor scheduling strategy, it evaluates the location score of each candidate cluster in the candidate cluster set based on geographical location information, and uses it as the status score for each candidate cluster.

[0009] Preferably, determining the target clusters corresponding to several sub-tasks based on the status score specifically includes: S1, sorting the candidate clusters in the candidate cluster set in descending order according to the status score, and sending a resource reservation request to the first candidate cluster after descending order; S2, receiving the resource reservation response result from the first candidate cluster, and judging the resource reservation response result, wherein the resource reservation response result includes reservation success or reservation failure; S3, in response to the resource reservation response result being reservation failure, removing the first candidate cluster from the candidate cluster set; repeating S1-S3 until the resource reservation response result is reservation success, and determining the first candidate cluster as the target cluster of its corresponding sub-task.

[0010] Preferably, after obtaining the multidimensional state data of all clusters, and before determining the candidate cluster set corresponding to several subtasks based on resource requirements and multidimensional state data, the scheduling method for cross-cluster jobs further includes: grouping all clusters based on the multidimensional state data of all clusters.

[0011] Preferably, the scheduling request also includes scheduling constraints corresponding to several subtasks. After determining the candidate cluster set corresponding to several subtasks based on resource requirements and multidimensional state data, the scheduling method for cross-cluster jobs further includes: grouping several subtasks according to the scheduling constraints corresponding to several subtasks; extracting the intersection of the candidate cluster sets corresponding to the same group of subtasks, and updating the intersection to the candidate cluster set corresponding to the same group of subtasks.

[0012] Secondly, the present invention also provides a scheduling device for cross-cluster operations, including a receiving module, a determining module, and a scheduling module. The receiving module is used to receive a scheduling request for a target job, wherein the scheduling request includes a scheduling strategy and resource requirements corresponding to several sub-tasks. The scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy. The determining module is connected to the receiving module and is used to determine the target clusters corresponding to the several sub-tasks according to the scheduling strategy and resource requirements. The scheduling module is connected to the determining module and is used to schedule the several sub-tasks to their corresponding target clusters.

[0013] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to implement the cross-cluster job scheduling method provided in the first aspect above.

[0014] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the cross-cluster job scheduling method provided in the first aspect above.

[0015] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for scheduling cross-cluster jobs. By offering one or more of load factor scheduling strategies, network factor scheduling strategies, cost factor scheduling strategies, data factor scheduling strategies, hardware affinity factor scheduling strategies, and location factor scheduling strategies, it more accurately schedules cross-cluster jobs to the optimal cluster. It fully considers the characteristics of heterogeneous clusters, such as computing architecture, geographical distribution, and resource data, avoiding scheduling inefficiencies caused by a single decision-making dimension. It can dynamically optimize scheduling based on different business scenarios and dynamic load changes, meeting diverse scheduling needs, improving the overall performance and efficiency of the computing platform, and balancing multi-dimensional and multi-factor objectives related to network, cost, and data. It reduces cross-cluster communication latency and transmission costs, and improves resource utilization. Therefore, this invention enables flexible and effective scheduling of cross-cluster jobs. Attached Figure Description

[0016] Figure 1 This is a flowchart of a cross-cluster job scheduling method according to Embodiment 1 of the present invention;

[0017] Figure 2 This is an example diagram illustrating how, in Embodiment 1 of the present invention, the target cluster corresponding to several sub-tasks is determined based on a status score.

[0018] Figure 3This is a flowchart of a cross-cluster job scheduling method according to Embodiment 2 of the present invention;

[0019] Figure 4 This is a schematic diagram of the structure of a cross-cluster job scheduling device according to Embodiment 3 of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0021] It is understood that the specific embodiments and accompanying drawings described herein are merely for explaining the invention and are not intended to limit the invention.

[0022] It is understood that, without conflict, the various embodiments and features in the embodiments of the present invention can be combined with each other.

[0023] It is understood that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, while the parts unrelated to the present invention are not shown in the drawings.

[0024] It is understood that each unit or module involved in the embodiments of the present invention may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple units or modules may be integrated into one entity structure.

[0025] It is understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of this invention may occur in a different order than that marked in the accompanying drawings.

[0026] It is understood that the flowcharts and block diagrams of this invention illustrate the possible architecture, functions, and operations of systems, apparatuses, devices, and methods according to various embodiments of this invention. Each block in the flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagram and flowchart can be implemented using a hardware-based system to achieve the specified function, or using a combination of hardware and computer instructions.

[0027] It is understood that the units and modules involved in the embodiments of the present invention can be implemented by software or by hardware. For example, the units and modules can be located in a processor.

[0028] Example 1:

[0029] like Figure 1 As shown, this embodiment provides a scheduling method for cross-cluster jobs.

[0030] In this embodiment, a cross-cluster job refers to a set of subtasks defined by the management platform, in which one or more workload units (including data processing tasks and application service components) are intelligently scheduled to be executed collaboratively on multiple heterogeneous clusters to achieve a common goal. Cross-cluster jobs include both activities centered on data flow and activities centered on computational distribution and service deployment. Data flow includes, but is not limited to, data transmission, synchronization, and sharing. The scheduling method for cross-cluster jobs is applied to a cross-cluster job scheduling system, which includes, but is not limited to, a scheduling policy definition unit, a ClusterGroup unit, a JobGroup unit, a Filter unit, a scheduling factor calculation unit, a Score unit, a Prebind unit (i.e., a resource reservation unit), and a control unit.

[0031] Cross-cluster job scheduling methods include:

[0032] S101, receive the scheduling request of the target job, wherein the scheduling request includes a scheduling strategy and resource requirements corresponding to several subtasks, and the scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy and location factor scheduling strategy.

[0033] In this embodiment, the scheduling strategy definition unit incorporates load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy. Users can select one or more combinations of these strategies through the scheduling strategy definition unit. Specifically, the load factor scheduling strategy schedules jobs to clusters with optimal load status data; the network factor scheduling strategy schedules jobs to clusters with optimal network status data; the cost factor scheduling strategy schedules jobs to clusters with optimal cost information; the data factor scheduling strategy schedules jobs to clusters with optimal mirror cache status data; the hardware affinity factor scheduling strategy schedules jobs to clusters with optimal hardware information; and the location factor scheduling strategy schedules jobs to clusters with optimal geographical location information. The control unit creates a scheduler object corresponding to the scheduling request after receiving the scheduling request for the target job. Different scheduler objects map to different Filter units and Score units. For example: / / Class structure definition (implicit)

[0034] CLASS FilteringAndPreferenceScheduler

[0035] PROPERTIES:

[0036] filters ARRAY <filter> / / list of filters

[0037] scorers ARRAY <scorerconfig> / / list of scorers

[0038] END CLASS

[0039] / / Define the main scheduling method

[0040] FUNCTION FilteringAndPreferenceSchedule(

[0041] job : Job, / / The job object to be scheduled

[0042] pools : ARRAY <resourcepoolscluster>, / / Resource pool set

[0043] ) RETURNS (ResourcePoolsCluster, ERROR)

[0044] / / Pre-check

[0045] IF pools IS EMPTY THEN

[0046] RETURN (NULL, "no resource pools provided")

[0047] / / === Filter phase ===

[0048] candidates = CALL s.filter(job, pools)

[0049] IF FAILED(candidates) THEN

[0050] RETURN (NULL, candidates.error)

[0051] IF candidates IS EMPTY THEN

[0052] RETURN (NULL, "no suitable resource pool found")

[0053] / / === Score phase ===

[0054] scoredPools = CALL s.score(job, candidates)

[0055] IF FAILED(scoredPools) THEN

[0056] RETURN (NULL, scoredPools.error)

[0057] / / Sort in descending order by score

[0058] SORT scoredPools BY pool.score DESCENDING

[0059] RETURN (scoredPools, NULL)

[0060] END FUNCTION. The control unit is also used to determine whether the Filter unit and Score unit mapped to the scheduler object are called and run a second time.

[0061] Specifically, resource requirements include at least one of the following: load requirements, hardware requirements, and taint tolerance requirements.

[0062] In this embodiment, the load requirement reflects the job's demand for cluster resources, including but not limited to: CPU (Central Processing Unit), Memory, GPU, and disk space. The hardware requirement reflects the job's demand for cluster hardware information, including but not limited to: configuration tags (e.g., clusterAffinity, clusterSelector, Processor:Ascend-310, gpu-type=a100), device version and kernel version (e.g., Linux 5.4+), and devices such as FPGA and NPU (Neural Network Processing Unit). The taint tolerance requirement reflects the job's tolerance for tainted information in the cluster, including but not limited to: special=true:NoSchedule.

[0063] Optionally, the scheduling request may also include scheduling constraints corresponding to several subtasks.

[0064] In this embodiment, scheduling constraints are used to reflect the specific needs, operating conditions and resource usage limitations of a job, so as to ensure that the job can run in the cluster in the expected way. Scheduling constraints include, but are not limited to, time-related constraints and dependency constraints.

[0065] S102, based on the scheduling strategy and resource requirements, determine the target clusters corresponding to several sub-tasks.

[0066] Specifically, in S102: Based on the scheduling strategy and resource requirements, determine the target clusters corresponding to several sub-tasks, including steps S1021-S1024:

[0067] S1021, obtain multi-dimensional status data of all clusters, including taint information, load status data, and hardware information.

[0068] Optionally, the multidimensional state data may also include network state data and cost information.

[0069] Optionally, the multidimensional state data may also include mirror cache state data and geographic location information.

[0070] In this embodiment, load status data includes, but is not limited to: CPU utilization, memory utilization, GPU utilization, and disk space utilization reported by the cluster node agent. Network status data includes, but is not limited to: minimum latency and latency of ICMP (Internet Control Message Protocol, network layer protocol) / RTT (Round-Trip Time) reported by the cluster node agent. Cost information includes, but is not limited to: unit price and maximum unit price in the cluster unit price configuration file. Image cache status data includes, but is not limited to: image list, pull time, and threshold reported by the cluster node agent. Geographical location information includes, but is not limited to: distance to the job.

[0071] S1022, Based on resource requirements and multidimensional state data, determine a set of candidate clusters corresponding to several sub-tasks. A candidate cluster refers to a cluster whose load state data, hardware information, and taint information meet the load requirements, hardware requirements, and taint tolerance requirements, respectively.

[0072] In this embodiment, the Filter unit incorporates multiple Check methods to filter clusters and determine the candidate cluster set for subtasks. These Check methods include, but are not limited to: CPUNumCheck, CPUTypeCheck, MemoryNumCheck, GPUNumCheck, GPUTypeCheck, StorageNumCheck, FSTypeCheck, LabelCheck, TaintCheck, and OSCheck. For example, considering resource requirements including an 8-core GPU, Processor: Ascend-310, special=true: NoSchedule (intolerant), and Linux 5.4+, clusters meeting at least one of the following criteria can be excluded: only 4-core GPU, mismatched Processor: Ascend-310, taint information including special=true: NoSchedule, and an old kernel. Taking the CPUNumCheck, CPUTypeCheck, LabelCheck, and TaintCheck methods incorporated into the Filter unit as an example, the pseudocode for determining the candidate cluster set for several subtasks using the Filter unit is as follows:

[0073] / / Define the function to generate the CPU count check filter

[0074] FUNCTION CPUNumCheck() RETURNS CheckFilterFunc

[0075] / / Returns the actual filtering function (closure)

[0076] RETURN FUNCTION(job: Job, pools: ARRAY <resourcepoolscluster>)

[0077] RETURNS (ARRAY <resourcepoolscluster>(ERROR)

[0078] / / Extract job CPU requirements

[0079] reqCPU = TRY_CAST(job.Config["cpu"], INT32)

[0080] IF reqCPU IS INVALID THEN

[0081] RETURN (NULL, "invalid CPU requirement")

[0082] / / Filter resource pools that meet CPU requirements

[0083] candidates = NEW ARRAY

[0084] FOR EACH pool IN pools DO

[0085] availCPU = TRY_CAST(pool.Resources["cpu"], INT32)

[0086] IF availCPU IS VALID AND availCPU >= reqCPU THEN

[0087] APPEND pool TO candidates

[0088] RETURN (candidates, NULL)

[0089] END FUNCTION

[0090] END FUNCTION

[0091] / / Define the CPU type check filter generation function

[0092] FUNCTION CPUTypeCheck() RETURNS CheckFilterFunc

[0093] / / Returns the actual filtering function (closure)

[0094] RETURN FUNCTION(job: Job, pools: ARRAY <resourcepoolscluster>)

[0095] RETURNS (ARRAY <resourcepoolscluster>(ERROR)

[0096] / / Extract the CPU type required by the job

[0097] reqCPUType = TRY_CAST(job.Config["cpu_type"], STRING)

[0098] IF reqCPUType IS INVALID THEN

[0099] RETURN (NULL, "invalid CPU type requirement")

[0100] / / Filter resource pools that support this CPU type

[0101] candidates = NEW ARRAY

[0102] FOR EACH pool IN pools DO

[0103] / / Get the list of CPU types supported by the resource pool

[0104] availableCPUTypes=TRY_CAST(pool.ResourceType["cpu"],ARRAY <string>)

[0105] IF availableCPUTypes IS INVALID THEN

[0106] CONTINUE

[0107] / / Check if the required CPU type is included

[0108] FOR EACH cpuType IN availableCPUTypes DO

[0109] IF cpuType == reqCPUType THEN

[0110] APPEND pool TO candidates

[0111] BREAK / / Stop checking the current pool as soon as it is found

[0112] RETURN (candidates, NULL)

[0113] END FUNCTION

[0114] END FUNCTION

[0115] / / Define the node label checking filter generation function

[0116] FUNCTION LabelCheck() RETURNS CheckFilterFunc

[0117] RETURN FUNCTION(job: Job, pools: ARRAY <resourcepoolscluster>)

[0118] RETURNS (ARRAY <resourcepoolscluster>(ERROR)

[0119] / / Extract the node labels required by the task

[0120] reqLabels = TRY_CAST(job.Config["node_labels"], MAP<STRING,STRING> )

[0121] IF reqLabels IS INVALID THEN

[0122] RETURN (NULL, "invalid node labels requirement")

[0123] / / Filter resource pools that meet tag requirements

[0124] candidates = NEW ARRAY

[0125] FOR EACH pool IN pools DO

[0126] poolLabels = TRY_CAST(pool.Metadata["labels"], MAP<STRING, STRING> )

[0127] IF poolLabels IS INVALID THEN

[0128] CONTINUE

[0129] / / Check if all label requirements are met

[0130] match = TRUE

[0131] FOR EACH (key, value) IN reqLabels DO

[0132] IF poolLabels[key] != value THEN

[0133] match = FALSE

[0134] BREAK

[0135] IF match THEN

[0136] APPEND pool TO candidates

[0137] RETURN (candidates, NULL)

[0138] END FUNCTION

[0139] END FUNCTION

[0140] / / Define the function to generate the blemish inspection filter

[0141] FUNCTION TaintCheck() RETURNS CheckFilterFunc

[0142] RETURN FUNCTION(job: Job, pools: ARRAY <resourcepoolscluster>)

[0143] RETURNS (ARRAY <resourcepoolscluster>(ERROR)

[0144] / / Extract taint tolerance settings for the job

[0145] tolerations = TRY_CAST(job.Config["tolerations"], ARRAY <toleration>)

[0146] IF tolerations IS INVALID THEN

[0147] RETURN (NULL, "invalid tolerations configuration")

[0148] / / Filter resource pools where taints are tolerable

[0149] candidates = NEW ARRAY

[0150] FOR EACH pool IN pools DO

[0151] taints = TRY_CAST(pool.Spec["taints"], ARRAY <taint>)

[0152] IF taints IS INVALID OR LENGTH(taints) == 0 THEN

[0153] APPEND pool TO candidates / / Pass directly without taints

[0154] CONTINUE

[0155] / / Check if all taints are tolerated

[0156] tolerated = TRUE

[0157] FOR EACH taint IN taints DO

[0158] found = FALSE

[0159] FOR EACH tol IN tolerations DO

[0160] IF (tol.Key == taint.Key OR tol.Key == " ") AND

[0161] (tol.Operator == "Exists" OR tol.Value == taint.Value) AND

[0162] (tol.Effect == taint.Effect OR tol.Effect == " ") THEN

[0163] found = TRUE

[0164] BREAK

[0165] IF NOT found THEN

[0166] tolerated = FALSE

[0167] BREAK

[0168] IF tolerated THEN [[ID=*57]]

[0169] APPEND pool TO candidates

[0170] RETURN (candidates, NULL)

[0171] END FUNCTION

[0172] END FUNCTION.

[0173] It should be noted that the CPUNumCheck method checks whether the number of available CPU cores in the cluster meets the job requirements; the CPUTypeCheck method checks whether the available CPU architecture in the cluster meets the job requirements; the MemoryNumCheck method checks whether the available memory size in the cluster meets the job requirements; the GPUNumCheck method checks whether the number of available GPUs in the cluster meets the job requirements; the GPUTypeCheck method checks whether the available GPU type in the cluster meets the job requirements; the StorageNumCheck method checks whether the available disk storage size in the cluster meets the job requirements; the FSTypeCheck method checks whether the available file system type in the cluster meets the job requirements; the LabelCheck method checks whether the node labels in the cluster meet the job requirements; the TaintCheck method checks whether the taints in the cluster are tolerated by the job; and the OSCheck method checks whether the kernel version in the cluster meets the job requirements.

[0174] Optionally, after obtaining the multidimensional state data of all clusters in S1021, and before determining the candidate cluster set corresponding to several subtasks based on resource requirements and multidimensional state data in S1022, the cross-cluster job scheduling method further includes:

[0175] S1025 groups all clusters based on their multidimensional state data.

[0176] In this embodiment, the similarity between the multidimensional state data of all clusters is calculated using ClusterGroup cluster group units and predefined ClusterGroup cluster group templates. Clusters with similarity reaching a preset value are grouped together. The ClusterGroup object grouping template includes, but is not limited to: cluster member selectors, grouping attributes, scheduling policy weights, and code snippets corresponding to health status checks. A YAML-formatted ClusterGroup cluster group template is also provided, for example:

[0177] apiVersion: scheduling.example.com / v1alpha1

[0178] kind: ClusterGroup

[0179] metadata:

[0180] name: gpu-a100-group

[0181] annotations:

[0182] scheduling.example.com / auto-join: "true" # Whether to automatically join clusters that match the criteria.

[0183] spec:

[0184] # Cluster member selector (choose one of two)

[0185] clusterSelector:

[0186] matchLabels:

[0187] Accelerator: nvidia-a100

[0188] arch: amd64

[0189] matchExpressions:

[0190] - key: topology.kubernetes.io / zone

[0191] operator: In

[0192] values: [us-west-1a, us-west-1b]

[0193] # or static member list

[0194] # staticMembers:

[0195] # - cluster-01

[0196] # - cluster-02

[0197] # Grouping attributes (used for scheduling decisions)

[0198] properties:

[0199] hardwareProfile:

[0200] cpuArch: "x86_64"

[0201] gpuModel: "NVIDIA-A100"

[0202] minMemoryPerNode: "128Gi

[0203] networkTopology:

[0204] region: "us-west"

[0205] latencyRequirement: "50ms"

[0206] dataAffinity:

[0207] datasets:

[0208] - "imagenet-1k"

[0209] - "coco-2023"

[0210] storageType: "cephfs"

[0211] # Scheduling strategy weights (affecting the Score phase)

[0212] schedulingPolicy:

[0213] weight: 1.5 # Global weight multiplier for this cluster

[0214] preferredResources: # Resource preferences (affects rating)

[0215] - resource: "nvidia.com / gpu"

[0216] weight: 2.0

[0217] - resource: "amd.com / gpu"

[0218] weight: 0.5

[0219] # Health Status Check

[0220] healthCheck:

[0221] interval: "5m"

[0222] failureThreshold: 3

[0223] successThreshold: 2. This embodiment supports multi-level filtering of the candidate cluster set. If there is only one cluster in the candidate cluster or the number of clusters in the candidate cluster is less than a preset threshold, the clusters in the same group of the candidate cluster can be filtered a second time to select the final candidate cluster, which greatly improves the efficiency of determining the candidate cluster set.

[0224] Optionally, in S1022: after determining the candidate cluster set corresponding to several sub-tasks based on resource requirements and multi-dimensional state data, the scheduling method for cross-cluster jobs further includes:

[0225] S1026, group several subtasks according to the scheduling constraints corresponding to the several subtasks.

[0226] S1027, extract the intersection of the candidate cluster sets corresponding to the same group of subtasks, and update the intersection to the candidate cluster set corresponding to the same group of subtasks.

[0227] In this embodiment, the number of subtasks, resource requirements, and scheduling constraints (such as requiring them to be in the same cluster) in the job scheduling request are parsed using the JobGroup unit and a predefined JobGroup template. Subtasks are then grouped according to resource requirements and scheduling constraints to subsequently select a candidate cluster set that meets the overall requirement of maximizing benefits. The predefined JobGroup template includes, but is not limited to, code segments for subtask configuration, scheduling constraints, resource requirement verification rules, advanced control correspondence, and lifecycle hooks, presented in YAML format, for example:

[0228] apiVersion: scheduling.example.com / v1alpha1

[0229] kind: JobGroup

[0230] metadata:

[0231] name: ai-training-group-001

[0232] labels:

[0233] app: distributed-training

[0234] user: alice

[0235] spec:

[0236] # Subtask Configuration

[0237] tasks:

[0238] count: 8 # Required field, total number of subtasks

[0239] template: # Subtask template (PodSpec)

[0240] containers:

[0241] - name: trainer

[0242] Image: PyTorch:1.9

[0243] resources:

[0244] requests:

[0245] CPU: 4

[0246] memory: 16Gi

[0247] nvidia.com / gpu: 1

[0248] limits:

[0249] CPU: 8

[0250] memory: 32Gi

[0251] command: ["python", "train.py"]

[0252] # Scheduling Constraints

[0253] schedulingPolicy:

[0254] coScheduling: # Co-scheduling requirements

[0255] mode: "MustSameCluster" # Allowed: MustSameCluster / PreferredSameCluster / Any

[0256] topologyKey: "rack" # Optional: Desired topology domain

[0257] resourceAggregation: "sum" # Resource aggregation method (sum / max)

[0258] # Resource Requirement Validation Rules

[0259] requirements:

[0260] minClusterCapacity: # Total resources required for the target cluster

[0261] CPU: 32

[0262] memory: 128Gi

[0263] nvidia.com / gpu:8

[0264] nodeSelector: # Node selector

[0265] accelerator: a100

[0266] # Advanced Controls

[0267] failurePolicy:

[0268] restartPolicy: "TerminateAll" # Any / All / TerminateAll

[0269] maxRetries: 3

[0270] # Lifecycle Hooks

[0271] hooks:

[0272] preScheduling:

[0273] command: [" / bin / sh", "-c", "prepare_data.sh"]

[0274] postCompletion:

[0275] Command: [" / bin / sh", "-c", "upload_results.sh"]. This embodiment groups subtasks and utilizes the characteristic that subtasks in the same group usually have similar resource requirements and operating characteristics to take the intersection of the candidate cluster set of the same group of subtasks. This allows for the selection of clusters that simultaneously meet the requirements of all subtasks in the group. This avoids distributing subtasks across different clusters, which could lead to some clusters being overused while others are idle. It helps reduce resource waste and improve overall resource utilization efficiency.

[0276] S1023, based on the scheduling strategy and the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks, evaluate the state score of each candidate cluster in the candidate cluster set.

[0277] In this embodiment, the scheduling factor calculation unit analyzes the content of the scheduling strategy and extracts the corresponding state data from the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks. For example, if the scheduling strategy is a hardware affinity factor scheduling strategy, then hardware information is extracted from the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks, and the corresponding state data is sent to the Score unit. After receiving the corresponding state data, the Score unit can evaluate the state score of each candidate cluster in the candidate cluster set. The specific pseudocode is as follows:

[0278] / / Resource rating function

[0279] FUNCTION Score(job: Job, pools: ARRAY <resourcepoolscluster>,scoreStrategy: MAP<STRING, ANY>) RETURNS (ARRAY <float>, ARRAY <bool>(ERROR)

[0280] / / 1. Initialize resource type configuration

[0281] resourceType = TRY_CAST(scoreStrategy["resource_type"], ARRAY <string>)

[0282] IF resourceType IS NULL OR EMPTY(resourceType) THEN

[0283] resourceType = ["cpu"] / / Default value

[0284] / / 2. Initialize resource weights

[0285] resourceWeights=TRY_CAST(scoreStrategy["resource_weights"], MAP<STRING, FLOAT> )

[0286] IF resourceWeights IS NULL THEN

[0287] resourceWeights = {

[0288] "cpu": 0.5,

[0289] "memory": 0.3,

[0290] "gpu": 0.2

[0291] / / Default weights

[0292] / / 3. Extract job resource requirements

[0293] reqVal = NEW MAP<STRING, FLOAT>

[0294] FOR EACH rt IN resourceType DO

[0295] val = TRY_CAST(job.Config[rt], FLOAT)

[0296] IF val IS INVALID THEN

[0297] RETURN (NULL, NULL, "invalid resource requirement")

[0298] reqVal[rt] = val

[0299] / / 4. Initialize the result array

[0300] scores = NEW ARRAY <float>(LENGTH(pools), 0.0)

[0301] proceed = NEW ARRAY <bool>(LENGTH(pools), FALSE)

[0302] / / 5. Calculate the maximum value for each resource type (for normalization)

[0303] maxVal = NEW MAP<STRING, FLOAT>

[0304] FOR EACH rt IN resourceType DO

[0305] maxVal[rt] = 0.0

[0306] FOR EACH pool IN pools DO

[0307] avail = TRY_CAST(pool.Resources[rt], FLOAT)

[0308] IF avail IS VALID AND avail > maxVal[rt] THEN

[0309] maxVal[rt] = avail

[0310] / / 6. Calculate the score for each resource pool

[0311] FOR i FROM 0 TO LENGTH(pools)-1 DO

[0312] pool = pools[i]

[0313] totalScore = 0.0

[0314] isValid = TRUE

[0315] FOR EACH rt IN resourceType DO

[0316] avail = TRY_CAST(pool.Resources[rt], FLOAT)

[0317] / / Check resource availability

[0318] IF avail IS INVALID OR avail == 0 THEN

[0319] scores[i] = 0.0

[0320] proceed[i] = FALSE

[0321] isValid = FALSE

[0322] BREAK

[0323] / / Calculate individual scores (including demand satisfaction and resource scale factor)

[0324] normalizedAvail = avail / maxVal[rt]

[0325] demandSatisfaction = 1 - (reqVal[rt] / avail)

[0326] score = demandSatisfaction normalizedAvail 100

[0327] / / Weighted cumulative

[0328] totalScore += score resourceWeights[rt]

[0329] IF isValid THEN

[0330] scores[i] = totalScore

[0331] proceed[i] = (totalScore >= 5) / / Threshold judgment

[0332] RETURN (scores, proceed, NULL)

[0333] END FUNCTION.

[0334] Specifically, S1023: Based on the scheduling strategy and the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks, evaluate the state score of each candidate cluster in the candidate cluster set, including: determining the content of the scheduling strategy, wherein the scheduling strategy includes one of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, hardware affinity factor scheduling strategy, data factor scheduling strategy, location factor scheduling strategy, and comprehensive scheduling strategy; in response to the scheduling strategy being a comprehensive scheduling strategy, based on the load state data, network state data, and cost information, evaluate the computing power load score, network state score, and cost score of each candidate cluster in the candidate cluster set, and perform a weighted summation of the computing power load score, network state score, and cost score of each candidate cluster to obtain the state score of each candidate cluster.

[0335] Optionally, after determining the content of the scheduling strategy, the cross-cluster job scheduling method further includes: responding to the scheduling strategy being a load factor scheduling strategy, evaluating the computing power load score of each candidate cluster in the candidate cluster set based on load status data, and determining it as the status score of each candidate cluster; responding to the scheduling strategy being a network factor scheduling strategy, evaluating the network status score of each candidate cluster in the candidate cluster set based on network status data, and determining it as the status score of each candidate cluster; responding to the scheduling strategy being a cost factor scheduling strategy, evaluating the cost score of each candidate cluster in the candidate cluster set based on cost information, and determining it as the status score of each candidate cluster; responding to the scheduling strategy being a hardware affinity factor scheduling strategy, evaluating the hardware affinity score of each candidate cluster in the candidate cluster set based on hardware information and hardware requirements, and determining it as the status score of each candidate cluster; responding to the scheduling strategy being a data factor scheduling strategy, evaluating the data score of each candidate cluster in the candidate cluster set based on mirror cache status data, and determining it as the status score of each candidate cluster; responding to the scheduling strategy being a location factor scheduling strategy, evaluating the location score of each candidate cluster in the candidate cluster set based on geographical location information, and determining it as the status score of each candidate cluster.

[0336] In this embodiment, according to the formula ,

[0337] ,

[0338] ,

[0339] ,

[0340] ,

[0341] It can evaluate the computing load score of each candidate cluster in the candidate cluster set separately. Network status score Cost rating Hardware affinity score Data scoring Location rating This embodiment uses computational load scoring to clearly understand the current computing resource usage of each candidate cluster. Assigning tasks to clusters with lower computational load avoids performance bottlenecks in high-load clusters while fully utilizing the computing resources of low-load clusters, improving overall resource utilization. For tasks with high computing power requirements, selecting clusters with high computational load scores (i.e., low load and high computing power) ensures fast and efficient task execution, reducing execution time and improving system response speed. By evaluating network status scores, assigning tasks requiring extensive data interaction to clusters with high network status scores reduces the risk of task failure due to network failures, enhancing system reliability and stability. Cost scoring facilitates cost-benefit analysis during resource allocation, encouraging resource allocation to be more cost-effective. The system prioritizes clusters with higher cost-effectiveness, maximizing resource utilization and avoiding unnecessary waste. By assigning tasks to clusters that match hardware characteristics through hardware affinity scoring, it fully leverages hardware performance advantages, improves hardware utilization, ensures tasks run in suitable hardware environments, and avoids task failures or performance degradation due to hardware incompatibility, thereby increasing task execution success rate and stability. Assigning tasks to clusters with high data scores (i.e., large data storage capacity and fast data access speed) reduces data transmission between different clusters, lowers data transmission overhead, and improves task execution efficiency. For tasks requiring real-time interaction, selecting clusters closer to users or data sources (high location scores) reduces network latency, improves system response speed, and enhances user experience.

[0342] It should be noted that the comprehensive scheduling strategy in this embodiment only considers key scheduling factors (i.e., load factor, network factor, and cost factor), and does not consider all scheduling factors (i.e., load factor, network factor, cost factor, hardware affinity factor, data factor, and location factor). This simplifies the complexity of scheduling decisions, reduces computation and analysis time, and improves scheduling efficiency. As the system scales up and its complexity increases, the scheduling strategy should have good scalability to cope with complex and changing environments, so as to integrate more scheduling factors and adapt to constantly changing needs. The comprehensive scheduling strategy can be expanded by adding or adjusting scheduling factors to consider more scheduling factors and optimize performance. For example, the comprehensive scheduling strategy may only consider hardware affinity and location factors.

[0343] S1024, determine the target cluster corresponding to several sub-tasks based on the status score.

[0344] Specifically, S1024: Determine the target clusters corresponding to several sub-tasks based on the status scores, including: S1, sorting the candidate clusters in the candidate cluster set in descending order according to the status scores, and sending a resource reservation request to the first candidate cluster after descending order; S2, receiving the resource reservation response result from the first candidate cluster, and judging the resource reservation response result, wherein the resource reservation response result includes reservation success or reservation failure; S3, in response to the resource reservation response result being reservation failure, removing the first candidate cluster from the candidate cluster set; repeating S1-S3 until the resource reservation response result is reservation success, and determining the first candidate cluster as the target cluster of its corresponding sub-task.

[0345] In this embodiment, as Figure 2 As shown, after sorting the candidate clusters in the candidate cluster set in descending order according to their state scores, the Prebind unit sequentially sends requests to each candidate cluster (i.e., ...) Figure 2 The target cluster sends a resource reservation request; each candidate cluster checks its own real-time resource status; if its real-time resource status is available, it sets its TTL=300s, locks the resource item that satisfies the resource reservation request, and sends a successful reservation feedback to the Prebind unit (i.e., ...). Figure 2 The resource reservation response result is HTTP 201 Created; if the real-time resource status is available, it reports reservation failure to the Prebind unit (i.e., ...). Figure 2 The resource reservation response result of the HTTP 409 Conflict.

[0346] S103 schedules several subtasks to their corresponding target clusters.

[0347] This embodiment provides a cross-cluster job scheduling method that, by offering one or more of load factor scheduling strategies, network factor scheduling strategies, cost factor scheduling strategies, data factor scheduling strategies, hardware affinity factor scheduling strategies, and location factor scheduling strategies, more accurately schedules cross-cluster jobs to the optimal cluster. It fully considers the characteristics of heterogeneous clusters, such as computing architecture, geographical distribution, and resource data, avoiding scheduling inefficiencies caused by a single decision-making dimension. It can dynamically optimize scheduling based on different business scenarios and dynamic load changes, meeting diverse scheduling needs, improving the overall performance and efficiency of the computing platform, balancing multi-dimensional and multi-factor objectives related to network, cost, and data, reducing cross-cluster communication latency and transmission costs, improving resource utilization, and achieving flexible and effective scheduling of cross-cluster jobs.

[0348] Example 2:

[0349] like Figure 3 As shown, this embodiment provides a method for scheduling cross-cluster jobs. The method for scheduling cross-cluster jobs includes:

[0350] S201, Receive the scheduling request for the target job, wherein the scheduling request includes a scheduling strategy and resource requirements corresponding to several subtasks. The scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy. The resource requirements include at least one of the following: load requirements, hardware requirements, and taint tolerance requirements.

[0351] In this embodiment, the target scheduling request is... Figure 3 The job scheduling request in the process.

[0352] S202, obtain multi-dimensional status data of all clusters, including taint information, load status data, and hardware information; based on resource requirements and multi-dimensional status data, determine a set of candidate clusters corresponding to several sub-tasks, where candidate clusters refer to clusters whose load status data, hardware information, and taint information meet the load requirements, hardware requirements, and taint tolerance requirements, respectively.

[0353] In this embodiment, a candidate cluster set corresponding to several sub-tasks is determined. Figure 3 The filtering is based on the Check method.

[0354] S203, determine the content of the scheduling strategy, wherein the scheduling strategy includes one of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, hardware affinity factor scheduling strategy, data factor scheduling strategy, location factor scheduling strategy, and comprehensive scheduling strategy; in response to the scheduling strategy including load factor scheduling strategy, network factor scheduling strategy, and cost factor scheduling strategy, based on load status data, network status data, and cost information, evaluate the computing power load score, network status score, and cost score of each candidate cluster in the candidate cluster set, and perform a weighted summation of the computing power load score, network status score, and cost score of each candidate cluster to obtain the status score of each candidate cluster; in response to the scheduling strategy being load factor scheduling strategy, evaluate the computing power load score of each candidate cluster in the candidate cluster set based on load status data, and determine it as the status score of each candidate cluster; in response to the scheduling strategy being network factor scheduling... The strategy evaluates the network status score of each candidate cluster in the candidate cluster set based on network status data and determines it as the status score of each candidate cluster. In response to the scheduling strategy of cost factor scheduling, it evaluates the cost score of each candidate cluster in the candidate cluster set based on cost information and determines it as the status score of each candidate cluster. In response to the scheduling strategy of hardware affinity factor scheduling, it evaluates the hardware affinity score of each candidate cluster in the candidate cluster set based on hardware information and hardware requirements and determines it as the status score of each candidate cluster. In response to the scheduling strategy of data factor scheduling, it evaluates the data score of each candidate cluster in the candidate cluster set based on mirror cache status data and determines it as the status score of each candidate cluster. In response to the scheduling strategy of location factor scheduling, it evaluates the location score of each candidate cluster in the candidate cluster set based on geographical location information and determines it as the status score of each candidate cluster.

[0355] In this embodiment, the computing load score is... Figure 3 The computational resource load score and cost score are... Figure 3 Resource cost scoring, data scoring Figure 3 Locality score of data in the dataset.

[0356] S204: Sort the candidate clusters in the candidate cluster set in descending order according to the status score, and send a resource reservation request to the first candidate cluster after descending order; receive the resource reservation response result fed back by the first candidate cluster, and judge the resource reservation response result, wherein the resource reservation response result includes reservation success or reservation failure; in response to the resource reservation response result being reservation failure, remove the first candidate cluster from the candidate cluster set; repeat until the resource reservation response result is reservation success, and determine the first candidate cluster as the target cluster of its corresponding subtask.

[0357] S205 schedules several subtasks to their corresponding target clusters.

[0358] This embodiment provides a cross-cluster job scheduling method that, by offering one or more of load factor scheduling strategies, network factor scheduling strategies, cost factor scheduling strategies, data factor scheduling strategies, hardware affinity factor scheduling strategies, and location factor scheduling strategies, more accurately schedules cross-cluster jobs to the optimal cluster. It fully considers the characteristics of heterogeneous clusters, such as computing architecture, geographical distribution, and resource data, avoiding scheduling inefficiencies caused by a single decision-making dimension. It can dynamically optimize scheduling based on different business scenarios and dynamic load changes, meeting diverse scheduling needs, improving the overall performance and efficiency of the computing platform, balancing multi-dimensional and multi-factor objectives related to network, cost, and data, reducing cross-cluster communication latency and transmission costs, improving resource utilization, and achieving flexible and effective scheduling of cross-cluster jobs.

[0359] Example 3:

[0360] like Figure 4 As shown, this embodiment also provides a scheduling device for cross-cluster jobs, including a receiving module 41, a determining module 42, and a scheduling module 43. The receiving module 41 is used to receive a scheduling request for a target job, wherein the scheduling request includes a scheduling strategy and resource requirements corresponding to several sub-tasks. The scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy. The determining module 42 is connected to the receiving module 41 and is used to determine the target clusters corresponding to the several sub-tasks according to the scheduling strategy and resource requirements. The scheduling module 43 is connected to the determining module 42 and is used to schedule the several sub-tasks to their corresponding target clusters.

[0361] Specifically, the determination module 42 includes: an acquisition unit 421, a first determination unit 422, an evaluation unit 423, and a second determination unit 424. The acquisition unit 421 is used to acquire multi-dimensional state data of all clusters, wherein the multi-dimensional state data includes taint information, load state data, and hardware information. The first determination unit 422 is used to determine a set of candidate clusters corresponding to several sub-tasks based on resource requirements and multi-dimensional state data, wherein the candidate clusters refer to clusters whose load state data, hardware information, and taint information meet the load requirements, hardware requirements, and taint tolerance requirements, respectively. The evaluation unit 423 is used to evaluate the state score of each candidate cluster in the candidate cluster set based on the scheduling policy and the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks. The second determination unit 424 is used to determine the target clusters corresponding to several sub-tasks based on the state scores.

[0362] Specifically, the evaluation unit 423 includes a judgment subunit and a first evaluation subunit. The judgment subunit is used to judge the content of the scheduling strategy, wherein the scheduling strategy includes one of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, hardware affinity factor scheduling strategy, data factor scheduling strategy, location factor scheduling strategy, and comprehensive scheduling strategy. The first evaluation subunit is used to respond to the scheduling strategy being a comprehensive scheduling strategy by evaluating the computing power load score, network status score, and cost score of each candidate cluster in the candidate cluster set according to the load status data, network status data, and cost information, and then performing a weighted summation of the computing power load score, network status score, and cost score of each candidate cluster to obtain the status score of each candidate cluster.

[0363] Optionally, the evaluation unit 423 further includes: a second evaluation subunit, a third evaluation subunit, a fourth evaluation subunit, a fifth evaluation subunit, a sixth evaluation subunit, and a seventh evaluation subunit. The second evaluation subunit, in response to a load factor scheduling strategy, evaluates the computing power load score of each candidate cluster in the candidate cluster set based on load status data and determines it as the status score of each candidate cluster. The third evaluation subunit, in response to a network factor scheduling strategy, evaluates the network status score of each candidate cluster in the candidate cluster set based on network status data and determines it as the status score of each candidate cluster. The fourth evaluation subunit, in response to a cost factor scheduling strategy, evaluates the network status score of each candidate cluster in the candidate cluster set based on cost information. The fifth evaluation subunit, in response to the scheduling strategy of hardware affinity factor scheduling, evaluates the hardware affinity score of each candidate cluster in the candidate cluster set based on hardware information and hardware requirements, and determines it as the status score of each candidate cluster. The sixth evaluation subunit, in response to the scheduling strategy of data factor scheduling, evaluates the data score of each candidate cluster in the candidate cluster set based on mirror cache status data, and determines it as the status score of each candidate cluster. The seventh evaluation subunit, in response to the scheduling strategy of location factor scheduling, evaluates the location score of each candidate cluster in the candidate cluster set based on geographical location information, and determines it as the status score of each candidate cluster.

[0364] Specifically, the second determining unit 424 includes: a sorting subunit, a receiving subunit, a removal subunit, and a determining subunit. The sorting subunit is used to sort each candidate cluster in the candidate cluster set in descending order according to the status score, and send a resource reservation request to the first candidate cluster after descending order. The receiving subunit is used to receive the resource reservation response result fed back by the first candidate cluster and judge the resource reservation response result, wherein the resource reservation response result includes reservation success or reservation failure. The removal subunit is used to remove the first candidate cluster from the candidate cluster set in response to the resource reservation response result being reservation failure. The determining subunit is used to determine the first candidate cluster as the target cluster of its corresponding subtask until the resource reservation response result is reservation success.

[0365] Optionally, the determining module 42 further includes a first grouping unit 425, used to group all clusters according to the multidimensional state data of all clusters.

[0366] Optionally, the determining module 42 further includes a second grouping unit 426 and an updating unit 427. The second grouping unit 426 is used to group several subtasks according to the scheduling constraints corresponding to several subtasks. The updating unit 427 is used to extract the intersection of the candidate cluster sets corresponding to the same group of subtasks and update the intersection to the candidate cluster set corresponding to the same group of subtasks.

[0367] Understandably, the cross-cluster job scheduling device provided above executes the cross-cluster job scheduling method corresponding to Embodiment 1 provided above. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects of the scheme corresponding to the cross-cluster job scheduling method of Embodiment 1 above, which will not be repeated here.

[0368] Example 4:

[0369] This embodiment also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to implement the cross-cluster job scheduling method in Embodiment 1 or Embodiment 2 above.

[0370] Example 5:

[0371] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the cross-cluster job scheduling method in Embodiment 1 or Embodiment 2 above.

[0372] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.< / bool> < / float> < / string> < / bool> < / float> < / resourcepoolscluster> < / taint> < / toleration> < / resourcepoolscluster> < / resourcepoolscluster> < / resourcepoolscluster> < / resourcepoolscluster> < / string> < / resourcepoolscluster> < / resourcepoolscluster> < / resourcepoolscluster> < / resourcepoolscluster> < / resourcepoolscluster> < / scorerconfig> < / filter>

Claims

1. A method for scheduling cross-cluster jobs, characterized in that, include: Receive a scheduling request for the target job, wherein the scheduling request includes a scheduling strategy and resource requirements corresponding to several subtasks, and the scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy; Based on the scheduling strategy and resource requirements, determine the target clusters corresponding to several sub-tasks; Several subtasks are scheduled to their corresponding target clusters.

2. The scheduling method for cross-cluster jobs according to claim 1, characterized in that, Resource requirements include at least one of the following: load requirements, hardware requirements, and taint tolerance requirements. The step of determining the target clusters corresponding to several sub-tasks based on scheduling strategies and resource requirements specifically includes: Obtain multidimensional status data for all clusters, including taint information, load status data, and hardware information. Based on resource requirements and multidimensional status data, a set of candidate clusters corresponding to several sub-tasks is determined. Among them, the candidate clusters refer to the clusters whose load status data, hardware information, and taint information meet the load requirements, hardware requirements, and taint tolerance requirements, respectively. Based on the scheduling strategy and the multidimensional state data of the candidate cluster set corresponding to several sub-tasks, evaluate the state score of each candidate cluster in the candidate cluster set; The target cluster corresponding to several sub-tasks is determined based on the status score.

3. The scheduling method for cross-cluster jobs according to claim 2, characterized in that, Multidimensional state data also includes network state data and cost information. The step of evaluating the state score of each candidate cluster in the candidate cluster set based on the scheduling strategy and the multi-dimensional state data of the candidate cluster set corresponding to several sub-tasks specifically includes: Determine the content of the scheduling strategy, which includes the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, hardware affinity factor scheduling strategy, data factor scheduling strategy, location factor scheduling strategy, and comprehensive scheduling strategy. The scheduling strategy is a comprehensive scheduling strategy. Based on load status data, network status data, and cost information, the computing power load score, network status score, and cost score of each candidate cluster in the candidate cluster set are evaluated respectively. The computing power load score, network status score, and cost score of each candidate cluster are weighted and summed to obtain the status score of each candidate cluster.

4. The scheduling method for cross-cluster jobs according to claim 3, characterized in that, Multidimensional state data also includes mirror cache state data and geographic location information. After determining the content of the scheduling policy, the following is also included: In response to the scheduling strategy being the load factor scheduling strategy, the computing power load score of each candidate cluster in the candidate cluster set is evaluated based on the load status data, and then determined as the status score of each candidate cluster. In response to the scheduling strategy being a network factor scheduling strategy, the network status score of each candidate cluster in the candidate cluster set is evaluated based on the network status data and determined as the status score of each candidate cluster. In response to the scheduling strategy being a cost factor scheduling strategy, the cost score of each candidate cluster in the candidate cluster set is evaluated based on cost information and determined as the status score of each candidate cluster. In response to the scheduling strategy being a hardware affinity factor scheduling strategy, the hardware affinity score of each candidate cluster in the candidate cluster set is evaluated based on hardware information and hardware requirements, and then determined as the status score of each candidate cluster. In response to the data factor scheduling strategy, the data score of each candidate cluster in the candidate cluster set is evaluated based on the image cache status data, and then determined as the status score of each candidate cluster. In response to the location factor scheduling strategy, the location score of each candidate cluster in the candidate cluster set is evaluated based on the geographical location information, and then determined as the status score of each candidate cluster.

5. The scheduling method for cross-cluster jobs according to claim 2, characterized in that, The step of determining the target cluster corresponding to several sub-tasks based on the status score specifically includes: S1, sort the candidate clusters in the candidate cluster set in descending order according to the status score, and send a resource reservation request to the first candidate cluster after descending order. S2, receive the resource reservation response result from the first candidate cluster, and determine the resource reservation response result, wherein the resource reservation response result includes reservation success or reservation failure; S3, in response to the resource reservation failure result, remove the first candidate cluster from the candidate cluster set; Repeat S1-S3 until the resource reservation response result is "reservation successful", then determine the first candidate cluster as the target cluster of its corresponding subtask.

6. The scheduling method for cross-cluster jobs according to claim 2, characterized in that, After obtaining the multidimensional state data of all clusters, and before determining the candidate cluster set corresponding to several sub-tasks based on resource requirements and multidimensional state data, the method further includes: All clusters are grouped based on their multidimensional state data.

7. The scheduling method for cross-cluster jobs according to claim 2, characterized in that, The scheduling request also includes scheduling constraints corresponding to several subtasks. After determining the candidate cluster set corresponding to several sub-tasks based on resource requirements and multi-dimensional state data, the process further includes: Group the subtasks according to the scheduling constraints corresponding to the subtasks; Extract the intersection of the candidate cluster sets corresponding to the same group of subtasks, and update the intersection with the candidate cluster sets corresponding to the same group of subtasks.

8. A scheduling device for cross-cluster operations, characterized in that, It includes a receiving module, a determining module, and a scheduling module. The receiving module is used to receive a scheduling request for the target job. The scheduling request includes a scheduling strategy and resource requirements corresponding to several subtasks. The scheduling strategy includes one or more combinations of the following: load factor scheduling strategy, network factor scheduling strategy, cost factor scheduling strategy, data factor scheduling strategy, hardware affinity factor scheduling strategy, and location factor scheduling strategy. The determination module, connected to the receiving module, is used to determine the target clusters corresponding to several sub-tasks based on the scheduling strategy and resource requirements. The scheduling module, connected to the determination module, is used to schedule several subtasks to their corresponding target clusters.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to implement a cross-cluster job scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a cross-cluster job scheduling method as described in any one of claims 1 to 7.

Images