Intelligent computing cloud platform based on topology-aware computing power anti-fragmentation scheduling method and device

Abstract

This invention provides a method and apparatus for anti-fragmentation scheduling of computing power based on topology awareness in intelligent computing cloud platforms, relating to the fields of intelligent computing centers, smart computing centers, computing power infrastructure, and smart computing cloud technologies. The method includes: Step S1, obtaining the computing power running tasks to be scheduled, wherein the computing power running tasks carry communication constraint information; Step S2, determining at least one candidate deployment scheme that meets the deployment constraints corresponding to the computing power running tasks to be scheduled based on the communication constraint information and the hierarchical topology tree; Step S3, calculating the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing power resources in each subdomain of the hierarchical topology tree; wherein the first fragmentation metric value is used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme; Step S4, selecting the candidate deployment scheme whose first fragmentation metric value meets the preset optimization target as the target deployment scheme, and deploying the computing power running tasks to be scheduled based on the target deployment scheme.

Description

Technical Field

[0001] This invention relates to the fields of intelligent computing centers, smart computing centers, computing infrastructure, and smart cloud technologies, specifically to a method and apparatus for topology-aware anti-fragmentation scheduling of computing power in an intelligent computing cloud platform. Background Technology

[0002] With the rapid development of artificial intelligence technology, "intelligent computing centers" and "smart computing centers" have emerged.

[0003] An "intelligent computing center" refers to a facility that provides the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models) by utilizing large-scale heterogeneous computing resources, including general-purpose and intelligent computing power. Intelligent computing centers encompass facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0004] "Intelligent computing center" includes, but is not limited to, "intelligent computing center".

[0005] "Intelligent computing center" or artificial intelligence computing center is a type of computing infrastructure that provides computing power services, data services, and algorithm services required for artificial intelligence applications, based on artificial intelligence theory and adopting artificial intelligence computing architecture.

[0006] "Computing power" is the core of "intelligent computing center" and "smart computing center". It is the ability of computer equipment or computing / data center to process parameters. It is the ability of computer hardware and software to work together to execute a certain computing requirement. It is the computing power to achieve the target result output by processing parameter data. It is a new type of productivity that integrates parameter computing power, network carrying capacity and data storage capacity. It mainly provides services to society through computing power infrastructure.

[0007] With the increasing prevalence of large-scale model computing tasks with hundreds of billions or trillions of parameters, a single computing task typically requires the simultaneous use of tens to thousands of AI accelerator cards (such as GPUs and NPUs), and these computing resources are required to be in place and run collaboratively in a "gang" manner. The computing task organizes computation and communication through various parallel strategies such as Tensor Parallelism (TP), Pipeline Parallelism (PP), and Data Parallelism (DP).

[0008] In related technologies, the scheduling method for computing power operation tasks in intelligent computing cloud platforms usually only guarantees that the amount of resources required for computing power operation tasks is met. This results in highly communication-sensitive parallel groups (such as tensor parallel groups) being deployed across nodes and racks, which leads to a decrease in actual communication bandwidth, introduces additional communication latency, and reduces the execution efficiency of computing power operation tasks. Summary of the Invention

[0009] This invention provides a topology-aware anti-fragmentation scheduling method and apparatus for intelligent computing cloud platforms to solve the problem in the prior art where communication-sensitive parallel groups are usually deployed across nodes and racks, resulting in reduced communication bandwidth, introduced additional communication latency, and reduced execution efficiency of computing tasks.

[0010] To solve the above problems, the present invention is implemented as follows: In a first aspect, the present invention provides a topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms, comprising: Step S1: Obtain the computing power task to be scheduled; wherein the computing power task to be scheduled carries communication constraint information, the communication constraint information includes the deployment constraint conditions of the first parallel group of the computing power task to be scheduled, the deployment constraint conditions are used to indicate that the computing nodes deploying the first parallel group must be located in the same subdomain in the hierarchical topology tree of the computing cluster with an interconnection bandwidth of not less than a first preset threshold. Step S2: Based on the communication constraint information and the hierarchical topology tree, determine at least one candidate deployment scheme that satisfies the deployment constraint conditions for the task to be scheduled to run computing power. Step S3: Calculate the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree; wherein, the first fragmentation metric value is used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme; Step S4: Select a candidate deployment scheme whose first fragmentation metric value meets the preset optimization target as the target deployment scheme, and deploy the task to be scheduled by computing power according to the target deployment scheme.

[0011] In one embodiment, step S3 includes: Step S3.1: Determine the total amount of idle computing resources of each subdomain in each level of the hierarchical topology tree, and determine the proportion of idle computing resources of each subdomain based on the amount of idle computing resources of each subdomain and the total amount of idle computing resources. Step S3.2: Calculate the information entropy of the level based on the proportion of idle computing resources in each subdomain as the fragmentation entropy of the level; wherein, the fragmentation entropy is used to characterize the degree of computing power fragmentation of the level. Step S3.3: Based on the weights of each level in the hierarchical topology tree, the fragment entropy of each level is weighted to determine the first fragment metric value.

[0012] In one embodiment, step S3.3 includes: Step S3.3.1: Based on the weight of each level and the total amount of idle computing resources of each level, the fragment entropy of each level is weighted to determine the first fragmentation metric value.

[0013] In one embodiment, the information entropy of the hierarchy is equal to the negative sum of the products of the proportion of idle computing resources in each subdomain of the hierarchy and the logarithm of the proportion of idle computing resources.

[0014] In one embodiment, the weights of each level are negatively correlated with the depth of each level in the hierarchical topology tree.

[0015] In one embodiment, the method further includes: Step S5: Continuously monitor the current second fragmentation metric value of the computing cluster. If the second fragmentation metric value is greater than the second preset threshold, select candidate computing tasks that meet the migration conditions from the currently running computing power running tasks. Step S6: Determine the computing power to be migrated from the candidate computing power running tasks; Step S7: Save the status information of the computing power running task to be migrated, and migrate the computing power running task to be migrated from the current deployment location to the target location.

[0016] In one embodiment, step S6 includes: Step S6.1: Determine the migration cost and migration benefits of the candidate computing power running tasks; Step S6.2: Based on the migration cost and migration benefit of the candidate computing power running tasks, determine the computing power running task to be migrated from the candidate computing power running tasks.

[0017] In one embodiment, the migration conditions include at least one of the following: the resource consumption of the task is less than a third preset threshold, the task supports checkpoint saving and restoration, the estimated remaining runtime of the task is less than a fourth preset threshold, and the task is not currently in the execution phase where the required latency is less than a fifth preset threshold.

[0018] In one embodiment, the method further includes: Step S8: Based on the load profile over a preset period of time, predict the increase in resource demand for the first resource pool at a future target time. Step S9: Before the target time, migrate candidate physical devices that are currently idle or whose computing power utilization rate is lower than the sixth preset threshold in the second resource pool to the first resource pool.

[0019] In one embodiment, step S9 includes: Step S9.1: If the physical device to be migrated among the candidate physical devices is running a first computing power running task, before performing preemptive scheduling on the first computing power running task, the status information of the first computing power running task is saved. Step S9.2: Determine the preemption cost for preempting the scheduling of the first computing power task based on the time and resource overhead of the save operation. Step S9.3: Based on the migration benefits corresponding to the migration of the physical device to be migrated and the preemption cost, determine whether to preemptively schedule the first computing power to run the task and migrate the physical device to be migrated to the first resource pool.

[0020] In one embodiment, the method further includes: Step S10: For the second computing power running task waiting in the queue, generate and output the first candidate solution and its corresponding estimated completion time, the second candidate solution and its corresponding estimated completion time; The first candidate solution is to wait for the computing power fragments to be reorganized and then deploy the second computing power running task according to the deployment constraints of the second computing power running task. The second candidate solution is to use a degraded topology scheme to deploy the second computing power running task immediately.

[0021] In one embodiment, the communication constraint information further includes deployment constraints for a second parallel group of the task to be scheduled to run, wherein the deployment constraints for the second parallel group are used to indicate that the computing nodes deploying the second parallel group are allowed to traverse multiple subdomains in the hierarchical topology tree where the interconnect bandwidth is lower than the first preset threshold.

[0022] Secondly, the present invention also provides a topology-aware anti-fragmentation scheduling device for intelligent computing cloud platforms, comprising: The acquisition module is used to acquire the computing power to be scheduled running tasks; wherein, the computing power to be scheduled running tasks carries communication constraint information, the communication constraint information includes the deployment constraint conditions of the first parallel group of the computing power to be scheduled running tasks, the deployment constraint conditions are used to indicate that the computing nodes deploying the first parallel group must be located in the same subdomain in the hierarchical topology tree of the computing cluster with an interconnection bandwidth of not less than a first preset threshold. The first determining module is used to determine at least one candidate deployment scheme that satisfies the deployment constraints for the task to be scheduled by computing power, based on the communication constraint information and the hierarchical topology tree. The computing module is used to calculate the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree; wherein, the first fragmentation metric value is used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme; The deployment module is used to select a candidate deployment scheme whose first fragmentation metric value meets a preset optimization target as the target deployment scheme, and deploy the task to be scheduled by computing power according to the target deployment scheme.

[0023] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the intelligent computing cloud platform based on topology-aware anti-fragmentation scheduling method as described in the first aspect above.

[0024] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms as described in the first aspect above.

[0025] Fifthly, the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms described in the first aspect above.

[0026] In this invention, by acquiring the scheduled computing power tasks carrying communication constraint information, and based on the communication constraint information and the hierarchical topology tree of the computing cluster, candidate deployment schemes that meet the first parallel group deployment constraint conditions are determined. Furthermore, based on the amount of idle computing power resources in each subdomain of the hierarchical topology tree, a first fragmentation metric value corresponding to each candidate deployment scheme is calculated to characterize the expected fragmentation level of the cluster's computing power after deployment. Finally, a target deployment scheme is selected based on a preset optimization objective to complete the deployment of the scheduled computing power tasks. Thus, by explicitly aligning the communication constraint information with the topological levels of the hierarchical topology tree, communication-sensitive parallel groups can be prevented from being incorrectly deployed to low-bandwidth interconnection levels, reducing the communication overhead of these parallel groups, increasing the effective bandwidth of the aggregate communication of these parallel groups, shortening communication time, and improving the throughput and execution efficiency of the computing power tasks. Moreover, by selecting the deployment scheme based on the fragmentation metric value of the computing cluster, the adverse effects of computing power fragmentation caused by the deployment of computing power tasks on the subsequent scheduling of computing power resources in the computing cluster can be mitigated. Attached Figure Description

[0027] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms provided by the present invention; Figure 2 This is a schematic diagram of a hierarchical topology tree for a computing cluster provided by the present invention; Figure 3 This is a flowchart of another intelligent computing cloud platform based on topology awareness for anti-fragmentation scheduling of computing power, provided by the present invention. Figure 4 This invention provides a comparison chart of cluster computing power resource distribution before and after computing power fragmentation. Figure 5 This is a schematic diagram of the overall architecture of an intelligent computing cloud platform based on topology-aware anti-fragmentation scheduling of computing power provided by the present invention; Figure 6 This invention provides a flowchart of topology-aware scheduling and computing power fragmentation. Figure 7 This is a structural diagram of a topology-aware computing power anti-fragmentation scheduling device for an intelligent computing cloud platform provided by the present invention; Figure 8 This is a structural diagram of an electronic device provided by the present invention. Detailed Implementation

[0029] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] The “computing power” mentioned in this invention refers to: the ability of computer equipment or computing / data center to process information; the ability of computer hardware and software to work together to perform a certain computing requirement; the computing power to achieve the target result output by processing information data; and a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, mainly providing services to society through computing power infrastructure.

[0031] The "computational power" (CP) described in this invention refers to the ability of a data center server to process data and output results. It is a comprehensive indicator of a data center's computing power, encompassing general computing power, supercomputing power, and intelligent computing power. The commonly used unit of measurement is floating-point operations per second (FLOPS, 1 EFLOPS = 10^18 FLOPS), with higher values ​​indicating stronger overall computing power. It is estimated that 1 EFLOPS is approximately the computing power output of 5 Tianhe-2A supercomputers, 500,000 mainstream server CPUs, or 2 million mainstream laptops. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0032] The "Network Power" (NP) mentioned in this invention refers to the performance of data transmission capability of computing facilities, which includes comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, and involves network transmission within and between data centers. It is a comprehensive indicator for measuring network transmission scheduling capability.

[0033] The "Storage Power" (SP) described in this invention refers to the comprehensive capabilities of a data center in four aspects: data storage capacity, performance, security and reliability, and green and low-carbon operation. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and internal storage devices within servers. The commonly used unit of measurement for storage capacity is exabytes (EB, 1EB = 2^60 bytes), while the commonly used unit of measurement for performance is the number of read / write operations per second (IOPS / TB). Disaster recovery ratio is an important indicator of security and reliability.

[0034] The "computing infrastructure" mentioned in this invention refers to a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity, enabling centralized computing, storage, transmission, and application of information.

[0035] The "new information infrastructure" mentioned in this invention refers to network infrastructure such as 5G networks, fiber optic broadband networks, backbone networks, international communication networks, and satellite internet; computing infrastructure such as data centers, general computing centers, intelligent computing centers, and supercomputing centers; and new technology facilities such as artificial intelligence, blockchain, and quantum computing.

[0036] The “computing power” mentioned in this invention includes: general computing power, intelligent computing power, and supercomputing power.

[0037] The "general computing power" mentioned in this invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0038] The "intelligent computing power" mentioned in this invention refers to: a computing platform deployed on a large scale based on dedicated chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and ASIC (Application Specific Integrated Circuit) for various artificial intelligence innovative applications, such as natural language processing and machine vision.

[0039] The “supercomputing power” mentioned in this invention refers to the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for computing in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0040] The "intelligent computing center" described in this invention refers to a facility that, through the use of large-scale heterogeneous computing resources, including general-purpose computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.), primarily provides the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models). The intelligent computing center encompasses facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0041] The "intelligent computing cloud platform" mentioned in this invention, abbreviated as "intelligent computing cloud", refers to a cloud computing platform that integrates hardware and software resources based on an intelligent computing center.

[0042] The "intelligent computing center" mentioned in this invention includes, but is not limited to, "smart computing center".

[0043] The "intelligent computing center" mentioned in this invention, also known as an artificial intelligence computing center, is a type of computing infrastructure that provides computing power services, data services, and algorithm services required for artificial intelligence applications, based on artificial intelligence theory and adopting an artificial intelligence computing architecture.

[0044] The "computing center" mentioned in this invention refers to a facility that is mainly composed of infrastructure such as wind, thermal, hydro, and electricity, and IT hardware and software equipment, and has computing power, carrying capacity, and storage capacity, including general data centers, intelligent computing centers, supercomputing centers, etc.

[0045] The "supercomputing center" mentioned in this invention refers to a supercomputing data center, which is a data center based on supercomputers or large-scale computing clusters. It can provide large-scale computing, storage and network services and is widely used in aerospace, defense, oil exploration, climate modeling and genome sequencing and other application scenarios.

[0046] The “computing resources” mentioned in this invention refer to the technologies and facilities required for the development of the digital society that have the ability to compute, transmit, store and apply information, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guaranteeing resources such as wind, fire, water and electricity.

[0047] The "model" mentioned in this invention includes, but is not limited to, "large language model" and "multimodal large model".

[0048] The "large language model" mentioned in this invention refers to a large-scale language model (LLM), which is a language model with a large number of parameters. It is designed to understand and generate human language, and is trained with a large amount of text data. It can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.

[0049] The “Multimodal Large Models” mentioned in this invention refer to models that combine multimodal information such as text, images, videos, and audio for training, including but not limited to multimodal large language models.

[0050] The “computing power running task” mentioned in this invention refers to a specific workload or job executed on computing power resources that requires a certain amount of computing power support, usually involving complex data processing, numerical calculation, model training or simulation scenarios.

[0051] It is important to emphasize that the difficulty of topology-aware anti-fragmentation scheduling of computing power in the intelligent computing cloud platform of this invention is far greater than that of existing scheduling schemes. The following is a detailed technical analysis of why the difficulty of topology-aware anti-fragmentation scheduling of computing power in the intelligent computing cloud platform of this invention is far greater than that of existing scheduling schemes: 1. The constraint matching dimensions are more refined.

[0052] Existing scheduling schemes only verify whether the required computing resources for a task are met, achieving only a coarse matching based on the quantity of computing resources. They completely ignore the differentiated communication needs of parallel groups within the task and fail to perform targeted matching and scheduling based on the cluster's physical topology. For highly communication-sensitive parallel groups, traditional scheduling easily leads to cross-node and cross-rack deployment issues. This invention proactively acquires the communication constraint information of the tasks to be scheduled, explicitly defining the hard constraint that highly communication-sensitive parallel groups must be deployed within a high-bandwidth topology subdomain. It precisely matches the high-level communication semantics of the computing tasks with the underlying physical topology of the cluster. Compared to the traditional single-quantity matching mechanism, it has more constraint dimensions and more refined matching logic, significantly increasing the difficulty of resource adaptation scheduling.

[0053] 2. Topology-aware scheduling has a more comprehensive dimension.

[0054] Existing scheduling schemes fail to recognize the hierarchical topology tree of the cluster, do not distinguish the differences in interconnect bandwidth between different subdomains, and cannot perceive the decisive impact of physical topology on the performance of aggregate communication. They only perform flat resource allocation, resulting in communication bandwidth differences of up to an order of magnitude for the same computing power running a task in different deployment locations, leading to extremely large fluctuations in training throughput. This invention performs scheduling calculations based on the complete hierarchical topology tree of the cluster, using the interconnect bandwidth threshold of topology subdomains as a deployment constraint. It specifically selects candidate deployment schemes suitable for the highly communication-sensitive parallel packet communication characteristics, achieving an upgrade from "flat resource allocation" to "structured topology precise deployment." The multi-layered logic superposition of topology parsing, subdomain selection, and bandwidth verification significantly improves the complexity of scheduling implementation.

[0055] 3. The quantitative evaluation dimensions of computing power fragments are more refined.

[0056] Existing scheduling schemes assess cluster fragmentation solely based on the proportion of unavailable computing resources. This only provides a macroscopic reflection of the overall strain on cluster computing resources, failing to characterize the discrete distribution of idle computing resources across hierarchical topological subdomains and completely ignoring the constraints of topological structure on fragmentation effectiveness. Consequently, the evaluation of computing fragmentation is singular in dimension and coarse in accuracy. This invention, before task deployment, traverses all candidate deployment schemes and combines the distribution of idle computing resources in each level of the hierarchical topological tree to pre-calculate the expected fragmentation metric for each scheme. It quantifies the level of computing fragmentation based on the actual dispersion of computing resources at the topological dimension, achieving a topology-aware, refined quantitative assessment of computing fragmentation. Compared to the coarse evaluation methods in related technologies that rely solely on the total amount of computing resources, the complex computational logic of this invention—including multi-scheme traversal evaluation, topological hierarchical resource statistics, and precise quantification of computing fragmentation—presents a significantly higher level of technical difficulty.

[0057] 4. The scheduling optimization dimensions are more forward-looking and more global.

[0058] Existing scheduling schemes focus solely on matching and deploying resources for the current computing power task, resulting in localized, passive, and immediate scheduling. They only require matching and verifying resources for a single computing power task, with simple calculation logic. They fail to consider the impact of this deployment on the overall topological integrity of the cluster and the long-term fragmentation of computing power, easily leading to fragmented high-bandwidth topological subdomains and subsequent large-scale computing power tasks being unable to deploy, resulting in long-term queuing and blocking. This invention integrates task deployment matching with cluster computing power fragmentation management upfront. It requires calculating fragmentation metrics for multiple candidate deployment schemes separately, and then filtering based on these metrics. It does not simply find a scheme that meets task communication constraints and then directly execute the deployment. While satisfying single-task topological communication constraints, it also considers the long-term integrity of the cluster's topological computing power resources, achieving optimal synergy between local task deployment effectiveness and global topological computing power resource utilization. This invention needs to simultaneously consider the constraints of real-time task deployment and the long-term fragmentation trend of cluster computing power. The optimization objectives are coupled in multiple dimensions. It requires traversing and evaluating multiple feasible candidate deployment schemes before making a decision. The computational overhead is large and the decision logic is complex. Therefore, the implementation difficulty is much higher than the scheduling schemes of existing technologies that only perform single computing power resource matching.

[0059] Existing scheduling schemes typically only guarantee that the amount of computing resources required by a task is met, failing to explicitly match the different interconnect bandwidth requirements of various parallel groups within the task with the hierarchical structure of the cluster's physical topology. This results in highly communication-sensitive parallel groups (such as tensor parallel groups) being deployed across nodes and racks, leading to reduced communication bandwidth, the introduction of additional communication latency, and decreased execution efficiency of the computing tasks. This invention addresses these problems.

[0060] Please see Figure 1 , Figure 1 This is a flowchart of a topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms provided by the present invention, as shown below. Figure 1 As shown, the method may include: Step S1: Obtain the computing power task to be scheduled; wherein, the computing power task to be scheduled carries communication constraint information.

[0061] In this invention, the communication constraint information can be the descriptive information carried along with the task when the task to be scheduled is submitted. The communication constraint information can be used to describe the topology deployment constraints of each parallel group within the task to be scheduled.

[0062] For example, the task to be scheduled for computing power can be a training task or an inference task, etc.

[0063] It should be noted that the scheduling method of this invention is applicable to any task that carries communication constraint information and includes parallel group deployment requirements (including but not limited to large model training tasks, inference deployment tasks with topological affinity requirements, fine-tuning tasks, etc.).

[0064] For example, parallel groups may include, but are not limited to, tensor parallel groups, pipeline parallel groups, data parallel groups, etc.

[0065] In some implementations, the communication constraint information may include deployment constraints for each parallel group of the task to be scheduled to run. Each parallel group may include at least a first parallel group. The deployment constraints for the first parallel group are used to indicate that the computing nodes that deploy the first parallel group must be located in the same subdomain of the hierarchical topology tree of the computing cluster with an interconnection bandwidth of not less than a first preset threshold.

[0066] For example, the computing node that deploys the first parallel group can refer to the computing node used to carry the first parallel group.

[0067] For example, the first parallel group may include, but is not limited to, at least one of tensor parallel group, expert parallel group, and sequence parallel group.

[0068] Among them, the tensor parallel group is a parallel group that partitions the model weight tensors and distributes them across multiple computing nodes; the expert parallel group is applied to the MoE (Hybrid Expert) model and can be used to distribute the various expert modules of the model, with high-frequency data routing interactions between expert modules; the sequence parallel group can be used to partition long sequence samples along the sequence dimension, sharing the memory pressure of a single GPU, with a large amount of activation data interaction between each partition. All of the above parallel groups involve high-frequency inter-GPU data interaction during task execution, placing high demands on the interconnect bandwidth between computing nodes. Therefore, they need to be constrained to be deployed within the same subdomain with an interconnect bandwidth not lower than a first preset threshold.

[0069] In some implementations, the communication constraint information may include deployment constraints for each parallel group of the task to be scheduled to run. Each parallel group includes at least a first parallel group and a second parallel group. The deployment constraints for the first parallel group are used to indicate that the computing nodes deploying the first parallel group must be located in the same subdomain of the hierarchical topology tree of the computing cluster with an interconnect bandwidth of not less than a first preset threshold. The deployment constraints for the second parallel group are used to indicate that the computing nodes deploying the second parallel group are allowed to traverse multiple subdomains of the hierarchical topology tree with an interconnect bandwidth of less than the first preset threshold.

[0070] For example, the second parallel group may include, but is not limited to, at least one of a data parallel group and a pipeline parallel group.

[0071] The data parallel group divides the data for the computing task into shards. Each computing node maintains a complete copy of the model and processes the corresponding data shard in parallel groups. Only a small amount of gradient or result aggregation synchronization occurs between nodes. The communication frequency between computing nodes is low, and the sensitivity to cluster interconnect bandwidth is low. The pipeline parallel group divides the model into multiple computing stages according to network layers. Different computing nodes carry a part of the model layer. Each node completes the computation in an alternating pipeline manner. Intermediate data is only passed between adjacent computing stages. Communication interaction is sparse, bandwidth pressure is low, and transmission across low-bandwidth topology links can be tolerated. Communication only occurs between adjacent pipeline stages. The inter-card communication pressure of the above two types of parallel groups is relatively low, and the deployment constraints are relaxed. Therefore, computing nodes deploying the second parallel group are allowed to cross multiple subdomains in the hierarchical topology tree with interconnect bandwidth below a first preset threshold.

[0072] In some implementations, the deployment constraints of the first parallel group can be set as hard constraints.

[0073] For example, the hierarchical topology tree of a computing cluster can be obtained by modeling the physical interconnect structure of the computing cluster.

[0074] The physical interconnect structure of computing clusters exhibits a naturally hierarchical characteristic: multiple accelerator cards within the same physical server can be directly interconnected via high-speed interconnect buses such as NVLink / NVSwitch, achieving bandwidths on the order of 900 GB / s; physical servers can be interconnected via Remote Direct Memory Access (RDMA) network cards through top-of-rack (ToR) switches, typically with bandwidths on the order of tens of GB / s; physical servers across racks require forwarding through the core / backbone switching layer (Spine), further reducing bandwidth and making them susceptible to the combined effects of multiple task traffic. Different parallel strategies exhibit orders of magnitude differences in their sensitivity to communication latency and bandwidth: tensor parallelism requires synchronizing intermediate computation results within each layer, resulting in frequent communication and being located on the computational critical path, making it extremely sensitive to bandwidth; taking training tasks as an example, data parallelism only synchronizes gradients once at the end of each training step, with a low communication frequency that can overlap with backpropagation computation, resulting in a significantly higher tolerance for communication latency.

[0075] For example, the hierarchical topology tree of a computing cluster can include, from bottom to top, a high-speed interconnect layer within a single machine, a node layer, an interconnect layer within a rack, and an interconnect layer across racks.

[0076] To facilitate understanding the hierarchical topology of a computing cluster, the following section combines... Figure 2 Please provide an explanation, such as Figure 2As shown, the hierarchical topology tree, from bottom to top, is as follows: Layer 0 is the high-speed interconnect layer within a single machine. This layer can include one or more subdomains (such as an NVLink domain containing 8 accelerator cards). The accelerator cards in each subdomain of Layer 0 are directly connected to each other via a high-speed interconnect bus, with a bandwidth of approximately 900GB / s; Layer 1 is the node layer. One or more subdomains of Layer 0 form a physical server node, for example... Figure 2 In the first layer, nodes A, B, C, and D are all physical server nodes. The second layer is the rack interconnect layer, where one or more physical server nodes can be deployed under a single top-of-rack switch (TOS). For example, node D is deployed under TOS 2. Multiple physical server nodes can be interconnected through TOS, such as nodes A, B, and C, which can be interconnected through TOS 1, with a bandwidth of approximately 50 GB / s. The third layer is the cross-rack interconnect layer (Spine) or core switching layer, where multiple TOR domains are interconnected. Figure 2 The top rack switch 1 and top rack switch 2 are interconnected through the core switching layer, which further reduces the bandwidth. Moreover, they are shared across racks for multiple tasks, making them prone to congestion.

[0077] For example, since tensor parallel groups communicate frequently, as a hard constraint, tensor parallel groups must be deployed within the same NVLink domain.

[0078] Because data parallel groups communicate at low frequencies and can overlap with computation, they can be deployed across nodes or racks. Therefore, as an example, deployment constraints for data parallel groups can be used to indicate that compute nodes deploying data parallel groups are allowed to span multiple subdomains with bandwidth below a first preset threshold.

[0079] As an example, the data structure for communication constraint information can include the task identifier of the task to be scheduled for computational power, the parallel strategy configuration (such as parameters like tensor parallelism, pipeline parallelism, and data parallelism), the group list corresponding to each parallel group, and the hierarchical requirement marker for each parallel group. The hierarchical requirement marker can include at least two types: "strong affinity markers" and "weak affinity markers." Strong affinity markers indicate that all computational nodes deploying the corresponding group must be located within the same subdomain of the k-th layer in the hierarchical topology tree (the specific value of k is configurable; for example, tensor parallel groups require k=0, such as the same NVLink domain). Weak affinity markers indicate that computational nodes deploying the corresponding group are allowed to span multiple subdomains of the k-th layer. Therefore, different hierarchical requirement markers can characterize the deployment constraints corresponding to each parallel group.

[0080] In some implementations, the communication constraint information may also be referred to as a communication shape declaration or other names. The present invention does not limit the name of the communication constraint information.

[0081] It should be noted that the first preset threshold in this invention can be determined according to actual needs, and this invention does not limit it.

[0082] The technical effect of step S1 is that by acquiring the computing power task to be scheduled carrying communication constraint information, the deployment constraints corresponding to the first parallel group can be obtained directly during the scheduling phase. This clarifies the bandwidth and subdomain deployment requirements of the computing node carrying the first parallel group on the hierarchical topology tree of the computing cluster, providing a basis for subsequent screening of candidate deployment schemes for the computing power task to be scheduled based on these deployment constraints.

[0083] Step S2: Based on the communication constraint information and the hierarchical topology tree, determine at least one candidate deployment scheme that meets the deployment constraints for the task to be scheduled.

[0084] In this invention, a candidate deployment scheme can refer to a deployment scheme that is a candidate for the task to be scheduled by the computing power and satisfies the deployment constraints of the first parallel group.

[0085] In some implementations, deployment schemes can be enumerated based on communication constraint information and hierarchical topology tree, provided that the deployment constraints of the first parallel group are met, to obtain at least one candidate deployment scheme for the task to be scheduled to run by computing power.

[0086] In some implementations, the communication constraint information may include deployment constraints for other parallel groups in addition to the deployment constraints for the first parallel group, such as the deployment constraints for the second parallel group. In this case, deployment schemes can be enumerated under the premise that the deployment constraints for the first parallel group and the deployment constraints for other parallel groups are satisfied simultaneously, so as to obtain at least one candidate deployment scheme for the task to be scheduled to run by computing power.

[0087] The technical effect of step S2 is as follows: By combining communication constraint information and the hierarchical topology tree of the computing cluster, candidate deployment schemes that are suitable for the task to be scheduled can be selected within the topology of the cluster based on the deployment constraints of the first parallel group. The deployment constraints on the task side are mapped to the actual physical topology dimension of the computing cluster, and candidate deployment schemes that meet the deployment constraints are obtained. This makes the communication constraint information and the topology hierarchy of the computing cluster explicitly aligned, providing an effective decision-making basis for the subsequent selection of deployment schemes.

[0088] Step S3: Calculate the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree.

[0089] In this invention, the first fragmentation metric corresponding to a candidate deployment scheme can be used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme. The first fragmentation metric can be regarded as the global fragmentation metric of the computing cluster. The larger the first fragmentation metric, the higher the expected degree of computing power fragmentation of the computing cluster; the smaller the first fragmentation metric, the lower the expected degree of computing power fragmentation of the computing cluster.

[0090] In some implementations, the first fragmentation metric corresponding to a candidate deployment scheme can also be understood as the expected fragmentation metric of the cluster calculated after executing the candidate deployment scheme.

[0091] In some implementations, for each candidate deployment scheme, the amount of idle computing resources in each subdomain of the hierarchical topology tree after adopting the candidate deployment scheme is estimated. Based on the estimated amount of idle computing resources in each subdomain of the hierarchical topology tree after adopting the candidate deployment scheme, a first fragmentation metric value corresponding to each candidate deployment scheme is calculated. For example, the amount of idle computing resources can be used to characterize the scale of schedulable computing resources within a subdomain or hierarchy. The amount of idle computing resources can be the number of idle accelerator cards, or at least one of the following: the number of idle CPU cores, idle storage capacity, and idle network bandwidth resources. For example, the amount of idle computing resources in a subdomain can be the number of idle accelerator cards within that subdomain, and the amount of idle computing resources in a hierarchy can refer to the number of idle accelerator cards within that hierarchy. For example, an accelerator card can be a hardware board with large-scale parallel computing capabilities, which can be used to carry out computational tasks such as large model training or inference, including AI processing boards such as GPUs and NPUs.

[0092] For example, the total amount of idle computing resources in each level can be determined based on the amount of idle computing resources in each subdomain, and the first fragmentation metric corresponding to the candidate deployment scheme can be determined based on the total amount of idle computing resources in each level.

[0093] The technical effect of step S3 is that by calculating the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree, the expected degree of computing power fragmentation that the computing cluster will form after executing the candidate deployment scheme can be quantitatively evaluated. The impact of the candidate deployment scheme on the cluster computing power fragmentation is transformed into a quantitative indicator, which facilitates the subsequent evaluation of different candidate deployment schemes based on the first fragmentation metric value.

[0094] Step S4: Select the candidate deployment scheme that satisfies the preset optimization target as the target deployment scheme, and deploy the computing power to be scheduled to run tasks according to the target deployment scheme.

[0095] In some implementations, the preset optimization objective can be to minimize the first fragmentation metric of the compute cluster after the candidate deployment scheme is executed. In other words, the candidate deployment scheme with the smallest first fragmentation metric can be selected as the target deployment scheme.

[0096] It should be noted that the above-mentioned preset optimization target is only an example. For example, the preset optimization target could also be that the first fragment metric value is less than a set metric value, etc. The specific target can be determined according to actual needs, and this invention does not limit it.

[0097] The technical effect of step S4 is that, under the premise of meeting the deployment constraints of the first parallel group, a candidate deployment scheme that meets the preset optimization objective is selected to deploy the scheduled computing power to run the task. Thus, the deployment scheme is selected and the task is deployed based on the fragmentation metric, which can reduce the adverse impact of computing power fragmentation on the subsequent computing power resource scheduling of the computing cluster caused by task deployment.

[0098] In this invention, by acquiring the scheduled computing power tasks carrying communication constraint information, and based on the communication constraint information and the hierarchical topology tree of the computing cluster, candidate deployment schemes that meet the first parallel group deployment constraint conditions are determined. Furthermore, based on the amount of idle computing power resources in each subdomain of the hierarchical topology tree, a first fragmentation metric value corresponding to each candidate deployment scheme is calculated to characterize the expected fragmentation level of the cluster's computing power after deployment. Finally, a target deployment scheme is selected based on a preset optimization objective to complete the deployment of the scheduled computing power tasks. Thus, by explicitly aligning the communication constraint information with the topological levels of the hierarchical topology tree, communication-sensitive parallel groups can be prevented from being incorrectly deployed to low-bandwidth interconnection levels, reducing the communication overhead of these parallel groups, increasing the effective bandwidth of the aggregate communication of these parallel groups, shortening communication time, and improving the throughput and execution efficiency of the computing power tasks. Moreover, by selecting the deployment scheme based on the fragmentation metric value of the computing cluster, the adverse effects of computing power fragmentation caused by the deployment of computing power tasks on the subsequent scheduling of computing power resources in the computing cluster can be mitigated.

[0099] In one embodiment of the present invention, step S3 above can be adopted. Figure 3 The method shown is implemented as follows: Figure 3 This is a flowchart of another intelligent computing cloud platform based on topology-aware anti-fragmentation scheduling method provided by the present invention, as shown below. Figure 3 As shown, step S3 above includes: Step S3.1: Determine the total amount of idle computing resources for each level based on the amount of idle computing resources in each subdomain of each level in the hierarchical topology tree, and determine the proportion of idle computing resources in each subdomain based on the amount of idle computing resources in each subdomain and the total amount of idle computing resources.

[0100] In this invention, when calculating the first fragmentation metric value corresponding to the candidate deployment scheme, the amount of idle computing resources in each subdomain of each level in the hierarchical topology tree can refer to the estimated amount of idle computing resources in each subdomain of each level in the hierarchical topology tree after adopting the candidate deployment scheme.

[0101] For example, the total amount of idle computing resources in each level can be determined by summing the amount of idle computing resources in each subdomain, and the proportion of idle computing resources in each subdomain can be determined by the ratio of the amount of idle computing resources in each subdomain to the total amount of idle computing resources in the level.

[0102] As an example, for the first layer of a hierarchical topology tree Layer, assuming that the layer has a total of Subdomains D1, D2, ..., D , No. i The amount of idle computing resources in each subdomain is denoted as Then the first The total amount of idle computing resources in the layer is The following formula (1) can be used to calculate the first... i Percentage of idle computing resources in each subdomain: (1) Understandably, when This level is not included in the calculation.

[0103] The technical effect of step S3.1: By obtaining the total amount of idle computing resources of each subdomain under each level in the hierarchical topology tree, the total amount of idle computing resources of the corresponding level is obtained. By combining the amount of idle computing resources of each subdomain with the total amount of idle computing resources of the level, the proportion of idle computing resources of each subdomain can be calculated. This can normalize the idle computing resources of each subdomain and quantify the relative distribution of idle computing resources among different subdomains within the same level, providing a standardized calculation basis for the subsequent calculation of fragmentation metric values.

[0104] Step S3.2: Calculate the information entropy of the level as the fragmentation entropy of the level based on the proportion of idle computing resources in each subdomain.

[0105] In this invention, the fragmentation entropy of a level can be used to characterize the degree of computing power fragmentation at that level. The larger the fragmentation entropy of a level, the higher the degree of computing power fragmentation at that level; the smaller the fragmentation entropy of a level, the lower the degree of computing power fragmentation at that level.

[0106] In some implementations, the fragmentation entropy of a hierarchy can also be used to characterize the degree of dispersion of idle computing resources within that hierarchy across its subdomains. The larger the fragmentation entropy of a hierarchy, the more dispersed the idle computing resources are across its subdomains; conversely, the smaller the fragmentation entropy of a hierarchy, the more concentrated the idle computing resources are across its subdomains.

[0107] In some implementations, the fragmentation entropy of a level can be determined using the Shannon information entropy method, based on the proportion of idle computing resources in each subdomain.

[0108] For example, the information entropy of each level can be equal to the negative sum of the products of the proportion of idle computing resources in each subdomain of that level and the logarithm of the proportion of idle computing resources in that level.

[0109] As an example, the following formula (2) can be used to calculate the first... Fragmentation entropy of the layer: (2) in, This indicates the summation process iterating through the first... Subdomains within a layer where the amount of idle computing resources is greater than 0. The range of values ​​for is [0, ]: When the When all the idle computing resources of a layer are concentrated in a single subdomain (i.e., a certain subdomain exists) i make ), This indicates that the fragmentation of computing power in this layer is the lowest, and that at least one fully usable subdomain exists in this layer; when idle computing power resources are evenly distributed across all... When there are subdomains, Get the maximum value This indicates that the computing power of this layer is most fragmented.

[0110] The technical effect of step S3.2: By calculating the proportion of idle computing resources in each subdomain, the fragmentation entropy of the corresponding level can be obtained. This can transform the distribution of idle computing resources in multiple subdomains within the same level into a quantifiable value, which intuitively reflects the degree of computing fragmentation of the resources at that level and provides a basis for calculating the first fragmentation metric value at the level.

[0111] Step S3.3: Based on the weights of each level in the hierarchical topology tree, weight the fragment entropy of each level to determine the first fragment metric.

[0112] Considering that different levels of the hierarchical topology tree have different impacts on computing power tasks, the lower the level (such as the NVLink domain), the more critical its complete availability is to communication-sensitive parallel packet processing. Therefore, in some implementations, a weight can be introduced for each level. Based on the weight of each level in the hierarchical topology tree, the fragmentation entropy of each level is weighted to obtain the first fragmentation metric.

[0113] For example, the weight of each level can be negatively correlated with the depth of each level in the hierarchical topology tree. The depth of a level in the hierarchical topology tree can refer to which level the level belongs to in the hierarchical topology tree; the lower the level, the smaller the depth. That is, the weight can increase as the level decreases, i.e., the lower the level, the greater the weight of the level.

[0114] In this embodiment, the weight of each level is negatively correlated with the depth of each level in the hierarchical topology tree. This allows for differentiated weighting of fragment entropy at different levels, giving higher weights to levels that are closer to the bottom high-speed interconnect layer and have a smaller depth. When calculating fragmentation metrics, the focus is on the fragmentation status of the high-speed interconnect layer, weakening the impact of computing power fragmentation at the upper levels on the overall fragmentation metric result. This makes the fragmentation metric more closely reflect the actual level of attention that communication-sensitive parallel packets pay to the resources of the bottom high-speed interconnect layer.

[0115] In some implementations, the total amount of idle computing resources at each level can also be introduced as a modulation factor.

[0116] For example, step S3.3 above may include: Step S3.3.1: Based on the weight of each level and the total amount of idle computing resources at each level, the fragment entropy of each level is weighted to determine the first fragment metric value.

[0117] As an example, the fragmentation entropy of each level can be multiplied by the total amount of idle computing resources to obtain the corresponding product result for each level. Then, according to the weight of each level, the product results corresponding to each level are weighted to obtain the first fragmentation metric.

[0118] The technical effect of step S3.3.1: By using the total amount of idle computing resources at each level as a modulation factor in the weighted calculation, the contribution of levels with smaller total amounts of idle computing resources to the overall fragmentation measurement result can be reduced, avoiding excessive interference of the fragmentation entropy of such levels on the first fragmentation measurement value, so that the obtained first fragmentation measurement value is more in line with the actual degree of impact of computing power fragmentation in the cluster.

[0119] In related technologies, the fragmentation of computing power is typically measured only by the proportion of unavailable resources. This indicator only reflects whether the total amount of resources is tight and cannot reflect the distribution of idle computing resources among the subdomains in the hierarchical topology tree. Based on this, this invention introduces a fragmentation metric to characterize the degree of computing power fragmentation of the computing cluster. This metric can reflect the distribution characteristics of idle computing resources in the topological subdomains and improve the accuracy of measuring the degree of computing power fragmentation of the computing cluster.

[0120] The technical effect of step S3.3: By utilizing the weights corresponding to each level in the hierarchical topology tree, the fragmentation entropy of each level is weighted, which can integrate the computing power fragmentation information represented by different levels of the topology tree into a unified first fragmentation metric value. This takes into account the importance of different topology levels in the cluster computing power resource scheduling, and obtains a quantitative indicator that can comprehensively reflect the overall expected computing power fragmentation degree of the cluster after executing the candidate deployment scheme. This facilitates horizontal comparison of the computing power fragmentation degree between different candidate deployment schemes.

[0121] In this embodiment of the invention, the proportion of idle computing resources in each subdomain is calculated based on the amount of idle computing resources in each level of the hierarchical topology tree. Further, based on the proportion of idle computing resources, fragmentation entropy, which characterizes the degree of computing power fragmentation at each level, is calculated. The fragmentation entropy is then weighted and fused with the corresponding weights at each level to obtain the first fragmentation metric. This transforms the distribution of idle computing resources in each subdomain into a unified quantitative indicator, comprehensively considering the impact of different levels on the cluster's computing power fragmentation state. This improves the accuracy of the first fragmentation metric and provides complete and quantitative data support for subsequent selection of candidate deployment schemes based on the first fragmentation metric.

[0122] After a cluster is deployed with multiple computing power tasks running in a mixed manner for a long period of time, computing power fragmentation may occur. That is, the total amount of idle computing power resources may seem sufficient, but they are scattered across multiple physical domains. No single domain can provide the complete resource block required for large-scale computing power tasks, resulting in continuous queuing of large-scale computing power tasks. Usually, the only solution is to passively wait for the computing power tasks to finish naturally to release resources, or rely on manual migration by maintenance personnel, which is slow and inefficient. Based on this, in some embodiments of the present invention, the above method may further include: Step S5: Continuously monitor the current second fragmentation metric value of the computing cluster. If the second fragmentation metric value exceeds the second preset threshold, select candidate computing tasks that meet the migration conditions from the currently running computing tasks. Step S6: Identify the computing power tasks to be migrated from the candidate computing power running tasks; Step S7: Save the status information of the computing power to be migrated running tasks, and migrate the computing power to be migrated running tasks from the current deployment location to the target location.

[0123] In some implementations, the current second fragmentation metric of the computing cluster can be continuously calculated at fixed intervals or in an event-triggered manner.

[0124] In some implementations, the calculation method for the second fragmentation metric is the same as that for the first fragmentation metric, so it will not be repeated here.

[0125] For example, when calculating the current second fragmentation metric of the cluster, the amount of idle computing resources of each subdomain in each level of the hierarchical topology tree is the amount of idle computing resources of each subdomain in each level of the hierarchical topology tree of the cluster. Therefore, the total amount of idle computing resources in each level is also the total amount of idle computing resources in each level.

[0126] For example, the total amount of idle computing resources at each level can be introduced as a modulation factor. Based on the weight of each level and combined with the total amount of idle computing resources at each level, the fragmentation entropy of each level is weighted to obtain the second fragmentation metric.

[0127] As an example, computing clusters in t The second fragment metric at time t can be calculated using the following formula (3): (3) in, Indicates that the computing cluster is in t The second fragmentation metric at time 1, Indicates the first l Layer weights Indicates the first l Layer in t Entropy of moments Indicates the first l Layer in t The total amount of idle computing resources at any given time.

[0128] Therefore, by using the total amount of idle computing resources at each level as a modulation factor in the weighted calculation, we can avoid the problem of triggering unnecessary subsequent fragmentation of computing resources when there are few idle computing resources at a certain level, thus saving computing resource costs.

[0129] In some implementations, the currently running computing task may refer to the computing task that is being executed.

[0130] In some implementations, migration conditions may include, but are not limited to, at least one of the following: the resource consumption of the task is less than a third preset threshold (i.e., a small task), the task supports checkpoint saving and restoration, the estimated remaining runtime of the task is less than a fourth preset threshold, and the task is not currently in an execution phase requiring a latency of less than a fifth preset threshold. Here, requiring a latency of less than the fifth preset threshold can be understood as being latency-sensitive.

[0131] It should be noted that the second preset threshold, the third preset threshold, the fourth preset threshold, the fifth preset threshold, etc. in the embodiments of the present invention can all be determined according to actual needs, and the present invention does not limit them.

[0132] In some implementations, the currently running computing power operation task may include training tasks and / or inference tasks, and correspondingly, the candidate computing power operation tasks may include training tasks and / or inference tasks.

[0133] In some implementations, all candidate computing power running tasks can be treated as computing power running tasks to be migrated.

[0134] In some implementations, step S6 may also include: Step S6.1: Determine the migration cost and benefits of candidate computing power running tasks; Step S6.2: Based on the migration cost and migration benefits of the candidate computing power running tasks, determine the computing power running tasks to be migrated from the candidate computing power running tasks.

[0135] For example, the migration cost of candidate computing power running tasks can be used to quantitatively characterize the system overhead required to migrate candidate computing power running tasks.

[0136] For example, the migration cost can be determined based on the time required for saving and restoring the state of the candidate computing power task and the amount of data to be migrated. The time required for saving and restoring the state can be obtained by statistical analysis based on the current running state scale, model parameter scale, and number of parallel groups within the task. The amount of data to be migrated can include all data that needs to be moved along with the computing power task, such as model parameters that need to be persisted during the operation of the computing power task.

[0137] As an example, the migration cost corresponding to the candidate computing power running task can be obtained by normalizing the time required for state saving and restoration and the amount of data to be migrated, and then performing a weighted sum.

[0138] For example, migration benefits can refer to the resource value of the complete topology domain that can be released after the candidate computing power runs the task and migrates. Here, the complete topology domain can refer to a complete subdomain.

[0139] As an example, the migration benefit of the candidate computing power task can be obtained by multiplying the base resource value of the complete topology domain released after the migration by the weight of the level to which the complete topology domain belongs. This migration benefit reflects the topology resource integrity gain that the computing cluster can obtain after the computing power task is migrated. The higher the level weight of the subdomain, the more beneficial it is for the released complete topology domain to improve the probability of subsequent scheduled computing power tasks matching resources that meet the deployment constraints.

[0140] For example, the difference between the migration benefit and migration cost of the candidate computing power running task can be calculated, and the computing power running task to be migrated can be determined from the candidate computing power running tasks based on the difference.

[0141] As an example, candidate computing power running tasks with a difference greater than zero can be used as computing power running tasks to be migrated.

[0142] As another example, candidate computing power tasks whose difference is greater than a set threshold can be used as computing power tasks to be migrated. Here, if the difference is greater than the corresponding set threshold, it can be considered that the migration benefit is significantly greater than the migration cost. That is, candidate computing power tasks whose migration benefit is significantly greater than the migration cost can be used as computing power tasks to be migrated. This can avoid migration behavior where the benefit is not enough to offset the cost, and can reduce the computation and time loss caused by unnecessary state saving and computing power task migration.

[0143] In some implementations, the state information of the task to be migrated includes checkpoint data for the training task or session state data for the inference task. For example, if a task to be migrated is a training task, then the state information of the training task includes checkpoint data for the training task; if a task to be migrated is an inference task, then the state information of the inference task includes session state data for the inference task.

[0144] The technical effect of step S6.1: By determining the migration cost and benefits of candidate computing power running tasks, the system overhead to be paid for the migration operation of computing power running tasks and the improvement effect of computing power fragmentation after migration can be transformed into quantifiable indicators, and the advantages and disadvantages of task migration can be transformed into comparable values, providing a quantitative evaluation basis for subsequent selection of computing power running tasks suitable for migration.

[0145] The technical effect of step S6.2: By comprehensively considering the migration cost and benefits of candidate computing power running tasks, the computing power running tasks to be migrated can be determined. This enables differentiated identification of candidate computing power running tasks, and selects suitable computing power running tasks to be migrated, avoiding blindly performing migration on candidate computing power running tasks and reducing unnecessary system overhead.

[0146] In some implementations, checkpoint saving technology can be used to save the status information of the computing power running tasks to be migrated, and migrate the computing power running tasks to be migrated from the current deployment location to the target location, thereby increasing the proportion of idle computing power resources in the subdomain where the current location is located, and gradually merging subdomains where the proportion of idle computing power resources reaches a preset integrity threshold.

[0147] To facilitate understanding of the computing power fragmentation method of the present invention, the following is combined with... Figure 4 To explain, Figure 4 This is a comparison chart of the distribution of cluster computing resources before and after the consolidation of computing power fragments, provided by the present invention.

[0148] like Figure 4 As shown, suppose there are four servers (A1, A2, A4) on a rack (i.e., the top-of-rack switch) in a cluster, each equipped with 8 accelerator cards. Each server's 8 accelerator cards correspond to one NVLink domain, meaning the number of subdomains is 4. The number of idle accelerator cards on the first four servers are 2, 3, 1, and 2 respectively. The total amount of idle computing resources... The proportion of idle computing resources in each subdomain is p=(0.25, 0.375, 0.125, 0.25). Substituting this into the above formula (2), we can calculate... Approaching the theoretical maximum value of this layer This indicates that idle computing resources are highly dispersed, making it impossible to form a complete 8-card NVLink domain. If a computing task requests an 8-card tensor parallel group at this time, it will continue to queue. Based on the migration conditions and the migration cost and benefits of computing tasks, among the four nodes, node A2 has the fewest currently running computing tasks (its occupied computing resources account for 5 / 8, making it the node with the lowest occupied and highest idle computing resources among the four), and the lowest migration cost. After the online computing fragmentation process of this invention migrates and distributes the 5 running computing tasks on node B to the idle slots of nodes A1, A3, and A4 (accommodating 2, 1, and 2 slots respectively, exactly filling the remaining idle computing resources of the three nodes), node A2 has 8 / 8 idle resources, and the other three nodes have 0 / 8 idle resources. At this time, the idle computing resources ratio of each subdomain is p=(0, 1, 0, 0). Node A2 becomes a fully usable NVLink domain, ready to immediately satisfy the newly arrived 8-card tensor parallel task.

[0149] The technical effect of step S5 is as follows: By continuously monitoring the current second fragmentation metric of the computing cluster, and when the second fragmentation metric exceeds the second preset threshold, candidate computing tasks that meet the migration conditions are selected from the running computing power tasks. This allows for real-time perception of changes in the computing power fragmentation status of the cluster. When the second fragmentation metric reaches the second preset threshold, the computing power fragmentation process is initiated, thereby starting the task migration process. This avoids meaningless task scanning and reduces the system processing overhead caused by routine monitoring.

[0150] The technical effect of step S6 is that by further identifying the computing power running tasks to be migrated among the candidate computing power running tasks, the feasibility of migrating each candidate computing power running task can be comprehensively evaluated. The computing power running tasks suitable for migration can be identified from the candidate computing power running tasks that meet the migration conditions, preventing the direct migration of all candidate computing power running tasks and reducing the ineffective system overhead caused by subsequent migration operations.

[0151] The technical effect of step S7: By saving the status information of the computing power running tasks to be migrated and migrating the computing power running tasks from the current deployment location to the target location, while ensuring the continuous operation of computing power running tasks as much as possible, the topology resources corresponding to the original deployment location are released, thereby organizing a complete topology domain that can be used by subsequent large-scale computing power running tasks, which can significantly shorten the queuing waiting time of large-scale computing power running tasks.

[0152] In related technologies, resource pool elastic scaling mechanisms often rely on preset static utilization thresholds to trigger scaling up or down, resulting in decision lag and jitter issues. Therefore, in some embodiments of the present invention, the above method may further include: Step S8: Based on the load profile over a preset period of time, predict the increase in resource demand for the first resource pool at a future target time. Step S9: Before the target time, migrate candidate physical devices that are currently idle or whose computing power utilization rate is lower than the sixth preset threshold in the second resource pool to the first resource pool.

[0153] In some implementations, the load profile can be constructed based on historical load data (such as hourly training or inference request time series) over a preset period of time. For example, the preset period of time can be one week, one month, or other durations, which can be determined according to actual needs.

[0154] In some implementations, the first resource pool may be a training resource pool, and the second resource pool may be an inference resource pool; or, the first resource pool may be an inference resource pool, and the second resource pool may be a training resource pool. Here, the training resource pool can refer to a resource pool containing the resources required for the training task, and the inference resource pool can refer to a resource pool containing the resources required for the inference task.

[0155] In some implementations, the target time may be a target moment or a target time period, etc., and the present invention does not limit it.

[0156] In some implementations, computing power utilization rate can refer to the proportion of computing power resources occupied on a physical device to its total computing power resources, which can be used to measure the busyness of the physical device; if the computing power utilization rate is lower than the sixth preset threshold, it can indicate that the physical device has a lot of idle computing power resources and is suitable as a candidate physical device to perform cross-resource pool migration.

[0157] For example, the intelligent computing platform is mainly loaded with inference services during the day and mainly with training tasks at night. Based on the load profile of the past week, it is predicted that the demand for training tasks will increase significantly at 2:00 a.m. the next day. Therefore, the tidal migration can be started in advance at 1:30 a.m. the next day to migrate the whole machine equipment in the inference resource pool that is at a low computing power utilization rate during the day to the training resource pool.

[0158] As an example, based on the load profile over a preset period of time, the future resource demand changes of the training and inference resource pools can be predicted. Before the resource demand changes occur, physical devices that are currently idle or whose computing power utilization is below a sixth preset threshold can be migrated between the training and inference resource pools. These resource demand changes can include increases or decreases in resource demand. For instance, if the resource demand of the training resource pool is predicted to decrease at a certain time in the future, before that time, candidate physical devices that are currently idle or whose computing power utilization is below the sixth preset threshold in the training resource pool can be migrated to the inference resource pool.

[0159] The technical effect of step S8: By predicting the increase in resource demand of the first resource pool in the future target time based on the load profile within the past preset time period, it is possible to explore the resource change patterns contained in the historical load, identify the subsequent resource demand of the resource pool in advance, and provide a basis for the pre-migration of physical devices.

[0160] The technical effect of step S9 is that by migrating candidate physical devices that are idle or whose computing power utilization is lower than the sixth preset threshold from the second resource pool to the first resource pool in advance before the target time, the pre-allocation of resources can be completed. This avoids the lag and jitter caused by passively responding only after the actual resource demand occurs, and allows the resource pool to prepare available physical devices in advance, thereby improving the resource pool's resource provision response capability when facing upcoming resource demands.

[0161] When preempting running computing power tasks due to resource scarcity, the state preservation cost of the preemption behavior itself is often not included in the quantitative consideration of scheduling decisions, which may lead to irrational scheduling where the preemption benefit is less than the preemption cost. Therefore, in some embodiments of the present invention, step S9 may include: Step S9.1: If the physical device to be migrated among the candidate physical devices is running a first computing power running task, save the status information of the first computing power running task before performing preemptive scheduling on the first computing power running task. Step S9.2: Determine the preemption cost for the task to be scheduled to run by the first computing power based on the time and resource overhead of the save operation. Step S9.3: Based on the difference between the migration benefits and the preemption cost of the physical device to be migrated, determine whether to preempt the scheduling of the first computing power to run the task and migrate the physical device to be migrated to the first resource pool.

[0162] In some implementations, the first computing power operation task may refer to the computing power operation task being executed in the physical device to be migrated. The first computing power operation task may be a running training task or a running inference task, etc., and there is no limitation thereto.

[0163] In some implementations, candidate physical devices can be traversed. Currently idle candidate physical devices can be directly migrated to the first resource pool, while candidate physical devices with computing power utilization rates lower than the sixth preset threshold may have deployed first computing power running tasks. Candidate physical devices with deployed first computing power running tasks can be called physical devices to be migrated. If a physical device to be migrated among the candidate physical devices has deployed a first computing power running task, the status information of the first computing power running task can be saved before preemptive scheduling is performed on the first computing power running task, so as to continue executing the running task later.

[0164] For example, the saving operation performed on the state information of the first computing power running task before the preemption scheduling can be used to retain the running context of the first computing power running task so that the first computing power task can resume execution after migration.

[0165] For example, this save operation can be implemented using incremental checkpoint saving. Incremental checkpoint saving only saves the changed task state, which can significantly reduce saving time and storage overhead.

[0166] It should be noted that incremental checkpoint saving is an exemplary implementation of this saving operation, that is, the saving of the running state of the computing power running task is completed through the incremental checkpoint saving mechanism. This invention does not limit the specific implementation of the state saving operation.

[0167] In some implementations, the sum of the time and resource overhead of the save operation can be used as the preemption cost for scheduling the first computing power task. The time overhead can be the result of normalizing the duration of the save operation, and the resource overhead can be the result of normalizing the amount of resources consumed by the save operation. After normalization, the time and resource overhead are on the same order of magnitude, allowing for direct summation.

[0168] In some implementations, the migration benefit can refer to the gain brought about by migrating the physical device to be migrated across resource pools (e.g., between a training resource pool and an inference resource pool). It can characterize the overall benefit obtained by the computing cluster's resource scheduling layer after the migration of the device. For example, it can include the resource supply gain brought by the migrated physical device to the target resource pool (e.g., the first resource pool), the scheduling benefit from alleviating the supply-demand gap in the target resource pool, and the benefit from avoiding response lag caused by passive scaling after demand changes. This migration benefit can be normalized using the same method as the total cost to ensure that the two are on the same order of magnitude and can be directly compared.

[0169] In some implementations, since restarting a task after an interruption of the computing power operation task will also introduce losses, the total cost can be determined based on the sum of the preemption cost and the task interruption cost of the first computing power operation task. The decision on whether to preempt the scheduling of the first computing power operation task and migrate the physical device to be migrated to the first resource pool can be made based on the difference between the migration benefit corresponding to the migration of the physical device to be migrated and the total cost.

[0170] For example, the task interruption cost can represent the additional loss introduced by the subsequent recovery of the first computing power running task after it is preempted and interrupted. It can include the training progress loss caused by the interruption of the first computing power running task, the additional processing overhead during the task recovery phase, etc.

[0171] As one possible implementation, if the difference between the migration benefit and the total cost is greater than zero, the first computing power task can be preemptively scheduled and the physical device to be migrated can be migrated to the first resource pool; if the difference is less than or equal to zero, the first computing power task will not be preemptively scheduled, the current deployment of the first computing power task will be maintained, and the search for a preemptive target will continue among other physical devices to be migrated.

[0172] As another possible implementation, if the difference is greater than the set value, the first computing power task can be preempted and the physical device to be migrated can be migrated to the first resource pool; if the difference is less than or equal to the set value, the first computing power task will not be preempted, the current deployment of the first computing power task will be maintained, and the search for a preemption target will continue among other physical devices to be migrated.

[0173] For example, the time and resource overhead of the save operation can be explicitly included in the cost function of the scheduling decision as the preemption cost. Taking the save operation implemented using incremental checkpoint saving as an example, the time and resource overhead of incremental checkpoint saving is explicitly included in the cost function of the scheduling decision as the preemption cost, as shown in the following formula (4): (4) in, This represents the decision value, used to determine whether to preempt scheduling and device migration for currently running computing tasks; This represents the migration benefit corresponding to the physical device to be migrated, and is used to characterize the scheduling benefit obtained by the computing cluster after the physical device to be migrated is completed. This represents the overhead of saving checkpoints, and characterizes the total overhead of performing incremental checkpoint saving. This represents the cost of task interruption.

[0174] The checkpoint saving overhead here includes the time overhead and resource overhead of performing incremental checkpoint saving. The time overhead is the cost corresponding to the execution time of incremental checkpoint saving, and the resource overhead is the cost corresponding to the storage, IO and other resources consumed in the execution process of incremental checkpoint saving. The execution time and resources consumed are normalized and then merged to obtain the checkpoint overhead.

[0175] For example, if an intelligent computing platform primarily handles inference service loads during the day and training task loads at night, and based on the load profile of the past week predicts a significant increase in training task demand at 2:00 AM the following day, a tidal migration can be initiated at 1:30 AM to migrate machines with low computing power utilization in the daytime inference resource pool to the training resource pool. For a physical machine still processing a small number of long-tail inference tasks, an incremental checkpoint is saved for that inference task before preemption. After factoring in the cost function, if the migration benefit is determined to be significantly greater than the cost, inference task preemption and physical machine migration are executed. After migration, the inference task can continue to be processed almost without loss on the new machine or other machines based on the saved checkpoint data.

[0176] The technical effect of step S9.1 is that before preempting the scheduling of the first computing power running task on the physical device to be migrated, the state information of the first computing power running task is saved in advance. This can preserve the running context of the first computing power running task and provide a basis for the resumption of running after the first computing power running task is preempted, thereby reducing the risk of task running failure caused by direct preemption.

[0177] The technical effect of step S9.2 is that by determining the preemption cost of scheduling the first computing power to run the task based on the time and resource overhead of the save operation, the overhead of saving the task state before preemption can be converted into the preemption cost, providing a basis for whether to preempt and migrate in the future.

[0178] The technical effect of step S9.3 is that by comprehensively considering the migration benefits and preemption costs of the physical devices to be migrated, it is decided whether to preemptively schedule the first computing power to run the task and execute the device migration, thus avoiding blind preemption and migration and reducing the ineffective overhead caused by irrational preemption operations.

[0179] In this embodiment of the invention, before preempting the computing power task occupying the device to be migrated, the state of the computing power task is actively saved. The time of the saving operation and the preemption cost are explicitly included in the scheduling decision as a quantitative basis for whether to perform preemption scheduling, which can avoid irrational preemption decisions.

[0180] In some embodiments of the present invention, the above method may further include: Step S10: For the second computing power running task waiting in the queue, generate and output the first candidate solution and its corresponding estimated completion time, the second candidate solution and its corresponding estimated completion time.

[0181] Among them, the first candidate scheme is to wait for the computing power fragments to be reorganized and then deploy the second computing power running tasks waiting in the queue according to the deployment constraints of the second computing power running tasks waiting in the queue. The second candidate scheme is to use a degraded topology scheme to deploy the second computing power running tasks waiting in the queue in real time.

[0182] For example, the second computing power can run a training task or an inference task, without limitation.

[0183] For example, a degraded topology scheme could be a first parallel group of tasks waiting in the queue to run across two subdomains, accepting a certain loss of communication efficiency.

[0184] The technical effect of step S10 is to generate and output two candidate schemes and their corresponding estimated completion times for computing power tasks that are in a queue, which can provide multiple optional deployment schemes for scheduling strategies or upper-level decisions and meet different processing needs.

[0185] To facilitate understanding of the topology-aware computing power anti-fragmentation scheduling method for the intelligent computing cloud platform in this embodiment of the invention, the following is combined with... Figure 5 and Figure 6 To explain, Figure 5 This is a schematic diagram of the overall architecture of an intelligent computing cloud platform based on topology-aware anti-fragmentation scheduling of computing power, provided by the present invention. Figure 6 This invention provides a flowchart for topology-aware scheduling and computing power fragmentation.

[0186] like Figure 5 As shown, the overall architecture of this invention includes a task layer, a scheduling core component layer, and a topology and migration foundation layer. The task layer is responsible for receiving tasks to be scheduled using computing power, carrying communication constraint information. The scheduling core component layer includes a topology matching solution module, a fragment entropy monitoring module, a computing power fragment sorting and orchestration module, and a tidal elastic control module. The topology and migration foundation layer maintains a hierarchical topology tree and provides migration base capabilities such as incremental checkpoints to support low-cost migration of computing power tasks.

[0187] The topology matching solution module is used to determine candidate deployment schemes that meet the preset deployment conditions of the first parallel group based on communication constraint information and hierarchical topology tree, and select the target deployment scheme that meets the preset optimization objective from the candidate deployment schemes; the fragmentation entropy monitoring module is used to monitor the fragmentation metric value of the computing cluster; the fragmentation sorting and orchestration module is used to sort the computing power fragments of the computing cluster when the current fragmentation metric value of the computing cluster is greater than the second preset threshold; the tidal elastic control module is used to predict the demand changes of the resource pool based on the load profile of the past preset duration, and before the demand rises in the first resource pool at the predicted future target time, migrate the currently idle or low computing power utilization physical devices in the second resource pool to the first resource pool.

[0188] like Figure 6 As shown, the topology-aware scheduling and computing power fragmentation process includes: a. Obtain the communication constraint information carried by the tasks to be scheduled by the computing power; b. Based on the communication constraint information and the hierarchical topology tree, determine at least one candidate deployment scheme that meets the deployment constraints, and select the candidate deployment scheme with the smallest fragmentation metric value to deploy the computing power to be scheduled to run the task; c. If the current fragmentation metric of the cluster is greater than the second preset threshold, filter the computing power to be migrated and merge them to obtain a complete topology domain for use by large-scale computing power running tasks; d. Predict and drive tidal migration between the training resource pool and the inference resource pool based on historical load data.

[0189] The current fragmentation metric of the cluster can be continuously calculated at fixed intervals or triggered by events. If the current fragmentation metric of the cluster exceeds a second preset threshold, the following steps are executed: (1) Screening the set of portable computing power running tasks: From the currently running computing power running tasks, select candidate computing power running tasks according to preset migration conditions. The migration conditions may include, but are not limited to: the amount of resources occupied by the task is less than the preset threshold (i.e., "small task"), the task supports checkpoint saving and recovery, the estimated remaining runtime of the task is less than the third preset threshold, and the task is not currently in the execution stage that is not sensitive to latency.

[0190] (2) Migration cost assessment: For each candidate computing power operation task, estimate its migration cost and migration benefit, and select the computing power operation task whose benefit is significantly greater than the cost as the computing power operation task to be migrated.

[0191] (3) Perform migration: Perform state saving operation on the computing power to be migrated (checkpoint saving technology can be used, or it can be combined with the lower cost incremental checkpoint saving technology), reschedule it to a new physical location, increase the proportion of idle computing power resources in the subdomain where the source location is located, and gradually merge into a complete topology domain that meets the preset integrity threshold.

[0192] (4) Dual-scheme output: For large-scale computing tasks that are waiting in line due to insufficient resources, two schemes, namely the first candidate scheme and the second candidate scheme, and their corresponding estimated completion times can be given at the same time, which can be left to the scheduling strategy or user decision-making.

[0193] The above-mentioned cleanup actions can be configured to be performed during periods when the overall load level of the computing cluster is low, so as to reduce interference with online computing tasks.

[0194] It should be noted that the present invention does not limit the execution order of steps ab, c, and d above.

[0195] This invention can avoid the training efficiency loss caused by placing communication-sensitive parallel groups across low-bandwidth levels, can actively eliminate the fragmentation of cluster computing resources, shorten the queuing time of large-scale tasks, and can avoid irrational preemption decisions, making it suitable for resource scheduling scenarios of large-scale artificial intelligence clusters.

[0196] Please see Figure 7 , Figure 7 This is a structural diagram of a topology-aware anti-fragmentation scheduling device for an intelligent computing cloud platform provided by the present invention, as shown below. Figure 7 As shown, the intelligent computing cloud platform's topology-aware anti-fragmentation scheduling device 700 includes: The acquisition module 701 is used to acquire the computing power to be scheduled running task; wherein, the computing power to be scheduled running task carries communication constraint information, the communication constraint information includes the deployment constraint conditions of the first parallel group of the computing power to be scheduled running task, the deployment constraint conditions are used to indicate that the computing nodes deploying the first parallel group must be located in the same subdomain in the hierarchical topology tree of the computing cluster with an interconnection bandwidth of not less than a first preset threshold. The first determining module 702 is used to determine at least one candidate deployment scheme that satisfies the deployment constraints for the task to be scheduled based on the communication constraint information and the hierarchical topology tree. The computing module 703 is used to calculate the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree; wherein, the first fragmentation metric value is used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme; The deployment module 704 is used to select a candidate deployment scheme whose first fragmentation metric value meets the preset optimization target as the target deployment scheme, and deploy the task to be scheduled by computing power according to the target deployment scheme.

[0197] In one embodiment, the calculation module 703 is used for: Based on the amount of idle computing resources in each subdomain of each level in the hierarchical topology tree, the total amount of idle computing resources in the level is determined, and based on the amount of idle computing resources in each subdomain and the total amount of idle computing resources, the proportion of idle computing resources in each subdomain is determined. Based on the proportion of idle computing resources in each subdomain, the information entropy of the level is calculated as the fragmentation entropy of the level; wherein, the fragmentation entropy is used to characterize the degree of computing power fragmentation of the level. Based on the weights of each level in the hierarchical topology tree, the fragment entropy of each level is weighted to determine the first fragment metric value.

[0198] In one embodiment, the calculation module 703 is used for: Based on the weights of each level and the total amount of idle computing resources at each level, the fragment entropy of each level is weighted to determine the first fragmentation metric.

[0199] In one embodiment, the information entropy of the hierarchy is equal to the negative sum of the products of the proportion of idle computing resources in each subdomain of the hierarchy and the logarithm of the proportion of idle computing resources.

[0200] In one embodiment, the weights of each level are negatively correlated with the depth of each level in the hierarchical topology tree.

[0201] In one embodiment, the device may further include: The filtering module is used to continuously monitor the current second fragmentation metric value of the computing cluster. If the second fragmentation metric value is greater than the second preset threshold, candidate computing tasks that meet the migration conditions are selected from the currently running computing tasks. The second determining module is used to determine the computing power to be migrated from the candidate computing power running tasks; The first migration module is used to save the status information of the computing power running task to be migrated, and to migrate the computing power running task from the current deployment location to the target location.

[0202] In one embodiment, the second determining module is configured to: Determine the migration cost and migration benefits of the candidate computing power running tasks; Based on the migration cost and migration benefit of the candidate computing power running tasks, the computing power running task to be migrated is determined from the candidate computing power running tasks.

[0203] In one embodiment, the migration conditions include at least one of the following: the resource consumption of the task is less than a third preset threshold, the task supports checkpoint saving and restoration, the estimated remaining runtime of the task is less than a fourth preset threshold, and the task is not currently in the execution phase where the required latency is less than a fifth preset threshold.

[0204] In one embodiment, the device may further include: The prediction module is used to predict the increase in resource demand of the first resource pool at a future target time based on the load profile over a preset period of time in the past. The second migration module is used to migrate candidate physical devices that are currently idle or whose computing power utilization is lower than a sixth preset threshold in the second resource pool to the first resource pool before the target time.

[0205] In one embodiment, the second migration module is used for: If the physical device to be migrated among the candidate physical devices is running a first computing power running task, the status information of the first computing power running task is saved before the preemption scheduling of the first computing power running task is performed. Based on the time and resource overhead of the save operation, determine the preemption cost for scheduling the first computing power task. Based on the migration benefits corresponding to the migration of the physical device to be migrated and the preemption cost, determine whether to preemptively schedule the first computing power to run the task and migrate the physical device to be migrated to the first resource pool.

[0206] In one embodiment, the device may further include: The output generation module is used to generate and output the first candidate solution and its corresponding estimated completion time, the second candidate solution and its corresponding estimated completion time for the second computing power running task waiting in the queue. The first candidate solution is to wait for the computing power fragments to be reorganized and then deploy the second computing power running task according to the deployment constraints of the second computing power running task. The second candidate solution is to use a degraded topology scheme to deploy the second computing power running task immediately.

[0207] In one embodiment, the communication constraint information further includes deployment constraints for a second parallel group of the task to be scheduled to run, wherein the deployment constraints for the second parallel group are used to indicate that the computing nodes deploying the second parallel group are allowed to traverse multiple subdomains in the hierarchical topology tree where the interconnect bandwidth is lower than the first preset threshold.

[0208] The intelligent computing cloud platform based on topology-aware computing power anti-fragmentation scheduling device provided by the present invention can realize the various processes of the above-mentioned intelligent computing cloud platform based on topology-aware computing power anti-fragmentation scheduling method. The technical features are one-to-one and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0209] It should be noted that the intelligent computing cloud platform based on topology-aware computing power anti-fragmentation scheduling device in this invention can be a device, or it can be a component, integrated circuit, or chip in an electronic device.

[0210] The present invention also provides an electronic device, see below. Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device includes a memory 801, a processor 802, and a program or instructions stored in the memory 801 that run on the processor 802. When the program or instructions are executed by the processor 802, they can achieve the following: Figure 1 The corresponding steps in the implementation of the topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms, and the achievement of the same beneficial effects, will not be elaborated here.

[0211] The processor 802 can be a CPU, ASIC, FPGA, or GPU.

[0212] Those skilled in the art will understand that all or part of the steps of the above-described intelligent computing cloud platform topology-aware anti-fragmentation scheduling method embodiment can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0213] The present invention also provides a readable storage medium on which a computer program is stored, and which, when executed by a processor, can perform the above-described functions. Figure 1The corresponding intelligent computing cloud platform is based on any step of the topology-aware anti-fragmentation scheduling method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here. The storage medium mentioned is such as read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0214] The present invention also provides a computer program product, including computer instructions that, when executed by a processor, implement the above-described... Figure 1 The corresponding intelligent computing cloud platform is based on the topology-aware computing power anti-fragmentation scheduling method implementation method, and each process can achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[0215] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. Additionally, the use of "and / or" in this invention indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: A alone, B alone, C alone, both A and B present, both B and C present, both A and C present, and A, B, and C present.

[0216] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0217] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or second terminal device, etc.) to execute the methods of the various embodiments of the present invention.

[0218] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms, characterized in that, include: Step S1: Obtain the computing power task to be scheduled; wherein the computing power task to be scheduled carries communication constraint information, the communication constraint information includes the deployment constraint conditions of the first parallel group of the computing power task to be scheduled, the deployment constraint conditions are used to indicate that the computing nodes deploying the first parallel group must be located in the same subdomain in the hierarchical topology tree of the computing cluster with an interconnection bandwidth of not less than a first preset threshold. Step S2: Based on the communication constraint information and the hierarchical topology tree, determine at least one candidate deployment scheme that satisfies the deployment constraint conditions for the task to be scheduled to run computing power. Step S3: Calculate the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree; wherein, the first fragmentation metric value is used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme; Step S4: Select a candidate deployment scheme whose first fragmentation metric value meets the preset optimization target as the target deployment scheme, and deploy the task to be scheduled by computing power according to the target deployment scheme.

2. The method as described in claim 1, characterized in that, Step S3 includes: Step S3.1: Determine the total amount of idle computing resources of each subdomain in each level of the hierarchical topology tree, and determine the proportion of idle computing resources of each subdomain based on the amount of idle computing resources of each subdomain and the total amount of idle computing resources. Step S3.2: Calculate the information entropy of the level as the fragmentation entropy of the level based on the proportion of idle computing resources in each subdomain; wherein, the fragmentation entropy is used to characterize the degree of computing power fragmentation of the level. Step S3.3: Based on the weights of each level in the hierarchical topology tree, the fragment entropy of each level is weighted to determine the first fragment metric value.

3. The method as described in claim 2, characterized in that, Step S3.3 includes: Step S3.3.1: Based on the weight of each level and the total amount of idle computing resources of each level, the fragment entropy of each level is weighted to determine the first fragmentation metric value.

4. The method as described in claim 2, characterized in that, The information entropy of the level is equal to the negative sum of the products of the proportion of idle computing resources in each subdomain of the level and the logarithm of the proportion of idle computing resources.

5. The method as described in claim 2, characterized in that, The weight of each level is negatively correlated with the depth of each level in the hierarchical topology tree.

6. The method as described in claim 1, characterized in that, The method further includes: Step S5: Continuously monitor the current second fragmentation metric value of the computing cluster. If the second fragmentation metric value is greater than the second preset threshold, select candidate computing tasks that meet the migration conditions from the currently running computing power running tasks. Step S6: Determine the computing power to be migrated from the candidate computing power running tasks; Step S7: Save the status information of the computing power running task to be migrated, and migrate the computing power running task to be migrated from the current deployment location to the target location.

7. The method as described in claim 6, characterized in that, Step S6 includes: Step S6.1: Determine the migration cost and migration benefits of the candidate computing power running tasks; Step S6.2: Based on the migration cost and migration benefit of the candidate computing power running tasks, determine the computing power running task to be migrated from the candidate computing power running tasks.

8. The method as described in claim 6, characterized in that, The migration conditions include at least one of the following: the resource consumption of the task is less than the third preset threshold, the task supports checkpoint saving and restoration, the estimated remaining runtime of the task is less than the fourth preset threshold, and the task is not currently in the execution phase where the required latency is less than the fifth preset threshold.

9. The method as described in claim 1, characterized in that, The method further includes: Step S8: Based on the load profile over a preset period of time, predict the increase in resource demand for the first resource pool at a future target time. Step S9: Before the target time, migrate candidate physical devices that are currently idle or whose computing power utilization rate is lower than the sixth preset threshold in the second resource pool to the first resource pool.

10. The method as described in claim 9, characterized in that, Step S9 includes: Step S9.1: If the physical device to be migrated among the candidate physical devices is running a first computing power running task, before performing preemptive scheduling on the first computing power running task, the status information of the first computing power running task is saved. Step S9.2: Determine the preemption cost for preempting the scheduling of the first computing power task based on the time and resource overhead of the save operation. Step S9.3: Based on the migration benefits corresponding to the migration of the physical device to be migrated and the preemption cost, determine whether to preemptively schedule the first computing power to run the task and migrate the physical device to be migrated to the first resource pool.

11. The method as described in claim 1, characterized in that, The method further includes: Step S10: For the second computing power running task waiting in the queue, generate and output the first candidate solution and its corresponding estimated completion time, the second candidate solution and its corresponding estimated completion time; The first candidate solution is to wait for the computing power fragments to be reorganized and then deploy the second computing power running task according to the deployment constraints of the second computing power running task. The second candidate solution is to use a degraded topology scheme to deploy the second computing power running task immediately.

12. The method according to any one of claims 1-11, characterized in that, The communication constraint information also includes the deployment constraints of the second parallel group of the task to be scheduled to run. The deployment constraints of the second parallel group are used to indicate that the computing nodes that deploy the second parallel group are allowed to cross multiple subdomains in the hierarchical topology tree with interconnect bandwidth lower than the first preset threshold.

13. A topology-aware anti-fragmentation scheduling device for intelligent computing cloud platforms, characterized in that, include: The acquisition module is used to acquire the computing power to be scheduled running tasks; wherein, the computing power to be scheduled running tasks carries communication constraint information, the communication constraint information includes the deployment constraint conditions of the first parallel group of the computing power to be scheduled running tasks, the deployment constraint conditions are used to indicate that the computing nodes deploying the first parallel group must be located in the same subdomain in the hierarchical topology tree of the computing cluster with an interconnection bandwidth of not less than a first preset threshold. The first determining module is used to determine at least one candidate deployment scheme that satisfies the deployment constraints for the task to be scheduled by computing power, based on the communication constraint information and the hierarchical topology tree. The computing module is used to calculate the first fragmentation metric value corresponding to the candidate deployment scheme based on the amount of idle computing resources in each subdomain of the hierarchical topology tree; wherein, the first fragmentation metric value is used to characterize the expected degree of computing power fragmentation of the computing cluster after executing the candidate deployment scheme; The deployment module is used to select a candidate deployment scheme whose first fragmentation metric value meets a preset optimization target as the target deployment scheme, and deploy the task to be scheduled by computing power according to the target deployment scheme.

14. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the intelligent computing cloud platform based on topology-aware anti-fragmentation scheduling method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the intelligent computing cloud platform based on topology-aware anti-fragmentation scheduling method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the topology-aware anti-fragmentation scheduling method for intelligent computing cloud platforms as described in any one of claims 1 to 12.

Images