A task scheduling method and device, electronic equipment and storage medium

Abstract

The application relates to the computer technical field, and in particular to a task scheduling method and device, electronic equipment and storage medium, to improve resource utilization and task execution efficiency. The method comprises the following steps: for each work node in a mixed deployment cluster, the following resource management operations are respectively performed: obtaining schedulable resource information for offline tasks according to resource usage information of each deployed task in multiple specified dimensions; each deployed task is an online task or an offline task deployed on the work node; performing resource interference detection based on the multiple specified dimensions to obtain conflict resource information representing resource details interfering with the online task due to resource shortage; updating the schedulable resource information after releasing resources of at least one target offline task in each deployed task according to the schedulable resource information and the conflict resource information; and scheduling a to-be-scheduled offline task according to the latest schedulable resource information of each work node.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task scheduling method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence (AI) and high-performance computing applications, the demand for graphics processing units (GPUs) has exploded. This demand stems not only from the practical needs of business expansion but also reveals the contradiction between existing GPU resources and ever-increasing demand. To alleviate this tension, the industry has widely adopted online and offline hybrid deployment technology, which reuses GPU resources during idle periods of online tasks for offline training to optimize resource allocation efficiency.

[0003] However, against the backdrop of the rapid evolution of large-scale AI models, despite the widespread application of GPUs due to their superior parallel computing capabilities, resource scarcity remains a prominent issue. Especially in traditional GPU-based offline / indirect deployment systems, resource allocation often considers only a single dimension. This exposes numerous problems when facing complex mixed workloads of online inference and offline training, such as insufficient performance guarantees for online tasks, inefficient data processing due to unreasonable offline task scheduling, and frequent resource evictions. These problems make current GPU-based offline / indirect deployment solutions, based on a single specified dimension, unable to meet the diverse needs of real-world business scenarios, thus impacting overall task quality and work efficiency.

[0004] In summary, effectively addressing the issue of decreased task quality and workload performance in traditional GPU offline mixed deployment solutions is a pressing problem that needs to be solved. Summary of the Invention

[0005] This application provides a task scheduling method, apparatus, electronic device, and storage medium to improve resource utilization and task execution efficiency.

[0006] One task scheduling method in this application embodiment includes:

[0007] For each worker node in the mixed-deployment cluster, perform the following resource management operations:

[0008] Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node.

[0009] Resource interference detection is performed based on the specified dimensions to obtain conflicting resource information; wherein...

[0010] The conflict resource information represents the details of resources that interfere with online tasks due to insufficient resources;

[0011] Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated.

[0012] Based on the latest schedulable resource information of each working node, the offline tasks to be scheduled are scheduled.

[0013] A task scheduling device according to an embodiment of this application includes:

[0014] The resource management unit is used to perform the following resource management operations for each worker node in the hybrid deployment cluster:

[0015] Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node.

[0016] Resource interference detection is performed based on the specified dimensions to obtain conflicting resource information; wherein...

[0017] The conflict resource information represents the details of resources that interfere with online tasks due to insufficient resources;

[0018] Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated.

[0019] The task scheduling unit is used to schedule offline tasks based on the latest schedulable resource information of each working node.

[0020] Optionally, the resource management unit is specifically used for:

[0021] Based on the schedulable resource information and the conflicting resource information, analyze whether there is a resource management anomaly; wherein, the resource management anomaly indicates that the amount of resources occupied by offline tasks needs to be reduced;

[0022] If a resource management anomaly exists, at least one target offline task matching each target dimension is selected from the deployed tasks; wherein each target dimension is the dimension with a resource management anomaly among the plurality of specified dimensions.

[0023] Based on the release strategy corresponding to each of the target dimensions, the corresponding resources occupied by the at least one target offline task are released.

[0024] Optionally, if the resource in the target dimension is an elastic resource, then the release strategy corresponding to the target dimension is: reduce the resource usage of the task in the target dimension;

[0025] If the resource in the target dimension is an inelastic resource, then the release strategy corresponding to the target dimension is: release all resources occupied by the task.

[0026] Optionally, the resource management unit is specifically used for:

[0027] For each target offline task, if the resources occupied by the target offline task in each target dimension include non-elastic resources, then by scheduling the target offline task, all resources occupied by the target offline task are released.

[0028] For each target offline task, if the resources occupied by the target offline task in each target dimension do not include non-elastic resources, then the resources occupied by the target offline task are released by reducing the resource usage of the target offline task in each target dimension.

[0029] Optionally, the resource management unit is specifically used for:

[0030] Tasks from target task combinations selected from each task combination are designated as target offline tasks; wherein each task combination consists of at least one offline task from each deployed task that occupies resources in the target dimension; the target task combination satisfies at least one of the following matching conditions:

[0031] Matching condition 1: Under each target dimension, the total amount of resources occupied by the combination of target tasks reaches the expected reduction in resources;

[0032] Matching condition two: The sum of the sacrifice levels of each task in the target task combination is the minimum; the sacrifice level of each task represents the intensity of the impact of executing the release strategy on the execution state of the corresponding task.

[0033] Matching condition three: The total number of tasks in the target task combination is the minimum.

[0034] Optionally, when applying matching condition two or matching condition three for filtering, the target task combination must simultaneously satisfy matching condition one.

[0035] Optionally, there are multiple hybrid clusters, which are located at the bottom layer of the hybrid architecture, which also includes an upper layer and a hybrid management and control layer; the device is applied to the hybrid architecture.

[0036] The task scheduling unit is specifically used for:

[0037] For each working node, the working node reports the corresponding schedulable resource information to the mixed-distribution management layer;

[0038] The mixed-deployment control layer aggregates the schedulable resource information reported by each working node and reports it to the upper layer.

[0039] The upper layer schedules the offline tasks to be scheduled based on the aggregated schedulable resource information.

[0040] Optionally, the task scheduling unit is specifically used for:

[0041] The upper layer determines the target working node for deploying the offline task to be scheduled from among the working nodes based on the aggregated schedulable resource information.

[0042] The mixed deployment control layer distributes the offline tasks to be scheduled to the target working nodes for deployment.

[0043] Optionally, each worker node includes a hybrid component; the hybrid component is used to periodically perform resource management operations for the worker node.

[0044] An electronic device according to an embodiment of this application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the above-described task scheduling methods, including:

[0045] For each worker node in the mixed-deployment cluster, perform the following resource management operations:

[0046] Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node.

[0047] Resource interference detection is performed based on the specified dimensions to obtain conflicting resource information; wherein...

[0048] The conflict resource information represents the details of resources that interfere with online tasks due to insufficient resources;

[0049] Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated.

[0050] Based on the latest schedulable resource information of each working node, the offline tasks to be scheduled are scheduled.

[0051] This application provides a computer-readable storage medium including a computer program. When the computer program is run on an electronic device, the computer program is used to cause the electronic device to perform the steps of any of the above-described task scheduling methods.

[0052] This application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. When the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any of the above-described task scheduling methods.

[0053] The beneficial effects of this application are as follows:

[0054] This application provides a task scheduling method, apparatus, electronic device, and storage medium. Specifically, the method first obtains schedulable resource information for offline tasks by mining the resource usage of each deployed task across multiple specified dimensions, such as Central Processing Unit (CPU), GPU, memory, disk, and storage. This step ensures a comprehensive understanding of the resource requirements of each task (including online and offline tasks) on the worker node, thereby more accurately identifying resources that can be reallocated to new tasks and improving overall resource utilization.

[0055] Next, resource interference detection is performed based on multiple specified dimensions to identify situations where insufficient resources may interfere with the normal operation of online tasks. This step analyzes conflicting resource information and takes measures to avoid or mitigate such interference, ensuring that the service quality of online tasks is not affected.

[0056] Subsequently, based on the schedulable resource information and conflicting resource information, resources are released for at least one target offline task, and the schedulable resource information is updated. This operation not only helps resolve resource conflict issues but also dynamically adjusts resource allocation strategies, enabling the system to flexibly respond to different load conditions.

[0057] Finally, based on the latest schedulable resource information of each worker node, the offline tasks to be scheduled are scheduled. This step ensures that new offline tasks can be reasonably allocated to worker nodes with sufficient available resources, thereby avoiding resource waste and unnecessary task delays, and further optimizing the system's operating efficiency.

[0058] In summary, this task scheduling method based on mixed-deployment clusters effectively solves the common problems of task quality and workload performance degradation in traditional GPU offline mixed-deployment schemes through refined resource management operations, significantly improving system resource utilization and task execution efficiency, while ensuring the performance of online tasks.

[0059] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0060] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0061] Figure 1 This is an optional schematic diagram of an application scenario in an embodiment of this application;

[0062] Figure 2 This is a schematic diagram of a hybrid architecture in an embodiment of this application;

[0063] Figure 3 This is an architecture diagram of a GPU cluster according to an embodiment of this application;

[0064] Figure 4 This is a flowchart illustrating the implementation of a task scheduling method in an embodiment of this application.

[0065] Figure 5 This is a schematic diagram illustrating a hybrid cluster and the task deployment on each working node included in this application embodiment.

[0066] Figure 6 This is a schematic diagram of statistical results of resource usage information in an embodiment of this application;

[0067] Figure 7 This is a schematic diagram illustrating the calculation logic of schedulable resource information in an embodiment of this application;

[0068] Figure 8 This is a schematic diagram illustrating the interaction logic between an offline resource manager, an interference detection controller, and a resource detection controller within a hybrid component according to an embodiment of this application.

[0069] Figure 9 This is a schematic diagram of the selection logic for a target offline task in an embodiment of this application;

[0070] Figure 10This is a logical diagram illustrating a resource release in one embodiment of this application;

[0071] Figure 11 This is a schematic diagram illustrating the application logic of schedulable resource information in an embodiment of this application;

[0072] Figure 12 This is a schematic diagram of a task execution state in an embodiment of this application;

[0073] Figure 13 This is a schematic diagram of yet another hybrid architecture in an embodiment of this application;

[0074] Figure 14 This is a schematic diagram of the composition structure of a task scheduling device provided in an embodiment of this application;

[0075] Figure 15 This is a schematic diagram of the hardware structure of an electronic device using an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0077] The following describes some of the concepts involved in the embodiments of this application.

[0078] 1. Mixed Offline Deployment: This refers to a technical solution that runs online and offline tasks simultaneously within the same cluster environment. Online tasks typically require high resource availability and exhibit significant load fluctuations; offline tasks, on the other hand, are less sensitive to latency and can be executed when resources are idle. By strategically scheduling these two types of tasks, hardware resource utilization efficiency can be maximized without affecting the quality of online tasks, especially by filling off-duty periods with offline tasks to fully utilize idle resources.

[0079] Taking a Kubernetes cluster environment as an example, Kubernetes (k8s) is an open-source platform used to manage containerized tasks across multiple compute nodes in a cloud platform, aiming to make deploying containerized applications simple and efficient. As a resource scheduling and management platform for container orchestration, k8s provides mechanisms for task deployment, planning, updating, and maintenance. Users can submit tasks on a Kubernetes-based cloud platform. These tasks run as containers, where a Pod is the smallest scheduling unit in k8s and can be considered a task instance. A complete task typically consists of multiple such instances. Through Kubernetes, users can achieve efficient resource management and automated operation and maintenance.

[0080] 2. Online Tasks: These typically refer to tasks that require immediate responses to user requests. These tasks are highly sensitive to latency, demanding that the system process them and return results to the user within a short timeframe. They are characterized by being short-lived, frequent, and crucial to user experience. Online tasks have strict requirements for response time and availability; they are usually services that can directly interact with the task submission platform, requiring rapid responses to platform requests and maintaining high availability and stability.

[0081] 3. Offline tasks: These are tasks that do not require immediate response, typically used for background data processing, analysis, and model training. These tasks have relatively low latency requirements and can tolerate some delay, but may require significant computing resources and time to complete. Unlike online tasks, offline tasks focus more on completing the task itself rather than responding to platform requests in real time.

[0082] 4. Hybrid Deployment Cluster (or simply Cluster): This refers to the simultaneous deployment and running of multiple different types of tasks, such as online and offline tasks, within a single computing resource cluster. Through resource sharing and dynamic scheduling technologies, it improves resource utilization and reduces operating costs. Hybrid deployment clusters utilize intelligent scheduling and isolation mechanisms to maximize the overall efficiency of the cluster while ensuring the performance of each application. It is suitable for various scenarios such as cloud computing, big data processing, and distributed systems. It supports flexible expansion and can automatically adjust resource configuration based on load to ensure efficient and stable service delivery.

[0083] In one alternative implementation, the hybrid cluster can be a GPU cluster. A GPU cluster specifically refers to a Kubernetes cluster whose nodes are equipped with graphics processing units (GPUs). Because GPUs have significant advantages in areas such as graphics rendering, scientific computing, and deep learning, specially built GPU clusters can provide powerful computing capabilities for these fields. Such clusters are particularly suitable for applications requiring high-performance parallel computing, such as artificial intelligence training and big data analytics.

[0084] 5. Worker Nodes (also called compute nodes, or simply nodes): These are the basic computing units in a hybrid deployment cluster. They can be physical servers or virtual machines, responsible for executing specific tasks. These nodes are equipped with various computing resources, including GPUs, CPUs, memory, disks, and storage, and are interconnected via a network. They are coordinated using unified management tools (such as Kubernetes) to achieve goals such as load balancing, fault tolerance, and high availability. In a hybrid deployment cluster, worker nodes not only support the deployment and running of online tasks, such as World Wide Web (Web) applications and Application Programming Interface (API) services, but also can deploy and run offline tasks, such as data analysis and machine learning model training, according to instructions from the cluster management system. This flexibility allows worker nodes to automatically adjust resource configurations based on real-time load, ensuring efficient and stable service delivery.

[0085] In one alternative implementation, the worker nodes in the hybrid cluster can be GPU nodes within a GPU cluster. These nodes are capable of handling not only offline tasks requiring high-performance graphics processing or parallel computing, but also supporting efficient response to online tasks. Through an intelligent scheduling mechanism, these nodes can maximize the use of idle resources to execute offline tasks while ensuring the quality of online tasks, thereby improving the resource utilization and economic efficiency of the entire cluster.

[0086] 6. Deployed Tasks: These refer to applications or tasks that have been started and are running on the worker nodes of the hybrid deployment cluster. These tasks can be divided into two categories: online tasks (such as services requiring fast response) and offline tasks (such as batch processing jobs that can tolerate latency). In this embodiment, the status and resource usage of deployed tasks are monitored for effective resource management and task scheduling.

[0087] 7. Specified Dimensions: These refer to specific attributes or metrics used to measure and evaluate task resource usage, such as GPU, CPU, memory, disk, and storage. Monitoring these dimensions provides a comprehensive understanding of the task's resource requirements and impact, thus providing a basis for resource allocation, interference detection, and optimization. Different application scenarios may focus on different combinations of dimensions; this article does not impose specific limitations on this.

[0088] 8. Resource Usage Information: This records the actual resource consumption of each deployed task under specified dimensions, such as GPU usage, CPU usage, memory usage, and disk read / write speed. This data is collected in real time by the monitoring system and used to analyze task behavior patterns and resource requirements. Accurate resource usage information helps optimize resource allocation strategies and improve the overall performance and efficiency of the system.

[0089] 9. Scheduled Resource Information: This indicates the amount of remaining resources on the current working node that can be used for scheduling new tasks (specifically, new offline tasks in this article), calculated based on the resource usage of existing tasks. It reflects the actual available resource status of the node and takes into account factors such as reserved resources and priority. When a new task needs to be scheduled, the system will refer to this information to determine whether the new task can be deployed on this node and how many resources can be allocated to the new task.

[0090] 10. Resource Interference Detection: This mechanism identifies inter-task interference caused by resource contention. By monitoring resource usage across specified dimensions, it determines whether resource shortages are affecting the normal operation of online tasks. Once a potential resource conflict is detected, the system will take measures to mitigate the interference, such as adjusting task priorities or reallocating resources, to ensure the service quality of critical tasks.

[0091] 11. Conflicting Resource Information: This section indicates specific resource details that may cause online tasks to be disrupted due to insufficient resources. Optionally, conflicting resource information includes, but is not limited to: the type of affected resource (e.g., CPU, memory), current usage and upper limits, and a list of associated tasks. This information helps administrators or automation systems understand the root cause of the problem and develop solutions, such as releasing some resources or migrating low-priority tasks.

[0092] 12. Resource Management Anomalies: These refer to abnormal states that occur during resource allocation and usage, potentially leading to system performance degradation or task failure. In mixed-deployment clusters, resource management anomalies typically manifest as a decline in service quality for online tasks due to excessive resource consumption by offline tasks. Examples include: resource over-allocation, where resources are allocated to tasks beyond their actual needs, resulting in resource waste; and resource conflicts, where multiple tasks compete for limited resources, preventing some tasks from obtaining sufficient resources for normal operation and impacting overall performance; and so on.

[0093] 13. Resource Release: This refers to freeing up resources that are no longer needed or can be temporarily borrowed from certain tasks so that they can be reallocated to other more urgent or important tasks. Optionally, this may involve terminating non-critical processes, reducing memory usage, and limiting input / output (I / O) operations. Resource release aims to optimize resource utilization, reduce waste, and ensure that high-priority tasks have sufficient resources to support their smooth execution.

[0094] 14. Task Scheduling: This is one of the core functions of a mixed-deployment cluster management system, responsible for determining when and where to execute which tasks. Based on the latest schedulable resource information of each worker node, the scheduler selects appropriate locations to deploy tasks to be executed, while considering factors such as task priority, resource requirements, and expected completion time. Effective task scheduling strategies can significantly improve the overall performance of the system, ensure service quality, and maximize resource utilization.

[0095] Optionally, a release strategy refers to the specific rules or algorithms used to guide how to release resources from existing tasks when resource management anomalies are detected. Release strategies can be formulated based on different dimensions, task priorities, task types, and other factors to ensure system stability and efficiency.

[0096] 15. Inflexible resources vs. flexible resources:

[0097] Inelastic resources refer to those resources required by tasks that are highly sensitive to resource changes. If resources are reduced for these tasks, it can immediately lead to a significant performance drop or even task failure, such as with GPUs.

[0098] Elastic resources refer to the resources required by tasks that have a certain tolerance for resource changes. These tasks can continue to run even when resources are reduced, although performance may decrease, but they will not fail immediately, such as CPU and memory.

[0099] 16. Sacrifice Level: This refers to the impact of resource release strategies (such as CPU, GPU, memory, etc.) on affected tasks under abnormal resource management conditions. The higher the sacrifice level, the greater the performance degradation, increased recovery costs, or severe business impact suffered by the task due to resource reduction. Therefore, during resource scheduling, tasks with lower sacrifice levels should be prioritized for resource release to minimize system performance loss.

[0100] 17. Kernel Metrics: These are key system measurement data used to monitor multi-dimensional resource usage and task execution status in offline / mixed deployment scenarios. These metrics can assess real-time resource utilization, guide task scheduling optimization, ensure the stability of online tasks, and improve the execution efficiency of offline / mixed deployment tasks. In this application embodiment, kernel metrics include, but are not limited to, CPU usage, memory usage, I / O performance, context switching, average load, number of processes and threads, file descriptor usage, interrupts and exceptions, and cache hit rate.

[0101] 18. Three-tier architecture: This is a special deployment model based on Kubernetes (or other cluster environments) designed to optimize resource utilization and task scheduling efficiency. This architecture is particularly suitable for large-scale distributed computing environments, improving system flexibility and resource utilization. It consists of the following three layers:

[0102] (1) Bottom layer: Composed of multiple independent Kubernetes clusters, typically used to execute specific computing tasks, such as online tasks and offline tasks. (2) Upper layer: At least one Kubernetes cluster, such as a single Kubernetes cluster, responsible for receiving user task requests and distributing these requests to the bottom layer clusters for execution. (3) Middle layer (also referred to as the hybrid deployment control layer in this article): Deploys hybrid deployment components across clusters, responsible for coordinating task allocation and scheduling between the upper and lower layer clusters, ensuring effective resource utilization, and also responsible for synchronizing task execution status between the bottom and upper layers.

[0103] The design concept of the embodiments of this application will be briefly described below:

[0104] Taking AI training and inference scenarios as an example, with the rapid development of cloud computing and AI technology, the hybrid deployment of online and offline tasks (i.e., mixed deployment) has gradually become an important means to improve resource utilization. In the traditional mixed deployment model, platforms such as Kubernetes utilize idle resources during the off-peak hours of online tasks to run low-priority offline tasks in order to improve overall resource utilization efficiency. However, this method often only focuses on a single dimension of task scheduling (such as only considering GPU or CPU), which can easily lead to resource conflicts. For example, online tasks, due to their need for real-time response and high-priority processing, exhibit significant fluctuations during operation, resulting in uneven resource utilization. When offline tasks attempt to reuse these resources, they may interfere with online tasks due to competition for resources such as disk and memory, or even cause frequent evictions due to unreasonable scheduling, increasing unnecessary system overhead.

[0105] Meanwhile, current AI training scenarios are shifting from relying on single GPU resources to CPU-GPU collaborative computing. This trend is driven not only by technological advancements, such as frameworks like CUDA and OpenCL supporting heterogeneous computing, but also by the need to meet the complementary characteristics of different resources: CPUs excel at handling complex logic and control flow, while GPUs are better suited for large-scale parallel computing. This collaborative approach can utilize computing resources more efficiently and optimize performance. However, despite this shift, existing GPU clusters in offline hybrid deployment practices still primarily focus on optimizing a single specified dimension, such as optimizing only GPU usage or CPU allocation, and have not yet fully leveraged the advantages of multiple specified dimensions to simultaneously ensure the quality of online tasks and improve offline data processing performance. This approach may encounter the following problems when handling complex online and offline hybrid workloads:

[0106] First, there is insufficient online support: online tasks typically require real-time response and high-priority processing, but in a mixed-distribution model with a single GPU and a specified dimension, the considerations are somewhat one-sided and may not provide sufficient resource guarantees for online tasks. For example, if CPU or memory resources are insufficient, even if GPU resources are sufficient, the execution of online tasks may still be disrupted.

[0107] Secondly, offline task scheduling is inefficient: In hybrid models with a single GPU and a specified dimension, offline task scheduling is typically based solely on GPU usage, ignoring the usage of other resources. This can lead to inefficient offline task scheduling; for example, offline tasks might be scheduled when CPU or memory resources are insufficient, thus impacting system performance.

[0108] Third, frequent evictions: In mixed-distribution models with a single GPU and a specified dimension, the lack of consideration for global resource availability may lead to frequent evictions and rescheduling of offline tasks by the offline training service to accommodate the demands of online tasks. Such frequent evictions and rescheduling can result in significant overhead and reduce system efficiency.

[0109] Therefore, a single GPU-based hybrid model with specified dimensions cannot provide satisfactory quality of service and performance when handling complex online and offline hybrid workloads.

[0110] In view of this, embodiments of this application provide a task scheduling method, apparatus, electronic device, and storage medium. Specifically, the method first obtains schedulable resource information for offline tasks by mining the resource usage of each deployed task across multiple specified dimensions. This step ensures a comprehensive understanding of the resource requirements of each task (including online and offline tasks) on the worker node, thereby more accurately identifying resources that can be reallocated to new tasks and improving overall resource utilization.

[0111] Next, resource interference detection is performed based on multiple specified dimensions to identify situations where insufficient resources may interfere with the normal operation of online tasks. This step analyzes conflicting resource information and takes measures to avoid or mitigate such interference, ensuring that the service quality of online tasks is not affected.

[0112] Subsequently, based on the schedulable resource information and conflicting resource information, resources are released for at least one target offline task, and the schedulable resource information is updated. This operation not only helps resolve resource conflict issues but also dynamically adjusts resource allocation strategies, enabling the system to flexibly respond to different load conditions.

[0113] Finally, based on the latest schedulable resource information of each worker node, the offline tasks to be scheduled are scheduled. This step ensures that new offline tasks can be reasonably allocated to worker nodes with sufficient available resources, thereby avoiding resource waste and unnecessary task delays, and further optimizing the system's operating efficiency.

[0114] In summary, this task scheduling method based on mixed-deployment clusters effectively solves the common problems of task quality and workload performance degradation in traditional GPU offline mixed-deployment schemes through refined resource management operations, significantly improving system resource utilization and task execution efficiency, while ensuring the performance of online tasks.

[0115] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0116] It should be noted that this application is applicable to complex computing environments that process both online and offline tasks simultaneously, such as data centers and cloud computing platforms. It can fully utilize schedulable resources to complete more offline tasks while ensuring the quality of online services and user experience.

[0117] The following is a brief introduction to the application scenarios of the task scheduling method provided in the embodiments of this application.

[0118] The solution provided in this application can be applied to cluster environments with hybrid online and offline deployments, such as the Kubernetes cluster environment mentioned above. Figure 1 The diagram shown is an application scenario diagram provided by an embodiment of this application. In this scenario, there are at least a terminal device 101, a scheduling node 102, and multiple working nodes 103.

[0119] Terminal device 101 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. In one optional implementation, the user can submit online tasks and offline tasks through terminal device 101.

[0120] Scheduling node 102 is the scheduling center of the cluster environment. It can receive task requests from terminal devices 101 and schedule the tasks. In this embodiment, scheduling node 102 is mainly used for unified management and scheduling of tasks.

[0121] Worker nodes 103 serve as the task execution endpoints for this cluster environment. They are responsible for executing scheduled tasks and collecting their schedulable resource information in real time. In this embodiment, each worker node 103 can be a computing node in this cluster environment, and several tasks can run on each worker node 103. Optionally, each worker node 103 can also periodically report the current available resource status to the scheduling node 102 to ensure that tasks can be allocated to the optimal worker node.

[0122] Optionally, both the scheduling node 102 and the worker node 103 can be implemented using servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal and server can be connected directly or indirectly via wired or wireless communication; this application does not impose any restrictions on this.

[0123] For example, after a user submits a task through terminal device 101, scheduling node 102 receives the task request and, based on the currently known schedulable resource information, combined with task requirements and system load, selects a suitable worker node 103 for task scheduling and issues execution instructions. After the task begins execution, worker node 103 periodically reports resource usage to scheduling node 102, optimizes resource allocation, and ensures efficient task operation.

[0124] In one possible application scenario, the cluster environment in this application embodiment can generally adopt a cloud platform approach. A cloud platform is an architecture platform built on cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, network, and other resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology applied in cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support, as the backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0125] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. Multiple types of virtual resources are deployed within the resource pool for external clients to choose from. The cloud computing resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, network devices, etc. These can be considered as the resource components of worker node 103, but this paper does not specifically limit their definition.

[0126] In another possible application scenario, the cluster environment of this application embodiment can serve as the implementation architecture for blockchain. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, blockchain is a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block.

[0127] For example, when there are multiple servers, the multiple servers can form a blockchain, and the servers are nodes on the blockchain; as disclosed in the task scheduling method based on mixed-deployment clusters in the embodiments of this application, the task-related information, resource-related information and release strategies involved can all be stored on the blockchain, such as resource usage information, schedulable resource information, online task and offline task-related information, etc., which are not specifically limited in this article.

[0128] In one alternative implementation, the terminal device 101, the scheduling node 102, and the working node 103 can communicate with each other via a communication network.

[0129] In one alternative implementation, the communication network is a wired network or a wireless network.

[0130] It should be noted that, Figure 1 The examples shown are merely illustrative; in reality, the number of terminal devices, scheduling nodes, and working nodes is unlimited and is not specifically limited in this embodiment.

[0131] Optionally, the embodiments of this application can be applied to various fields, including but not limited to cloud technology, artificial intelligence, intelligent transportation, and assisted driving. Specifically, they can be applied to task scheduling scenarios based on hybrid clusters in these fields. Several specific application scenarios are briefly listed below:

[0132] (I) AI training and inference scenarios.

[0133] In the field of AI, the proper scheduling of online and offline tasks is crucial for ensuring service quality and efficient utilization of computing resources. Online tasks mainly include real-time speech recognition, image recognition services, and recommendation systems. Real-time speech recognition involves users inputting commands via voice, requiring the system to quickly recognize and return results. Image recognition services involve users uploading images, requiring the system to quickly analyze the objects in the images and provide recognition results. Recommendation systems dynamically adjust recommendation content based on users' real-time behavior to provide personalized recommendations. Offline tasks mainly include model training, data preprocessing, and batch prediction. Model training utilizes large amounts of historical data to train deep learning models, typically involving high computational demands but less stringent real-time requirements. Data preprocessing includes data cleaning, format conversion, and feature extraction to provide high-quality data for training tasks. Batch prediction involves offline prediction of large-scale datasets, rather than processing them one by one in real-time, thus improving computational efficiency. In a mixed-deployment cluster environment, the system can prioritize computing resources for online inference tasks to ensure low-latency responses, while offline training tasks can run when the system load is low or utilize idle resources unused by online tasks to maximize computing power.

[0134] (ii) E-commerce platform scenario.

[0135] E-commerce systems need to ensure smooth user interaction while supporting backend tasks such as large-scale data analysis and inventory management. Online tasks mainly include user browsing and searching, and shopping cart management. User browsing and searching refer to users querying product information in real time, and the system needs to quickly retrieve and display relevant results. Shopping cart management refers to users adding or deleting items from their shopping carts, and the system needs to update the status in real time to avoid delays that could lead to a poor shopping experience. Offline tasks mainly include inventory synchronization and sales data analysis. Inventory synchronization involves periodically retrieving the latest inventory data from various warehouses and updating it to the database to ensure accurate inventory information. Sales data analysis is based on historical sales data to support decisions such as targeted marketing and inventory optimization. In a hybrid architecture, online tasks need to receive priority computing resources to ensure a good user experience, while offline tasks (such as inventory synchronization and data analysis) can run during off-peak hours to reduce interference with real-time business operations.

[0136] (iii) Social media platforms.

[0137] Social media platforms need to support real-time interaction from massive numbers of users while simultaneously managing and analyzing backend content. Online tasks primarily include dynamic posting and sharing, and messaging. Dynamic posting and sharing refers to users uploading images, videos, or posting updates, which the system must process instantly and push to followers. Messaging refers to instant messaging, requiring rapid message transmission to ensure a smooth user experience. Offline tasks mainly include social graph construction and content moderation. Social graph construction analyzes users' social relationships (such as friends, interaction frequency, etc.) to build a social graph for recommendation and information flow optimization. Content moderation uses AI models to review user-generated content; while this usually involves some latency, it doesn't need to be completed immediately. In a mixed-deployment environment, online tasks (such as push notifications) prioritize computing resources to ensure low-latency interaction, while offline tasks (such as social graph construction) can be executed in batches when system load is low, improving overall computational efficiency.

[0138] (iv) Financial services.

[0139] Financial systems need to handle a large number of real-time transactions while simultaneously performing back-end data analysis to ensure security and compliance. Online tasks mainly include real-time transaction processing and account balance inquiries. Real-time transaction processing refers to the bank or payment system processing and confirming user transaction requests in real time, while account balance inquiries involve users checking their account balances or recent transaction records, with the system returning the data immediately. Offline tasks mainly include risk assessment and anti-fraud analysis, and report generation. Risk assessment and anti-fraud analysis builds anti-fraud models based on historical transaction data to identify potential fraudulent transactions. Report generation generates monthly, quarterly, or annual financial statements, which can be scheduled for execution during off-peak periods. In a hybrid architecture, computing resources for online transaction tasks are prioritized to ensure smooth transaction flow, while offline tasks such as anti-fraud analysis and financial statement generation can be processed in batches to reduce the impact on peak business periods.

[0140] It should be noted that the application scenarios listed above are just simple examples, and other application scenarios are also applicable to the embodiments of this application, which will not be elaborated here.

[0141] The above briefly introduces the task scheduling logic and the application scenarios of this application. The method provided in this application is applied to mixed-deployment clusters, such as online and offline mixed-deployment scenarios in GPU clusters, improving system resource utilization through comprehensive resource management and interference detection mechanisms. It not only optimizes GPU resources but also considers multiple resource dimensions such as CPU, disk, and memory, ensuring that online and offline tasks can run efficiently and providing multi-dimensional quality of service assurance to avoid the impact of single-dimensional resource conflicts on system and job efficiency.

[0142] Furthermore, this application aims to provide a more secure resource isolation and scheduling strategy between online and offline tasks, ensuring the quality of online services while reducing resource shortages or frequent evictions caused by improper offline task scheduling, thereby reducing system overhead and improving overall processing efficiency. This comprehensive management approach enables a more stable and efficient GPU deployment in mixed online and offline environments.

[0143] Based on the above ideas, this application also proposes a three-layer hybrid architecture. Before introducing the method provided in the embodiments of this application, taking a GPU cluster as an example, the structure of the hybrid architecture in the embodiments of this application is first described:

[0144] Please see Figure 2 The diagram shown is a schematic of a hybrid architecture according to an embodiment of this application. The hybrid architecture is a three-layer architecture, including an upper layer, a middle layer, and a lower layer. In this hybrid architecture, the upper layer and the lower layer interface with different task submission platforms, achieving cluster-level resource isolation between the online task submission layer and the offline task submission layer.

[0145] like Figure 2 As shown, simply put, the upper layer is used to receive offline tasks submitted by users (such as object 210 and object 220) through the offline task submission platform and to schedule offline tasks; the upper layer is a single GPU cluster, such as... Figure 1 The scheduling node 102 in the middle can be regarded as the scheduler in the upper-layer GPU cluster.

[0146] The middle layer is a mixed-department control layer used for information relay. Figure 1 Not shown, its main function is to summarize the schedulable resource information reported by each working node 103 and report it to the upper layer. In addition, it assists the scheduling node 102 in the upper layer in intelligent task allocation.

[0147] The underlying layer receives online tasks submitted by users (such as object 230 and object 240) through the online task submission platform and schedules these tasks. Additionally, it can receive offline tasks assigned by the upper layer, allowing for the mixed execution of online and offline tasks. The underlying layer consists of multiple GPU clusters, each containing multiple GPU nodes. Figure 1 Working node 103 in the process.

[0148] It's important to note that the upper layer uses a single GPU cluster, while the lower layer consists of multiple GPU clusters. This is primarily based on optimization considerations for task management and resource utilization. The upper layer, acting as the scheduling center, is responsible for the submission and unified scheduling of offline tasks. The single GPU cluster architecture helps to centrally manage tasks, reduce scheduling complexity, and ensure the consistency of scheduling policies. This single GPU cluster architecture provides a global perspective, facilitating the scheduling nodes to plan and coordinate task requirements. Furthermore, the mixed-deployment control layer coordinates the underlying computing resources, thereby improving scheduling efficiency. The lower layer, composed of multiple GPU clusters, is responsible for executing both online and offline tasks. This maximizes the utilization of computing resources and improves the flexibility and scalability of GPU resources. The multi-GPU cluster architecture allows different clusters to be optimized for different task types. For example, some clusters can prioritize high-priority online tasks, while others can be used for mixed-deployment execution of both online and offline tasks. This achieves reasonable resource allocation and efficient execution of mixed-deployment tasks. The overall architecture, through unified upper-layer scheduling by the single GPU cluster and lower-layer computational execution by multiple GPU clusters, ensures the stability of online tasks while improving the execution efficiency of offline tasks, achieving optimal resource utilization.

[0149] See Figure 3 As shown, this is an architecture diagram of a GPU cluster according to an embodiment of this application. Figure 3 A simple example Figure 2 The composition of a GPU cluster.

[0150] Optionally, the GPU cluster consists of multiple GPU nodes, each equipped with a hybrid agent to coordinate resource scheduling and management for online and offline tasks. GPU nodes can run different types of tasks, such as online and offline tasks, while ensuring efficient utilization of computing resources and flexible task scheduling.

[0151] Optionally, a GPU node typically refers to a computing node containing one or more GPU cards. Specifically, a GPU node is a complete computing unit, usually including a CPU, system memory (RAM), storage devices, and one or more GPU cards, capable of executing online and offline tasks and communicating with other computing nodes via a network. The GPU card is the computing component within the GPU node, primarily responsible for graphics processing or general-purpose computing. Each GPU card has its own computing cores (such as CUDA cores) and video memory, providing high-performance parallel computing capabilities.

[0152] Optionally, a GPU node may contain multiple GPU cards and allow them to work together to support high-concurrency computing tasks.

[0153] In GPU clusters, task deployment is typically achieved through container (Pod) deployment. Each container is isolated from others, has its own file system, and processes within containers do not interfere with each other, thus differentiating computing resources. Figure 3 As shown, one of the GPU nodes contains online and offline containers, which are used to manage online and offline tasks respectively, in order to achieve dynamic resource adjustment and task isolation.

[0154] Optionally, some of these online and offline tasks may have multi-dimensional resource requirements. For example, some deep learning tasks not only require high-performance GPU computing power, but also rely on CPU for preprocessing, or require a large amount of memory for data loading and model storage. Therefore, the hybrid agent is responsible for monitoring the usage of multi-dimensional resources such as GPU, CPU, memory, and disk, and comprehensively considering the availability of these resources during task scheduling in order to optimize the overall computing efficiency and task execution quality.

[0155] pass Figure 2 and Figure 3 The architecture described can dynamically adjust the allocation of GPU, CPU, memory, and disk resources, ensuring sufficient resources for online tasks while making full use of unused computing resources to run offline tasks, thereby improving the overall utilization and computing efficiency of the cluster.

[0156] It should be emphasized that the specific embodiments of this application involve object-related data, such as the task-related information submitted by the objects listed above. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the objects is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0157] The task scheduling method provided by the exemplary embodiments of this application will be described below with reference to the accompanying drawings and the application scenarios described above. It should be noted that the application scenarios described above are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0158] See Figure 4 The diagram shown is a flowchart of a task scheduling method according to an embodiment of this application. The specific implementation process of this method is as follows, including S41 to S42:

[0159] S41: For each worker node in the mixed-deployment cluster, execute the following resource management operations S411 to S413 respectively:

[0160] S411: Based on the resource usage information of each deployed task under multiple specified dimensions, obtain the schedulable resource information for offline tasks.

[0161] Each deployed task is either an online or offline task deployed on this worker node. Correspondingly, each deployed task includes both online and offline tasks deployed on this worker node.

[0162] Specifically, online tasks refer to those tasks that require immediate response to user requests, including but not limited to:

[0163] (1) Web Application: Provides user interface and interactive functions. (2) API Service: Supports calls from other applications or services. (3) Database Service: Handles data storage, retrieval, and management operations, ensuring data consistency and efficient access. (4) Inference Service: Refers to tasks that process and respond to user requests in real time, such as predicting or analyzing data in real time through models to quickly provide results or decision support. (5) Real-time Transaction Processing Service: Processes customer-initiated requests for immediate purchases, payments, etc., ensuring transactions are completed quickly and securely. (6) User Authentication Service: Verifies user identity information, ensuring that only authorized users can access system resources and services. (7) Tasks in the Order Management System: Manages and monitors the entire process of orders from creation to delivery, ensuring the accuracy and timeliness of order processing.

[0164] Offline tasks refer to tasks that do not require an immediate response, and are typically used in background data processing, analysis, model training, and other scenarios, including but not limited to:

[0165] (1) Batch processing jobs: Processing large amounts of data or performing complex computational tasks. (2) Data analysis: Performing in-depth analysis of historical data to extract valuable information. (3) Log processing: Regularly collecting and analyzing system log files for monitoring system health, troubleshooting, and performance optimization. (4) Report generation: Involving background processing and analysis of large amounts of data to generate detailed reports, suitable for scenarios such as data aggregation, trend analysis, and decision support. (5) Data backup: Regularly copying important data to external storage media to prevent data loss and support disaster recovery. (6) Machine learning model training: Using large amounts of data to train complex models to improve prediction and decision-making capabilities; also known as offline training tasks.

[0166] It should be noted that the online and offline tasks listed above are just simple examples. Other online or offline tasks are also applicable to the embodiments of this application, and will not be described in detail here.

[0167] In the mixed-deployment cluster environment of this application embodiment, the status and task execution of each worker node can be very flexible. That is, for a worker node, the deployed tasks on that worker node can be empty or one or more; when there are multiple deployed tasks, these tasks can all be online tasks, all be offline tasks, or include both online and offline tasks.

[0168] The following is a brief explanation of the specific circumstances of these situations:

[0169] In practice, each worker node does not continuously execute tasks. In some cases, a worker node may not have any tasks assigned to it, meaning there are no deployed tasks on the worker node. Consider the following scenarios:

[0170] Scenario 1, Dynamic Resource Allocation: Depending on the overall load of the cluster, some worker nodes may not be assigned any tasks temporarily. These worker nodes are in an idle state, waiting to receive new task requests.

[0171] Scenario 2, Resource Reservation: In order to cope with sudden traffic or computing demands, the cluster usually reserves a portion of unused worker nodes as backup resources.

[0172] Scenario 3, Maintenance and Upgrades: Some work nodes may be temporarily taken offline for hardware maintenance, software updates or troubleshooting, during which time they will not perform any tasks.

[0173] In other words, at any given point in time, the worker nodes in a mixed-deployment cluster may include active worker nodes that are executing tasks and temporarily idle worker nodes. The scenarios listed above are just simple examples, and other possible scenarios also apply to the embodiments of this application, which will not be elaborated here.

[0174] In mixed deployment scenarios, the tasks being executed on a single worker node also offer high flexibility, including active worker nodes currently performing tasks. Specifically, the following scenarios are possible:

[0175] Scenario 1: Independent execution of online or offline tasks: A worker node may run only online tasks (such as web services or real-time transaction processing) or only offline tasks (such as batch data processing or machine learning model training). This depends on the cluster scheduling strategy and the current workload distribution, which is not specifically limited in this article.

[0176] Scenario 2: Mixed execution of online and offline tasks: In some cases, online and offline tasks can run simultaneously on the same worker node. In this case, it is necessary to ensure that the service quality of the online tasks is not affected by the offline tasks.

[0177] Scenario 3: Dynamic Task Adjustment: As the cluster load changes, the scheduler can dynamically adjust the tasks on each worker node based on real-time monitoring data. For example, during off-peak hours, the number of online tasks can be reduced, and more offline tasks can be executed using the remaining resources; while during peak hours, the resource needs of online tasks can be prioritized.

[0178] In other words, at any given point in time, the worker nodes in a mixed-deployment cluster may include those executing online tasks, those executing offline tasks, or both online and offline tasks. The scenarios listed above are merely simple examples; other possible scenarios also apply to the embodiments of this application, and will not be elaborated upon here.

[0179] It should be noted that the purpose of mixed deployment is to improve resource utilization, reduce costs by sharing resources, and ensure the stability of online tasks. Therefore, in a mixed deployment scenario, a worker node typically handles both online and offline tasks simultaneously. However, other scenarios are also possible, which will not be specifically limited in this article.

[0180] like Figure 5 As shown, it is a schematic diagram of a mixed-distribution cluster and the task deployment on each working node included in this application embodiment.

[0181] Suppose there is a hybrid cluster with four worker nodes, such as Figure 5 As shown:

[0182] At any given time, four worker nodes on this mixed-deployment cluster are executing various tasks. Two of these worker nodes are dedicated to running online tasks to ensure high response times and quality of service, such as... Figure 5 Worker nodes 3 and 4 only have online tasks deployed; the other two worker nodes execute a mix of online and offline tasks, such as... Figure 5 Worker Node 1 and Worker Node 2 in the system have a mix of online and offline tasks deployed.

[0183] In summary, in step S411, for a worker node, regardless of its state (no tasks, running only online tasks, running only offline tasks, or running both online and offline tasks), it is necessary to collect resource usage information for each deployed task on that worker node across multiple specified dimensions, and calculate the schedulable resource information for offline tasks based on these collected results. The following is a detailed explanation:

[0184] In S411, resource usage information is first collected for this working node.

[0185] Specifically, for each worker node, regardless of its current task status, resource usage information for each deployed task across multiple specified dimensions will be collected periodically. These specified dimensions include, but are not limited to, GPU, CPU, memory, disk, and storage.

[0186] The following are simple examples illustrating resource usage information under these specified dimensions (hereinafter referred to as dimensions):

[0187] (1) GPU usage.

[0188] GPU usage specifically refers to the number of GPU cards used per task. It reflects the task's demand for high-performance parallel computing and is particularly suitable for tasks requiring a large number of matrix operations, such as deep learning, image processing, and big data analysis.

[0189] In this embodiment, monitoring GPU usage ensures efficient resource utilization and identifies situations of resource idleness or overload. Of course, other information, such as video memory usage or others, can also be monitored at the GPU level; this document does not specifically limit this.

[0190] (2) CPU usage.

[0191] CPU usage specifically represents the number of CPU cores used by each task, demonstrating the task's general computing power requirements. Most types of computing tasks require a certain amount of CPU support, especially when dealing with tasks not optimized for GPUs.

[0192] In the embodiments of this application, monitoring CPU usage helps to understand the overall system load and optimize task scheduling to avoid resource bottlenecks. Of course, other information, such as CPU load or others, can also be monitored at the CPU level, and this document does not specifically limit this.

[0193] (3) Memory usage.

[0194] Memory usage records the actual amount of physical memory used by each task, which is crucial for storing frequently read and write temporary data. Sufficient memory can significantly improve task execution efficiency, especially when caching intermediate results or loading large datasets.

[0195] In this embodiment, by monitoring memory usage, memory leaks or resource shortages can be detected in a timely manner. Similarly, other memory-related usage information can also be monitored at the memory level, but this document does not specifically limit this.

[0196] (4) Disk read / write speed, also known as disk I / O bandwidth.

[0197] Disk read / write rate monitoring tracks the frequency of read / write operations and data transfer rates, reflecting the task's requirements for persistent storage solutions. Although disk access is relatively slow, its large capacity makes it suitable for long-term data storage, such as training datasets, log files, and model weights.

[0198] In the embodiments of this application, monitoring disk read / write speeds can help identify performance bottlenecks and optimize data storage strategies. Similarly, other disk-related usage information can also be monitored at the disk level, and this document does not specifically limit this.

[0199] (5) Storage.

[0200] Storage encompasses not only traditional disk storage but also more efficient storage media such as solid-state drives (SSDs) and distributed storage systems. Applications such as storing large-scale datasets and managing databases rely on efficient and stable storage solutions.

[0201] In this embodiment of the application, by monitoring storage usage, the security and availability of data can be ensured, and the allocation of storage resources can be optimized.

[0202] Specifically, in this embodiment, it is necessary not only to consider a single type of resource (such as GPU), but also to comprehensively manage multiple specified dimensions to ensure efficient system operation and effective resource utilization. For example, online inference tasks may mainly rely on GPUs for real-time prediction, but may also require a certain amount of CPU and memory to process input data and output results. Offline training tasks typically require a large amount of GPUs, CPUs, memory, and disk space for large-scale model training, and also have high requirements for network bandwidth, especially in distributed training scenarios.

[0203] Of course, in addition to the dimensions listed above, appropriate monitoring dimensions can be selected based on specific application scenarios, such as Input / Output Operations Per Second (IOPS). IOPS can be used to assess the I / O requirements of a specific task on the storage system. By monitoring the IOPS of a single task, one can more accurately understand the task's usage of storage resources and its impact on other tasks. Monitoring these specific resources helps to gain a more comprehensive understanding of the system's operating status and to make targeted optimizations and adjustments. For example, a large-scale data processing task that frequently reads and writes data from the disk has high IOPS requirements and may consume a large amount of storage bandwidth, thereby affecting the I / O performance of other tasks running on the same worker node. Therefore, monitoring IOPS in a high-concurrency environment can effectively prevent storage performance problems; and so on.

[0204] It should be noted that the specified dimensions and corresponding resource usage information listed above are merely simple examples. Other dimensions and corresponding resource usage information also apply to the embodiments of this application, and will not be elaborated upon here. In the embodiments of this application, by periodically collecting resource usage information of each deployed task under these specified dimensions, the actual resource status of the work nodes can be dynamically grasped, providing strong support for subsequent task scheduling and resource management.

[0205] Optionally, in addition to the task-level resource usage information listed above, node-level resource usage information can also be collected, such as network bandwidth. Network bandwidth is used to assess the data flow to and from worker nodes, affecting the speed and efficiency of communication between worker nodes, which is especially important in distributed training. Data transmission, parameter synchronization, and worker node communication in distributed computing all rely on sufficient network bandwidth.

[0206] In this embodiment, monitoring the network bandwidth of the working node can help identify network congestion points and optimize data transmission strategies to improve overall efficiency.

[0207] It should be noted that the resource usage information at the worker node level listed above is only a simple example. Resource usage information in other dimensions is also applicable to the embodiments of this application, and will not be described in detail here. In the embodiments of this application, by periodically collecting resource usage information of each worker node under these specified dimensions, the actual resource status of the worker node can be dynamically grasped, providing strong support for subsequent task scheduling and resource management.

[0208] When collecting multi-dimensional resource information for each deployed task on a worker node, the collection details are as follows for the different scenarios mentioned above:

[0209] (1) Idle state: If there are no deployed tasks on the worker node, the relevant resource usage information will be displayed as zero or close to zero. At this time, all available resources on the worker node can be regarded as potential schedulable resources. (2) Online tasks only: The focus is on monitoring the resource usage of online tasks to ensure that they can respond to user requests quickly. (3) Offline tasks only: The focus is on the resource consumption of offline tasks in order to optimize their execution efficiency and completion time. (4) Hybrid tasks: When online and offline tasks are running on this worker node at the same time, it is necessary to record the resource usage of each task in detail in various dimensions, especially to identify which resources may become bottlenecks and affect the service quality of online tasks.

[0210] In S411, after collecting multi-dimensional resource usage information for this working node, a series of calculations can be performed based on the collected resource usage information to determine the remaining amount of resources available for scheduling new offline tasks on the current working node, thus providing the scheduling resource information. The specific steps are as follows:

[0211] First, summarize resource usage. Specifically, summarize the total resource usage of tasks deployed on this worker node across various specified dimensions. For example, calculate the total CPU usage, total memory usage, total GPU usage, and total disk I / O bandwidth for these tasks.

[0212] Next, the remaining resources are calculated. Specifically, the total resource usage of deployed tasks is subtracted from the total resource capacity of the worker nodes to obtain the remaining available resources.

[0213] The following example uses the collection of resource usage information such as CPU usage, GPU usage, memory usage, and disk I / O bandwidth:

[0214] Suppose there is a worker node that is currently running two online tasks (denoted as online task A and online task B) and one offline task (denoted as offline task C).

[0215] See Figure 6As shown, this is a schematic diagram illustrating the statistical results of resource usage information in an embodiment of this application. Figure 6 The following is a summary of the multi-dimensional resource usage information for each task at this work node, along with the corresponding summation of used resources (obtained by summing each dimension):

[0216] (1) CPU usage (i.e.) Figure 6 (CPU usage): Online task A uses 3 CPU cores (i.e., 3 cores), online task B uses 2 CPU cores (i.e., 2 cores), and offline task C uses 1 CPU core (i.e., 1 core). Therefore, the total CPU usage = 3 + 2 + 1 = 6 cores.

[0217] (2) GPU usage (i.e.) Figure 6 (GPU card usage): Offline task C uses 1 GPU card (i.e., 1 card). Therefore, total GPU usage = 1 card.

[0218] (3) Memory usage (i.e. Figure 6 (Memory usage): Online task A used 2GB, task B used 1GB, and offline task C used 5GB. Therefore, total memory usage = 2GB + 1GB + 5GB = 8GB.

[0219] (4) Disk I / O bandwidth (i.e. Figure 6 (Disk I / O bandwidth): Online Task A: Read rate 50MB / s, Write rate 20MB / s. Online Task B: Read rate 30MB / s, Write rate 10MB / s. Offline Task C: Read rate 100MB / s, Write rate 50MB / s. Therefore, the total disk I / O bandwidth is: Read = 50MB / s + 30MB / s + 100MB / s = 180MB / s; Write = 20MB / s + 10MB / s + 50MB / s = 80MB / s.

[0220] Optionally, the resource usage information for online tasks mentioned in this paper can also be predicted using algorithms, including but not limited to: time series analysis based on historical data, such as Autoregressive Integrated Moving Average (ARIMA) and exponential smoothing; machine learning methods such as regression trees and support vector machines; and deep learning techniques such as Long Short-Term Memory (LSTM) networks, which are particularly suitable for handling data with long-term dependencies. Furthermore, reinforcement learning can be used to adjust the prediction model based on real-time feedback, improving the accuracy of resource usage predictions. These algorithms help accurately predict the resource requirements of online tasks by analyzing existing data, and this paper does not specifically limit their application.

[0221] See Figure 7 As shown, this is a schematic diagram of the calculation logic for schedulable resource information in an embodiment of this application. Figure 7 Assuming the total resources of this worker node are: 8 CPU cores, 32GB of memory, and 2 GPUs; the maximum disk I / O bandwidth is 500MB / s for read speed and 200MB / s for write speed. Then, by subtracting the total node resources from the summary result (subtracting each dimension separately), we can obtain the remaining resources, and thus the scheduling resource information, as follows:

[0222] If the total CPU usage is 6 cores, then the remaining 2 CPU cores (i.e., 2 cores) can be used for new offline tasks.

[0223] Total memory usage is 8GB, with 24GB of memory remaining available for new offline tasks.

[0224] The total GPU usage is 1 card, and the remaining 1 GPU card (i.e., 1 card) can be used for new offline tasks.

[0225] With a total disk I / O bandwidth of 180MB / s for reading and 80MB / s for writing, the remaining disk I / O bandwidth available for new offline tasks is: Read: 500MB / s - 180MB / s = 320MB / s; Write: 200MB / s - 80MB / s = 120MB / s.

[0226] In summary, after deducting the resource requirements of deployed tasks (including online and offline tasks), the remaining available resources are determined. These resources are the resources that can be used to schedule new offline tasks, i.e., the scheduleable resource information, as follows:

[0227] The current working node can provide up to 2 CPU cores, 24GB of memory, and 1 GPU card for new offline tasks. Disk I / O bandwidth: read: 320MB / s, write: 120MB / s.

[0228] This information will serve as an important basis for subsequent task scheduling decisions, ensuring that both the high-performance requirements of online tasks are met and resources are used efficiently to execute offline tasks.

[0229] In this way, we can dynamically understand the actual resource status of each working node and make reasonable task scheduling decisions accordingly, ensuring that we can meet the high performance requirements of online tasks and efficiently utilize resources to execute offline tasks.

[0230] Optionally, after calculating the remaining resources but before generating schedulable resource information, the reserved resources can be adjusted. Specifically, to ensure the service quality of online tasks, a portion of resources can be reserved for offline task scheduling.

[0231] The proportion of these reserved resources can be set based on business needs and Service Level Agreements (SLAs). For example, if critical online services require higher priority, the proportion of reserved resources can be increased; conversely, it can be appropriately reduced to optimize overall resource utilization and cost-effectiveness. This ensures both the quality of online services and the flexibility to allocate resources to meet different business needs.

[0232] Using the above example, suppose that to ensure the quality of online tasks, it is decided to reserve 1 CPU core and 10GB of memory for offline task scheduling. Then the actual schedulable resources become: 1 CPU core and 14GB of memory. The schedulable resource information indicates that the current working node can provide a maximum of 1 CPU core and 14GB of memory for a new offline task.

[0233] The above implementation involves management and scheduling of various specified dimensions to improve the overall resource utilization of mixed-distribution clusters (such as GPU clusters) and ensure effective isolation and collaborative work between online and offline tasks.

[0234] Furthermore, in order to enhance the self-management capabilities of each working node and to perform resource management operations periodically, this application introduces a hybrid deployment component.

[0235] Optionally, each worker node includes a hybrid deployment component, which is responsible for periodically performing resource management operations on each worker node, namely the steps included in S41 of this paper, such as S411 to S413. This mechanism not only improves the resource utilization of individual worker nodes, but also better supports the resource optimization and scheduling decisions of the overall system.

[0236] For example Figure 3 The hybrid proxy shown is an example of a hybrid component on a worker node in this application embodiment. This hybrid component can be a standalone software module or a service integrated into the worker node; this document does not specifically limit its functionality.

[0237] In this embodiment of the application, when the hybrid component performs resource management operations periodically, it can do so at preset fixed time intervals. These time intervals can be flexibly set according to experience, business needs, system load, etc.

[0238] Optionally, the time interval can be dynamically adjusted based on business needs and load conditions to quickly respond to resource changes. For example, it can be executed once per minute during off-peak hours and shortened to once every 10 seconds during peak hours; or once per hour during off-peak hours and shortened to once every 10 minutes during peak hours; and so on. This flexible time setting helps maximize resource utilization and system stability while ensuring online service performance.

[0239] Optionally, in addition to periodic execution, resource management operations performed by the mixed deployment component can also be triggered by specific events, such as resource usage exceeding a preset threshold, the arrival of a new task, or the completion of an existing task. This approach makes resource management more flexible and responsive, enabling real-time adjustments to resource allocation to cope with unforeseen circumstances, ensuring the stability and efficiency of online services, while optimizing resource utilization for offline tasks.

[0240] Taking periodic execution as an example, for instance, the mixed deployment component periodically (e.g., every 10 minutes) collects the resource usage information of each deployed task on the corresponding work node and the total resource status of the node, and calculates the schedulable resource information corresponding to the work node, i.e., S411. For specific implementation details, please refer to the above embodiments.

[0241] Optionally, each hybrid component includes an offline resource manager, which is used to collect resource usage information of each deployed task and the total resource status of the node. Since the embodiments in this application require the collection of multi-dimensional resource usage information, the offline resource manager can consist of multiple sub-dimensional offline resource managers, such as a CPU offline resource manager, a GPU offline resource manager, a memory offline resource manager, and a disk offline resource manager.

[0242] The offline resource manager acts as the top-level manager, responsible for initializing and coordinating the work of each sub-offline resource manager. Each sub-offline resource manager focuses on a specific type of resource and is responsible for monitoring and evaluating the status of that type of resource.

[0243] During the initialization phase, when a worker node starts or restarts, the offline resource manager first initializes, starting its various sub-offline resource managers, such as the GPU offline resource manager, CPU offline resource manager, memory offline resource manager, and disk offline resource manager. Each sub-offline resource manager performs initialization operations on its corresponding specified dimension, such as setting initial parameters (e.g., periodic time intervals) and connecting to the appropriate hardware interface or monitoring tools.

[0244] In this way, each sub-offline resource manager periodically evaluates the resource status of its respective domain according to preset time intervals. For example, the GPU offline resource manager monitors GPU usage, video memory usage, etc., and assesses the amount of GPU resources available to support offline tasks; the memory offline resource manager monitors the memory usage of nodes and assesses the amount of memory available for offline tasks; the CPU offline resource manager monitors CPU usage, CPU load, and other metrics and assesses the amount of CPU resources available for offline tasks.

[0245] Optionally, each sub-offline resource manager can consider the elasticity and inelasticity of the resources it manages, and analyze the resource usage of online and offline tasks as well as the total resource situation of the node. For example:

[0246] The memory offline resource manager identifies which parts are elastic resources and which are non-elastic resources; the CPU offline resource manager considers CPU usage and calculates the number of remaining CPU cores (i.e., the number of CPU cores) that can be used for offline tasks; and so on.

[0247] It should be noted that the analysis of elastic and inelastic characteristics can be used to release resources for target offline tasks using different release strategies, which will not be repeated here.

[0248] Optionally, to ensure the service quality of online tasks, a portion of resources can be reserved for non-offline task scheduling. The proportion of these reserved resources can be set according to business needs and SLAs. For example, one CPU core can be reserved for high-priority online tasks.

[0249] Based on this, each sub-resource manager calculates the corresponding schedulable resource information. For example, the CPUoffline resource manager subtracts the total CPU usage of deployed tasks from the total number of CPU cores on the current worker node, and then subtracts the reserved resources to obtain the remaining available CPU cores; the GPU offline resource manager similarly calculates the number of GPUs available for new offline tasks; and so on.

[0250] Through this layered collaboration approach, the offline resource manager and its sub-offline resource managers can effectively manage and optimize resource allocation, ensuring that worker nodes can meet the needs of online tasks while making the most efficient use of remaining resources to support offline tasks.

[0251] Optionally, each hybrid component also includes a resource detection controller. Each sub-resource manager can report its computed schedulable resource information to the resource detection controller. For example, the GPU offline resource manager reports the amount of GPU resources available for new offline tasks on the current worker node; the CPU offline resource manager reports the amount of CPU resources available for new offline tasks on the current worker node; and so on.

[0252] Optionally, when each sub-offline resource manager feeds back the above information to the resource detection controller, it can use a unified data format, such as JavaScript Object Notation (JSON) or Extensible Markup Language (XML), and transmit information through APIs or message queues to ensure efficient and accurate data exchange.

[0253] The resource monitoring controller receives data from each sub-resource manager, integrates and optimizes it. It may further adjust resource allocation strategies to ensure maximum overall resource utilization and guaranteed online task quality.

[0254] Optionally, based on the above calculation results, the resource detection controller can also generate a detailed schedulable resource report to describe the schedulable resource information of the current worker node. For example, the report may indicate that the current worker node has 4 CPU cores, 8GB of memory, and 1 GPU card available for new offline tasks.

[0255] Furthermore, the hybrid deployment component can also execute S412 and S413, and finally, through the hybrid control management layer, report the schedulable resource information to the scheduler in the upper layer, so that the scheduler can schedule the offline tasks to be scheduled. Specifically, the resource detection controller in the hybrid deployment component can report schedulable resource information, interference detection information, etc., to the hybrid control management layer. Detailed implementations of S412 and S413 are described below and will not be repeated here.

[0256] The above implementation significantly improves the self-management capabilities and resource utilization efficiency of each working node by introducing a hybrid deployment component. First, the hybrid deployment component can periodically perform resource management operations, ensuring that node resources are rationally allocated and efficiently utilized, avoiding resource waste and over-reservation. Second, by providing detailed resource management feedback to the hybrid deployment control layer, this method supports global resource optimization and dynamic adjustment, further enhancing the system's flexibility and adaptability, ensuring efficient operation even in complex multi-cluster environments.

[0257] S412: Perform resource interference detection based on multiple specified dimensions to obtain conflict resource information; whereby the conflict resource information represents the resource details that interfere with online tasks due to insufficient resources.

[0258] During offline and online deployment, offline tasks reuse idle resources from online tasks, leading to resource contention and conflicts. To ensure the service quality of online tasks is not affected by offline tasks, this application proposes a method for resource interference detection based on multiple specified dimensions. By monitoring kernel metrics across multiple different specified dimensions, the method detects the existence of specified dimensions that interfere with online tasks and adjusts resource allocation or scheduling strategies accordingly. An optional implementation is as follows:

[0259] Under each specified dimension, acquire the resource indicator data related to that specified dimension; then, combine this resource indicator data, analyze the resource usage information of the working node under the specified dimension through resource interference detection, and identify bottlenecks or conflict points that may affect the performance of online tasks; if potential interference is detected, generate conflict resource information of the working node under that specified dimension based on the analysis results.

[0260] Specifically, first, establish reasonable resource metrics for each dimension, including but not limited to thresholds and baselines. These values ​​can be determined based on the SLA of online tasks, performance requirements, and historical data. For example, CPU usage exceeding 80% (e.g., using more than 8 out of 10 CPU cores) may be considered high load; memory usage approaching physical limits may lead to swapping, impacting performance. By setting these thresholds and baselines, it's possible to more accurately determine whether resource usage exceeds normal limits and respond promptly.

[0261] Subsequently, for each specified dimension, corresponding algorithms are applied to detect any interference with online tasks. This includes, but is not limited to, methods such as trend analysis, anomaly detection, and correlation analysis. For example, time series analysis is used to detect long-term trends and short-term fluctuations in resource usage, statistical methods are used to identify resource usage exceeding normal ranges, and correlations between different dimensions are examined to identify patterns or combinations of conditions that may indicate resource conflicts. This method can effectively identify potential resource competition problems and provide a basis for subsequent optimization.

[0262] Thus, once resource usage in a certain dimension is detected to exceed a set threshold and affect the normal operation of online tasks, detailed conflict resource information will be generated. This information includes, but is not limited to, some or all of the following: the types of resources affected and a list of affected tasks (i.e., which resources are experiencing problems and their scope of impact), specific resource usage details, and recommended measures. For example, it may indicate which resource is conflicting, list the online tasks affected by the resource conflict and their detailed information, and propose some feasible solutions, such as adjusting task priorities or migrating some tasks to other nodes.

[0263] In this embodiment, by monitoring key indicators such as CPU, memory, and disk I / O bandwidth, interference sources can be accurately identified, and resource allocation strategies can be dynamically adjusted to ensure that online tasks receive sufficient resource support. Furthermore, continuous monitoring and optimization reduce the negative impact of resource contention, improving system stability and efficiency. This step not only detects the existence of resource conflicts but also provides detailed information on conflicting resources, guiding subsequent operations to ensure the service quality of online tasks and guarantee efficient and stable operation.

[0264] Optionally, the hybrid component also includes an interference detection controller. S412 can be implemented using an interference detection controller. Optionally, each specified dimension can correspond to an independent interference detection controller to monitor the key kernel indicators of the resource it is responsible for in real time, identify potential resource bottlenecks or conflict points, and report relevant information to the resource detection controller for subsequent steps. An optional implementation is as follows:

[0265] Under each specified dimension, an independent interference detection controller is activated to acquire relevant resource metric data, such as CPU usage, memory consumption, and disk I / O bandwidth. Combining this resource metric data, interference detection analysis is used to identify bottlenecks or conflict points that may affect the performance of online tasks. Once potential interference is detected, an alarm message containing detailed kernel metric data and analysis results is immediately generated and reported to the resource detection controller. Based on the conflicting resource information, the resource detection controller determines whether resource allocation needs to be adjusted or measures taken (such as migrating tasks, releasing offline resources, etc.) to ensure that the service quality of online tasks is not affected.

[0266] Specifically, in S412, multiple independent interference detection controller threads are launched for different specified dimensions. Each controller focuses on a specific type of resource and monitors key kernel metrics for that resource in real time. For example, the CPU interference detection controller monitors CPU usage (and could also be context switching frequency), the memory interference detection controller monitors memory usage (and could also be page swapping), and the I / O interference detection controller monitors disk read / write rates. By analyzing historical and real-time monitoring data, bottlenecks that may cause performance degradation in online tasks are identified. Furthermore, machine learning algorithms are used to predict potential future resource bottlenecks and make adjustment suggestions in advance.

[0267] When resource usage in a certain dimension may interfere with online tasks, the interference detection controller can generate corresponding conflict resource information. Each interference detection controller will then report the generated conflict resource information to the resource detection controller. Optionally, the reported information format can be standardized, such as using the unified data format listed above, to facilitate further comprehensive analysis and decision-making by the resource detection controller.

[0268] This detailed monitoring and analysis mechanism enables the timely detection and resolution of potential resource management issues, ensuring the stability and efficiency of online tasks. Furthermore, the collaborative work of independent controllers in each dimension ensures the system's flexibility and adaptability, making resource management more intelligent and efficient.

[0269] like Figure 8 As shown, this is a schematic diagram illustrating the interaction logic between the offline resource manager, interference detection controller, and resource detection controller within a hybrid component according to an embodiment of this application. Simply put, the offline resource manager is responsible for resource detection based on its multiple internal sub-offline resource managers, such as... Figure 8 The process includes GPU resource detection, CPU resource detection, memory resource detection, disk resource detection, etc., to collect schedulable resource information for each specified dimension and report it to the resource detection controller, as shown in the dashed box 81. The implementation of this process can be found in the relevant description in section S411 above, and will not be repeated here.

[0270] The interference detection controller is responsible for detecting interference across various specified dimensions, such as... Figure 8 The system performs GPU interference detection, CPU interference detection, memory interference detection, and disk interference detection, collects conflict resource information, and reports it to the resource detection controller, as shown in dashed box 82. The implementation of this process can be found in the relevant description in section S412 above, and will not be repeated here. Multiple interference detection controllers can be used. Figure 8 This is just a simple example.

[0271] It should be noted that the timing of S411 and S412 is not specifically limited in this article. They can be executed simultaneously or sequentially, such as executing S411 first and then S412, or executing S412 first and then S411.

[0272] Based on this, the resource detection controller receives conflicting resource information and schedulable resource information, analyzes this data to determine if there are any situations requiring adjustment, especially when it is found that certain resources are causing or may cause a decline in the performance of online tasks. If it is determined that it is necessary to reduce the resources occupied by offline tasks to protect the quality of online tasks, the resource detection controller will decide how to perform resource compression, for example, by selecting a suitable target offline task from the current working nodes for resource release. For specific implementation details, please refer to the embodiments in the relevant part of S413 below, which will not be repeated here.

[0273] S413: Based on the schedulable resource information and conflicting resource information, release resources for at least one target offline task among the deployed tasks, and then update the schedulable resource information.

[0274] In step S413, based on schedulable resource information and conflicting resource information, the system can identify and select at least one target offline task for resource release in order to alleviate resource conflicts and ensure the service quality of online tasks.

[0275] In practice, the first step is to select suitable target offline tasks. Then, based on a predefined release strategy (such as by priority or elasticity), resources are released for these target offline tasks. Specifically, some or all of their resources can be released, and the schedulable resource information on the nodes is updated. This ensures that subsequent task scheduling decisions are optimized based on the latest resource status, thereby improving the resource utilization efficiency and stability of the entire cluster.

[0276] To effectively manage resources and ensure that the service quality of online tasks is not affected by offline tasks, this application proposes to dynamically adjust the resource allocation of offline tasks based on schedulable resource information and conflicting resource information. Specifically, in step S413, based on the detected resource management anomalies, appropriate offline tasks are identified and selected for resource release, and the schedulable resource information on the node is updated to maximize resource utilization and improve system stability.

[0277] Therefore, an optional implementation method is to carry out S413 according to the following process, including the following steps S4131 to S4133 ( Figure 4 (Not shown):

[0278] S4131: Based on schedulable resource information and conflicting resource information, analyze whether there are any resource management anomalies; where resource management anomalies indicate that the amount of resources occupied by offline tasks needs to be reduced.

[0279] To ensure that the service quality of online tasks is not affected by offline tasks, the system performs resource management anomaly detection. Specifically, it first analyzes whether there are resource management anomalies in the current working node based on schedulable resource information and conflicting resource information.

[0280] In this embodiment, resource management anomaly refers to a situation where, based on the current schedulable resource information and conflicting resource information, one or more of the following states are found on a working node. These states can be categorized as resource overload, resource waste, resource conflict, resource bottleneck, etc.

[0281] The following is a brief explanation of these states and their analysis process:

[0282] (1) Resource overload.

[0283] Resource overload refers to a situation where the usage of certain critical resources (such as CPU, memory, I / O, etc.) continuously exceeds a preset safety threshold, causing online tasks to be unable to obtain sufficient resources, thereby affecting service quality. In this embodiment, it can be determined by checking whether there are cases where resource usage has been exceeding the safety threshold for an extended period.

[0284] For example, if CPU usage consistently exceeds 80% (a safety threshold, such as continuously using more than 8 out of 10 CPU cores), it will lead to longer response times for online tasks. Another example is when memory usage approaches physical limits, frequently triggering swapping and impacting system performance.

[0285] (2) Waste of resources.

[0286] Resource waste refers to the existence of a large amount of unused resources, indicating unreasonable resource allocation or excessive reservation, resulting in low resource utilization. In the embodiments of this application, this can be determined by assessing whether a large amount of unused resources exist.

[0287] Resource over-issuance is a specific form of resource waste, which refers to allocating more resources than actually needed for a task, resulting in resource waste.

[0288] For example, allocating far more CPU cores to a task than it actually needs results in idle resources. Or, excessive memory resources are reserved for high-priority tasks, but these resources are not actually utilized.

[0289] (3) Resource conflict (also known as resource competition conflict).

[0290] Resource conflict refers to a severe resource competition between online and offline tasks due to improper resource allocation, which interferes with the normal operation of online tasks. In this embodiment, this can be determined by analyzing whether improper resource allocation is causing severe resource competition between online and offline tasks.

[0291] For example, offline tasks consume a large amount of CPU resources, leading to increased response latency for online tasks. Another example is multiple tasks competing for limited disk read / write bandwidth, causing a decrease in data read / write speed and impacting online processing efficiency.

[0292] (4) Resource bottleneck.

[0293] A resource bottleneck refers to the identification of a specific resource that becomes a bottleneck in system performance, causing overall efficiency to decrease even when other resources have spare capacity. In the embodiments of this application, this bottleneck can be identified by evaluating whether a specific resource becomes a bottleneck in system performance.

[0294] For example, despite sufficient CPU and memory resources, insufficient network bandwidth can lead to slow data transmission, impacting the completion time of distributed computing tasks. Another example is when a worker node reaches its IOPS limit, becoming the performance bottleneck of the entire system.

[0295] If any one or more of the above-mentioned resource management anomalies are detected, it indicates a need to reduce the amount of resources consumed by offline tasks to alleviate resource pressure and optimize the resource utilization and system stability of the entire cluster. In this case, a reasonable resource release strategy can effectively solve these problems, ensure the service quality of online tasks, and guarantee the stable and efficient operation of the system.

[0296] In summary, if a resource management anomaly is detected in S4131, it indicates that the usage of certain critical resources (such as CPU, memory, disk I / O bandwidth, etc.) continuously exceeds the preset safety threshold, causing online tasks to be unable to obtain sufficient resources, thereby affecting service quality. Therefore, further measures are needed to mitigate these problems, such as continuing to execute subsequent processes (e.g., S4132-S4133) to identify and select appropriate target offline tasks for resource release and update schedulable resource information to ensure the service quality of online tasks.

[0297] If no resource management anomalies are found, it indicates that the usage of critical resources is within safe thresholds, and there are no issues such as resource overload, waste, conflicts, or bottlenecks. In this case, the existing resource allocation strategy is sufficient to meet the needs of both online and offline tasks, requiring no additional resource release or adjustment. Therefore, there is no need to perform a subsequent resource release process; the current resource allocation status can be maintained, and the system's operational status can continue to be monitored to ensure efficient and stable resource usage. In other words, in this situation, step S413 can be ignored, and step S414 can be executed directly; alternatively, it can be assumed that the schedulable resource information has not changed before and after the update, and step S414 can then be executed.

[0298] The execution process of S4132 to S4133 is explained below:

[0299] S4132: If a resource management anomaly exists, select at least one target offline task that matches each target dimension from all deployed tasks; wherein each target dimension is the dimension with a resource management anomaly among multiple specified dimensions.

[0300] Specifically, if a resource management anomaly is confirmed in the current worker node, at least one target offline task matching each target dimension must be selected from the deployed tasks on that worker node. Each target dimension is the dimension with resource management anomalies among multiple specified dimensions. For example:

[0301] If CPU usage is too high, select offline tasks that consume significant CPU resources as the target. If disk I / O bandwidth is nearing its limit, select offline tasks that frequently perform disk reads and writes as the target.

[0302] Optionally, for a worker node, when selecting a target offline task from among the deployed tasks on that worker node, one or more of the following factors can be considered to optimize resource management and scheduling decisions:

[0303] (1) Elasticity: Prioritize offline tasks that are more tolerant of resource reduction, i.e., offline tasks with high elasticity. Here, the tolerance of an offline task to resource changes is measured, i.e., the extent to which an offline task can adapt to resource reduction without significantly affecting its performance or results. Offline tasks with high elasticity typically refer to those that are less sensitive to resource fluctuations, such as certain batch processing offline tasks or data analysis offline tasks.

[0304] For example, some offline big data analytics tasks may continue to run even with reduced resources. Although the processing speed may decrease, it will not cause the offline task to fail or the results to be distorted.

[0305] (2) Priority: In the resource management process, low-priority offline tasks are prioritized for resource release to minimize the impact on important business operations. For example, offline tasks can be classified according to their importance and urgency, such as into three levels: high, medium, and low. Higher-priority offline tasks have a greater impact on business operations and require more resource support; while lower-priority offline tasks have a smaller impact on business operations and their resource usage can be appropriately reduced when resources are scarce.

[0306] For example, in a financial trading platform, historical data backup and market trend analysis are low-priority offline tasks that can reduce their CPU or memory usage when resources are scarce.

[0307] (3) Task execution status: Select the target offline task based on the current execution status of the offline task (e.g., running, waiting, paused, etc.). For example, prioritize offline tasks that are in a waiting state or can be paused for resource release.

[0308] For example, some batch offline tasks can be paused at any point in time and resumed later without affecting the final result.

[0309] (4) Historical performance: The adaptability of offline tasks is evaluated based on their historical performance data (such as completion time, failure rate, resource usage patterns, etc.). For example, offline tasks that have shown strong adaptability in the past are selected.

[0310] For example, some offline tasks can maintain good performance even under resource fluctuations, and these offline tasks can be prioritized for resource adjustments.

[0311] (5) Dependencies: Analyze the dependencies between offline tasks to ensure that the selected offline tasks will not break critical dependency chains due to resource release. For example, avoid selecting offline tasks that depend on other important offline tasks.

[0312] For example, if a high-priority offline task depends on the result of a low-priority offline task, the resource usage of the low-priority offline task should not be reduced easily.

[0313] (6) Task Type: Differentiate the requirements and tolerances of different offline tasks based on their type (e.g., batch processing, data analysis, etc.). For example, prioritize resource release for offline tasks that are not processed in real time.

[0314] For example, batch processing offline tasks typically have a high tolerance for latency, so their resource usage can be appropriately reduced without affecting the overall business process.

[0315] (7) Time window: Selected based on the time sensitivity of offline tasks and the needs of specific time periods. For example, identify offline tasks that can release resources during specific time periods (such as off-peak periods).

[0316] For example, some offline log processing tasks can have their resources adjusted during off-peak hours at night to reduce the impact on the system.

[0317] (8) Resource Bottlenecks: Identify offline tasks that consume a lot of specific resources (such as CPU, memory, disk I / O bandwidth, etc.), especially when these resources become system bottlenecks. For example, prioritize releasing resources from offline tasks that consume a lot of bottleneck resources.

[0318] For example, when the system detects that memory has become a bottleneck, it can select offline tasks that consume a lot of memory for resource adjustment.

[0319] (9) Cost-effectiveness: Evaluate the cost-effectiveness of resource usage based on the economic benefits of offline tasks (such as computing costs, storage costs, etc.). For example, select offline tasks with lower economic benefits for resource release.

[0320] For example, in a cloud computing environment, some offline tasks may consume a lot of expensive resources but bring limited benefits. It is advisable to prioritize reducing the resource usage of such offline tasks.

[0321] In practical applications, each task can be scored by comprehensively considering one or more of the above factors, and resources can be released for the task with the lowest score. Furthermore, resource management methods can be flexibly adjusted according to different application scenarios and needs, improving the system's flexibility and adaptability, ensuring efficient operation and maintaining the continuity of critical business operations even with limited resources. This approach not only improves the scientific rigor and accuracy of resource release decisions but also minimizes the impact on system performance.

[0322] Furthermore, it should be noted that the factors listed above are merely simple examples. In addition, other selection methods are also applicable to the embodiments of this application. For example, a flexible configuration interface can be provided to allow administrators or users to set custom rules (such as some users may want to prioritize reducing the resource usage of certain types of offline tasks to meet special business needs), thereby flexibly defining which offline tasks should be prioritized as the objects of resource release; and so on. These will not be elaborated on in this article.

[0323] To more accurately select target offline tasks that require resource release, this application further proposes a method for screening target offline tasks, as described below. This method not only considers the existence of resource management anomalies but also determines target offline tasks by screening task combinations that meet specific matching conditions. An optional implementation is as follows:

[0324] Tasks from the target task combinations selected from various task combinations are designated as target offline tasks. For a given worker node, each task combination consists of at least one offline task from among the deployed tasks on that worker node that consumes resources in the target dimension. This approach, by combining different offline tasks, facilitates finding task sets that satisfy resource release requirements while minimizing system impact. This simplifies resource management operations, reduces complexity and risk, and ensures efficient resource utilization and system stability.

[0325] Optionally, the combination of target tasks satisfies at least one of the following matching conditions:

[0326] Matching condition 1: Under each target dimension, the total amount of resources occupied by the combination of target tasks reaches the expected reduction in resources.

[0327] Specifically, for each objective dimension, the total amount of resources required for each task combination is calculated. If this total amount reaches or exceeds the expected reduction in resources (i.e., the expected resource release target), then the task combination is considered to meet this condition.

[0328] For example, if the goal is to reduce memory usage by 10GB, and all tasks in a given task set consume a total of 12GB of memory, then that task set meets the requirement because its memory usage exceeds the desired reduction (10GB), making it a candidate for resource adjustment or optimization. This matching ensures the effectiveness of resource release.

[0329] Matching condition two: The sum of the sacrifice levels of each task in the target task combination is minimized; the sacrifice level of each task represents the intensity of the impact of the execution release strategy on the execution state of the corresponding task.

[0330] The degree of sacrifice refers to the impact of resource release measures taken to mitigate resource management anomalies on the affected tasks. In the embodiments of this application, the degree of sacrifice can be assessed based on one or more of the following factors:

[0331] (1) Elasticity: See the relevant description above for details. Optionally, the higher the elasticity of an offline task, the lower the corresponding sacrifice. For an offline task, its elasticity represents the elasticity of the resources it occupies. For example, the elasticity is sorted from low to high as follows: elastic resources < inelastic resources.

[0332] (2) Performance loss: refers to the degree of performance degradation of a task after resource reduction. It is usually measured by comparing performance indicators (such as response time, throughput, etc.) before and after resource reduction. Optionally, the greater the performance loss, the greater the sacrifice.

[0333] If a task is highly sensitive to resource changes, even a small reduction in resources can lead to a significant performance degradation. For example, some real-time transaction processing systems may experience a substantial increase in response time after a reduction in CPU or memory resources, thus impacting user experience and business efficiency. Therefore, the performance loss for such tasks is significant, and the degree of sacrifice is also high.

[0334] (3) Recovery cost: refers to the time and resource cost required for the task to regain the necessary resources and resume normal operation, including but not limited to restart time, data recovery time, and the cost of reallocating resources. Optionally, the higher the recovery cost, the greater the sacrifice.

[0335] Some tasks may take a long time to recover from a resource-reduced state, or the recovery process may require additional resources. For example, some batch jobs may need to be restarted after resource reduction, requiring the reloading of large amounts of data, which can lead to high recovery costs. Therefore, these types of tasks are more vulnerable, as they are not only affected by resource reduction but also consume more resources and time during the recovery process.

[0336] (4) Business Impact: This refers to the impact of resource release on business functions and service quality, especially on critical business operations. Business impact can include direct service interruptions, decreased service quality, or reduced user satisfaction; this paper does not specify a particular impact. Optionally, the greater the business impact, the greater the degree of sacrifice.

[0337] Some tasks can directly impact the normal operation of critical business activities. If these tasks cannot operate normally due to resource reductions, it will have a significant impact on the entire business, potentially leading to severe financial losses or customer churn. Therefore, the business impact of these tasks is greater, and the degree of sacrifice is also higher. Conversely, some non-critical tasks, even if affected by resource reductions, will not have a significant impact on the overall business, and the degree of sacrifice is relatively lower.

[0338] In practical applications, the degree of sacrifice can be expressed as the evaluation result of one of the above factors, that is, by evaluating a certain factor, a corresponding score is obtained, and this score represents the degree of sacrifice; the degree of sacrifice can also be expressed as the comprehensive evaluation result of multiple factors, that is, by weighted summation of the scores corresponding to each factor, and other methods such as scoring, to obtain a comprehensive score, which represents the degree of sacrifice. By comprehensively considering these factors, the degree of sacrifice for each task can be evaluated more comprehensively. Of course, the factors listed above are just simple examples, and other relevant factors are also applicable to the embodiments of this application, which will not be elaborated here.

[0339] Specifically, the impact of the release strategy on the execution state of each task (i.e., the degree of sacrifice) is calculated, and the task combination with the smallest total degree of sacrifice is selected. For example, some tasks may be more sensitive to resource changes, and reducing their resources may lead to a significant performance drop; therefore, such tasks should be avoided as much as possible. Furthermore, in situations of resource scarcity, priority should be given to tasks with minimal performance loss, low recovery costs, and minimal impact on business operations for resource adjustments, thereby achieving efficient and stable system operation.

[0340] Matching condition 3: The total number of tasks in the target task combination is the minimum.

[0341] Specifically, select the task combination with the fewest tasks. This helps simplify subsequent operations, reduces the complexity and potential risks caused by resource adjustments, and ensures the effective release of resources and the stable operation of the system.

[0342] For example, if there are two task combinations, one involving adjusting the resource allocation of five tasks and the other requiring only adjustments to two tasks, choose the latter. This approach not only improves efficiency but also reduces the uncertainty that might arise from adjusting multiple tasks.

[0343] In practical applications, the first step is to select at least one offline task from the deployed tasks that consumes resources in these dimensions based on the target dimension of resource scarcity (such as CPU, memory, etc.), and then form multiple task combinations.

[0344] Next, these task combinations are evaluated based on matching criteria. From the task combinations that satisfy any of the above matching criteria, one is selected as the final target offline task combination. The specific choice depends on the system's current needs and priority settings. This method not only improves the scientific rigor and accuracy of resource release decisions but also minimizes the impact on system performance, ensuring that critical business operations are not disrupted.

[0345] The above implementation proposes a task selection mechanism based on multi-dimensional matching conditions, aiming to select the optimal combination of target offline tasks from all deployed tasks. By setting three different matching conditions (achieving the desired reduction in resources, minimizing the degree of sacrifice, and minimizing the number of tasks), different resource management needs can be flexibly addressed. This method not only improves the scientific nature and accuracy of resource release decisions but also effectively balances the relationship between resource utilization and task execution status, ensuring that the system can still operate efficiently under resource constraints. It also ensures that while reducing resource consumption, the impact on system performance is minimized, and the continuity and stability of critical business operations are guaranteed.

[0346] Based on the above, in order to further optimize the task selection process, this application proposes the following ideas:

[0347] Optionally, when applying matching condition two or matching condition three for filtering, the target task combination must also satisfy matching condition one. This means that regardless of which matching condition is used, the final selected task combination can effectively reduce resource consumption to achieve the expected resource release goal.

[0348] Specifically, when applying matching condition two or matching condition three, first ensure that the candidate task combinations meet the requirements of matching condition one. For example, if matching condition two (minimum sacrifice) is used, only those task combinations that can achieve the expected reduction in resource usage are considered, and target task combinations that meet matching condition two are selected from these task combinations; similarly, if matching condition three (minimum total number of tasks) is used, only those task combinations that can achieve the expected reduction in resource usage are considered, and target task combinations that meet matching condition three are selected from these task combinations; and so on.

[0349] This method emphasizes that when applying matching condition two and matching condition three for filtering, the combination of target tasks must simultaneously satisfy matching condition one. This multi-level filtering mechanism, by utilizing matching conditions, ensures that resource release can effectively alleviate resource shortages, and by utilizing matching conditions two and three, minimizes the impact on cluster performance as much as possible, thereby improving the system's flexibility and adaptability.

[0350] The following is combined Figure 9 Here's a simple example:

[0351] like Figure 9The diagram illustrates a target offline task selection logic in an embodiment of this application. Assume that, in addition to online tasks, a certain working node has three deployed offline tasks, denoted as Offline Task 1, Offline Task 2, and Offline Task 3. The CPU usage of these three offline tasks is 2.5 cores, 1.5 cores, and 1.5 cores respectively. If a resource management anomaly is detected on this working node, specifically manifested as a sustained CPU usage of 5 cores, leading to prolonged online task response time, a target offline task must be selected for resource release to reduce CPU usage by at least 2 cores (i.e., the expected resource release target).

[0352] Therefore, when selecting a task, you can perform the following operations:

[0353] First, based on matching condition one, we need to filter out task combinations that can reduce CPU usage by at least two cores. For example, there are five task combinations A, B, C, D, and E. Task combination A contains offline task 1, task combination B contains offline tasks 2 and 3, task combination C contains offline tasks 1 and 2, task combination D contains offline tasks 1 and 3, and task combination E contains offline tasks 1, 2, and 3. Then, among these task combinations that satisfy matching condition one, we apply matching conditions two and three to select the task combination with the least sacrifice and the fewest number of tasks.

[0354] For example:

[0355] Task Combination A: Low sacrifice, 1 offline task in total. Task Combination B: Medium sacrifice, 2 offline tasks in total. Task Combination C: Medium sacrifice, 2 offline tasks in total. Task Combination D: Medium sacrifice, 2 offline tasks in total. Task Combination E: High sacrifice, 3 offline tasks in total.

[0356] In this scenario, task combination A is ultimately chosen as the target task combination because it has the lowest level of sacrifice and the fewest number of tasks, and resources are subsequently released for offline task 1.

[0357] The above implementation further strengthens the rigor of task selection. When applying matching condition two or matching condition three, the requirement of matching condition one must also be met. This constraint ensures that regardless of the matching condition used, the final selected task combination can effectively reduce resource consumption and achieve the expected resource release goal. This method not only improves the accuracy of resource management but also reduces the risk of system performance fluctuations caused by improper resource release strategies, enhancing system stability and reliability. In this way, a solid guarantee is provided for achieving efficient resource management and scheduling.

[0358] Optionally, if multiple task combinations simultaneously meet all matching conditions, the final selection can be made based on other additional criteria (such as task importance and priority). This approach ensures that resource release not only effectively alleviates resource constraints but also minimizes the impact on overall system performance.

[0359] Suppose that after initial screening, multiple task combinations are found to simultaneously satisfy matching condition one, matching condition two, and matching condition three. In this case, other additional criteria can be introduced to make the final selection.

[0360] For example, based on matching conditions one, two, and three, it has been determined that two task combinations, F and G, both meet the requirements: Combination F: Reduces CPU usage by 2.5 cores with a relatively small sacrifice, mainly involving some batch processing jobs (a total of 2 offline tasks). Combination G: Also reduces CPU usage by 2.5 cores with a relatively small sacrifice, involving some low-priority log analysis tasks (a total of 2 offline tasks).

[0361] Among these two task combinations that meet all matching criteria, we can consider using task importance and priority as additional criteria for further filtering, as follows:

[0362] Although the batch processing jobs in task combination F have a small degree of sacrifice, since they are non-critical business operations, even if they are delayed, they will not have a significant impact on the overall business.

[0363] Although the log analysis tasks in task group G are also low priority, they are very important for subsequent troubleshooting and system optimization, and should not be easily interrupted.

[0364] In this scenario, although the two combinations exhibit similar performance in terms of resource release and sacrifice, task combination F has a lower importance and priority, resulting in a smaller impact on cluster performance and no impact on the functionality and service quality of critical businesses. Therefore, task combination F can ultimately be selected as the target task combination, and resources can be released for the target offline tasks within that target task combination.

[0365] This additional standard-assisted decision-making approach provides extra selection criteria when multiple task combinations meet the basic matching conditions. This allows resource release to not only alleviate resource shortages but also take into account the importance and priority of tasks, thereby improving the system's flexibility and adaptability.

[0366] Of course, in practical applications, in addition to the three matching conditions mentioned above, other matching conditions or selection methods can be introduced, such as priority, task execution status, historical performance, dependency, task type, time window, resource bottleneck, cost-effectiveness, etc., to further optimize the screening process of target offline tasks.

[0367] In addition, other methods can be used to select target offline tasks. For example, machine learning-based predictive models can be used for screening. Specifically, machine learning models predict the resource requirements and performance of different offline tasks over a future period, thereby selecting the most suitable offline task for resource release. Another example is using a dynamic feedback mechanism for screening. Specifically, a dynamic feedback mechanism is implemented to monitor the performance changes of offline tasks in real time and dynamically adjust resource allocation strategies based on the feedback results. For instance, if the performance of an offline task drops significantly after resource reduction, its resource usage can be restored promptly.

[0368] In another alternative implementation, if administrators are allowed to set custom filtering rules for offline tasks, they can flexibly define which offline tasks should be prioritized for resource release. This approach offers greater flexibility and can be adjusted according to actual needs.

[0369] In summary, by introducing more matching conditions and selection methods in this embodiment, the accuracy and flexibility of resource management can be further improved, ensuring that the system can still operate efficiently and guarantee the continuity of critical business even when resources are limited.

[0370] In this embodiment of the application, after selecting one or more target offline tasks that need to release resources through any of the above methods, the following S4133 can be executed:

[0371] S4133: Release the corresponding resources occupied by at least one target offline task according to the release strategy corresponding to each target dimension.

[0372] Specifically, for the selected target offline task, the corresponding resources occupied by it are released according to the release strategy corresponding to each target dimension.

[0373] In resource management, a release strategy refers to the specific methods and rules for reclaiming or releasing resources that are no longer needed or can be temporarily borrowed from certain tasks when resource management anomalies occur in specific dimensions (such as CPU, memory, disk I / O bandwidth). A reasonable release strategy can effectively alleviate resource shortages, ensure the service quality of online tasks, and optimize the resource utilization of the entire cluster.

[0374] This application, taking into account the differences in characteristics of different types of resources, proposes a method for setting release strategies based on resource elasticity to maximize resource utilization and ensure service quality. Specifically, elastic resources (such as CPU, disk I / O bandwidth, etc.) can be compressed without affecting the normal operation of tasks, usually by reducing usage or priority. These resources have high flexibility and can adjust their allocation without interrupting tasks. In contrast, non-elastic resources (such as GPUs) cannot be compressed, so tasks need to be evicted or migrated to release resources; otherwise, tasks cannot continue. Once these resources are occupied, it is difficult to reduce their usage without affecting task execution. Therefore, if it is necessary to release resources occupied by offline tasks (referred to as offline resources), it is necessary to first determine whether the resources to be compressed are elastic resources, non-elastic resources, or a combination of both.

[0375] Based on the above ideas, to further refine the release strategy, this application proposes a method for formulating different release strategies according to the resource type (such as elastic or inelastic resources) of the target dimension. This classification method is based on the task's sensitivity to resource changes and aims to minimize the impact of resource reduction on system performance and stability. Specific implementation details are as follows:

[0376] If the resources in the target dimension are elastic resources, then the release strategy corresponding to the target dimension (denoted as release strategy one) is: reduce the resource usage of the task in the target dimension.

[0377] If the resources in the target dimension are inelastic resources, then the release strategy corresponding to the target dimension (denoted as release strategy two) is: release all resources occupied by the task.

[0378] In the embodiments of this application, different types of resources have different degrees of impact on the task. For example, resources such as CPU and memory usually have a certain degree of elasticity and can continue to run even when reduced, although performance may be degraded; while resources such as GPU may cause a significant drop in performance or even task failure once reduced.

[0379] Specifically, elastic resources refer to the resources required by tasks that have a certain tolerance for resource changes. These tasks can continue to run even when resources are reduced, although performance may be degraded, but they will not fail immediately.

[0380] For elastic resources, this application releases them by reducing the resource usage of tasks at the target level. For example, the CPU usage of a task is reduced from 8 cores to 6 cores.

[0381] In practical applications, to avoid a significant drop in task performance due to a one-time reduction of too many resources, adjustments can be made gradually, and the system response after each adjustment can be observed.

[0382] For example, on a worker node, some batch processing jobs have a certain tolerance for changes in CPU resources. When resource strain is detected on that node, a gradual usage reduction strategy can be adopted to reduce the CPU usage of these tasks from the current 8 cores to 6 cores.

[0383] In one alternative implementation, the gradual reduction strategy includes, but is not limited to, the following:

[0384] (1) Phased adjustment: Reduce resource usage gradually to alleviate system pressure and avoid excessive impact on task performance.

[0385] For example, each adjustment should not exceed 20% of the current usage, such as reducing from 8 cores to 6.4 cores, then to 5.1 cores, and so on, with at least 5 minutes between each stage to ensure the system has enough time to adapt to the changes.

[0386] (2) Dynamic step size algorithm: Based on historical performance data, the step size is automatically calculated and adjusted to optimize the resource release process.

[0387] For example, the system can automatically calculate the safe adjustment step size based on historical performance data. Each adjustment step is 2 cores, so the system can be reduced from 8 cores to 6 cores, then to 4 cores, and so on, with an interval of at least 5 minutes between each stage to ensure that the system has enough time to adapt to changes.

[0388] This method uses intelligent analysis of the historical performance of tasks to dynamically adjust the amount of resource reduction, which can effectively alleviate resource shortages and maintain task performance to the maximum extent. It is particularly suitable for complex and variable workload environments.

[0389] Optionally, when reducing resource usage, you can also set resource limits (such as no less than 0.5 CPU cores and no less than 512MB of memory) to prevent excessive compression from causing the task to crash.

[0390] Optionally, after implementing the resource release strategy, the performance of the task can be continuously monitored, and subsequent strategies can be adjusted based on the actual effects to ensure that the normal operation of the task is not affected during the resource release process. For example, for elastic resources, the resource usage can be gradually adjusted and its impact observed. For instance, if continuous monitoring of task performance indicators (such as response time, throughput, etc.) reveals that reducing CPU usage has a significant impact on task performance, some resource usage can be appropriately restored to ensure that the task can run normally under the new resource conditions.

[0391] Non-elastic resources refer to the resources required by tasks that are highly sensitive to resource changes. For these tasks, resource reduction can immediately lead to a significant performance drop or even task failure, such as GPUs.

[0392] For inelastic resources, this application adopts a method of completely releasing all resources occupied by the task, which means that it may be necessary to stop or migrate the task to release resources.

[0393] Specifically, if resource scarcity is detected, the first step is to migrate the task to a node with sufficient resources to minimize the impact on the task. Before migration, the resource status of the target worker node must be assessed, and it must be ensured that the task data has been saved or backed up. If migration is not feasible, task eviction measures are taken, i.e., deleting the container running the target offline task to release resources. Before eviction, it is also necessary to ensure the secure preservation of data and notify relevant parties of the task's eviction. This multi-layered resource release method can effectively alleviate resource scarcity problems while maximizing the protection of task continuity and data integrity. The specific implementation is as follows:

[0394] For inelastic resources, if possible, prioritize migrating the target offline task to a node with sufficient resources, rather than directly stopping the target offline task. This minimizes the impact on the task and ensures that the task can continue running uninterrupted on the new node. Before performing the migration, it is necessary to first assess the feasibility of the migration and check the resource status of the target worker node to ensure that there are sufficient resources to support the migrated task. Once it is confirmed that the target worker node has sufficient available resources, container or virtual machine technology can be used to migrate the task from a resource-constrained node to a resource-sufficient node. Throughout the migration process, downtime should be minimized to avoid unnecessary impact on task performance.

[0395] If migration is not feasible, task eviction must be implemented, which involves releasing resources by deleting the container running the target offline task. Before eviction, ensure all necessary data is saved or backed up to guarantee data security. This "safe eviction" method effectively prevents data loss due to resource release. Simultaneously, establish a notification mechanism to send notifications to relevant users or systems, informing them of the reason for task eviction and the estimated recovery time, helping users understand the current situation and make appropriate adjustments. This approach not only alleviates resource constraints promptly but also minimizes the impact on business processes.

[0396] Optionally, ensuring that task data has been saved or backed up before releasing resources through migration or task eviction is a crucial step. This helps prevent data loss and ensures rapid task recovery. To this end, tasks can be configured to automatically save their current state and data before migration or termination, ensuring critical information is properly handled. Furthermore, a regular backup mechanism can be established for rapid recovery even in the event of unforeseen circumstances. In this way, the system can maximize the protection of task data integrity and continuity during resource release operations, ensuring uninterrupted business operations.

[0397] For example, in a deep learning training environment, some model training tasks rely heavily on GPU resources. When GPU resource scarcity is detected, the system first attempts to migrate these tasks to other nodes with sufficient resources. If migration is not possible, these tasks are stopped, and their state is saved for later resumption.

[0398] Optionally, after implementing the release strategy, the performance of the task can be continuously monitored to ensure that the normal operation of the task is not affected during the resource release process. For example, for non-elastic resources, it is necessary to ensure that the safe eviction of the task does not lead to data loss or other problems; and so on.

[0399] Furthermore, it should be noted that memory, as a special resource, can be considered either elastic or inelastic under different circumstances:

[0400] When used as a non-elastic resource: directly reclaiming memory resources allocated to offline tasks and currently in use may cause the application to crash or experience performance degradation. Therefore, in this case, one should not easily attempt to reclaim this memory; instead, it is necessary to release all resources occupied by the task through methods such as eviction or task migration.

[0401] When used as a flexible resource:

[0402] Unused memory reclamation: It's possible to safely reclaim memory resources allocated to offline tasks but not yet used, without affecting task execution. For example, if a mixed-deployment cluster supports memory compression, compression can be used to free up some memory without completely evicting tasks. This approach allows the system to optimize memory usage efficiency while keeping tasks running.

[0403] In summary, memory resource management should flexibly assess its elasticity and inelasticity based on actual conditions and adopt corresponding release strategies. For example, during periods of high load, priority should be given to reclaiming unused memory and applying memory compression techniques; in extreme cases, if more memory must be released, the possibility of evictring low-priority tasks can be evaluated. Of course, other resources with similar characteristics should be handled in the same way, and will not be elaborated upon here.

[0404] In the above implementation, by differentiating between elastic and inelastic resources and formulating different release strategies, the flexibility and effectiveness of resource management are significantly improved. For elastic resources, reducing the resource usage of tasks can release more available resources without affecting the overall performance of the tasks; while for inelastic resources, directly releasing the actually occupied resources can quickly alleviate resource shortages and ensure the service quality of online tasks. This differentiated approach allows the system to make more precise adjustments under resource constraints, thereby ensuring the stability and efficient operation of the system.

[0405] Optionally, to further refine the specific operations of resource release and ensure the effective implementation of resource management strategies, based on the above two release strategies, an optional implementation method is to implement S4133 according to the following process, specifically divided into the following two cases:

[0406] Scenario 1: For each target offline task, if the resources occupied by the target offline task in each target dimension, including non-elastic resources, are released by scheduling the target offline task.

[0407] Here, scheduling target offline tasks refers to the process of dynamically allocating and reallocating tasks or resources, including but not limited to: migrating target offline tasks to other working nodes (such as idle nodes), or directly evictting tasks to release resources.

[0408] For example, if an offline task simultaneously consumes a large amount of GPU (non-elastic resource) and a small amount of disk I / O bandwidth (elastic resource), it is more efficient to evict the task to free up all resources. This approach can quickly alleviate resource constraints and ensure stable system operation.

[0409] Scenario 2: For each target offline task, if the resources occupied by the target offline task in each target dimension do not include non-elastic resources, then the resources occupied by the target offline task will be released by reducing the resource usage of the target offline task in each target dimension.

[0410] Here, reducing resource usage means compressing the amount of resources allocated to the task in order to optimize resource usage without completely stopping the task.

[0411] For example, if an offline task only consumes a large amount of CPU (elastic resource), some resources can be freed up by reducing its CPU usage, without having to completely stop the task. This method allows for gradual adjustment of resource allocation without affecting the normal operation of tasks, ensuring the efficiency and stability of the system.

[0412] In summary, the two scenarios described above specify how to adopt corresponding release strategies based on the type of resources (elastic or inelastic) occupied by the target offline task, such as releasing inelastic resources by evicting or migrating the task, or releasing elastic resources by reducing usage (i.e., compressing resources).

[0413] Optionally, when implementing S4133, a resource assessment must first be performed on each target offline task to determine whether its resource usage includes non-elastic resources. This step can be achieved by using resource monitoring tools to obtain the current resource usage of the tasks, ensuring a comprehensive understanding of the resource requirements and characteristics of each task.

[0414] For tasks containing inelastic resources, if it is found that the resources occupied by the target offline task include inelastic resources, consider releasing all resources by migrating or evictring the task (see the above embodiments for details, which will not be repeated here). The evictment operation may involve pausing or terminating the task, depending on the actual situation and system configuration.

[0415] For tasks that only contain elastic resources, if all resources used by the target offline task are elastic resources, then resource compression is used to release some resources. This method can optimize resource allocation without affecting the normal operation of the task.

[0416] Taking the simultaneous fulfillment of matching conditions one, two, and three as an example, simply put, when compressing offline resources, the first step is to determine whether the resource to be compressed is an elastic resource, a non-elastic resource, or a combination of both. Then, based on the determination results, an algorithm is used to find the resource compression or eviction scheme that minimizes the degree of resource sacrifice and involves the fewest number of tasks while satisfying the resource compression requirements across all dimensions, thereby minimizing the impact of resource reduction on system performance and stability.

[0417] When searching for suitable resource compression or eviction schemes using algorithms, greedy algorithms can be employed, prioritizing tasks that meet the aforementioned matching conditions for resource adjustment to gradually achieve the goal. Alternatively, heuristic algorithms, such as genetic algorithms and simulated annealing, can be used to explore the possible solution space and find the optimal target offline task. Furthermore, linear programming methods can be used to construct mathematical models, optimize resource allocation, and minimize the impact on tasks. These algorithms can intelligently select the best strategy based on resource type and task requirements, ensuring system stability and efficiency.

[0418] Based on the above definitions and characteristics, the following examples further illustrate how to select a target offline task and release resources based on the elasticity of resources.

[0419] Scenario 1: Suppose a resource management anomaly is detected on a worker node, causing prolonged online task response time. The resource to be compressed is one CPU core; in this case, the target dimension is CPU. The offline tasks deployed on this worker node and their resource usage are as follows:

[0420] Offline Task 1: Using 0.7 core CPUs, 2 instances (total 1.4 core CPUs).

[0421] Offline Task 2: Using a 0.6-core CPU and 2 GPU cards, there are 2 copies in total (a total of 1.2-core CPU and 4 GPU cards).

[0422] In this scenario, compressing the CPU resources of offline task 1 is prioritized because CPU is an elastic resource; the demand can be met by reducing CPU usage without evicting the task. This minimizes the sacrifice of resources required for compression, reduces the number of tasks, and satisfies the three matching conditions mentioned above.

[0423] Specifically, the target offline task is offline task 1, and the resources it occupies in the target dimension, namely CPU resources, are elastic resources. Therefore, by adjusting the CPU usage of the two copies of offline task 1 to compress resources, the required 1 CPU core resource can be released without affecting other resources such as the GPU. This satisfies the resource compression requirements while minimizing interference with the system.

[0424] The above scenario 1 uses a single objective dimension as an example. Below, we will briefly explain multiple objective dimensions in conjunction with scenario 2:

[0425] Scenario 2: Suppose a resource management anomaly is detected on a worker node, causing prolonged online task response time. The resource requirements for compression are 1 CPU core and 2 GPU cards. In this case, the target dimensions are CPU and GPU. The offline tasks deployed on this worker node and their resource usage are as follows:

[0426] Offline Task 1: Using 0.5 CPU cores, 2 instances (total 1 CPU core).

[0427] Offline Task 2: Using 1 CPU core and 2 GPU cards (total of 1 CPU core and 2 GPU cards).

[0428] Offline Task 3: Use 0-core CPU and 1 GPU card, for a total of 2 copies (2 GPU cards in total).

[0429] In this scenario, evictting offline task 2 is the preferred option because it contains inelastic resources (GPUs). Evicting one offline task 2 satisfies all compression requirements with minimal impact and the fewest number of tasks required. Even though offline task 2 contains inelastic resources, evicting one offline task 2 satisfies all dimensions of compression resources with minimal impact.

[0430] If offline task 1 and offline task 3 are selected, the CPU resources of offline task 1 need to be compressed, and both offline tasks 3 need to be evicted. Although offline task 1 only compresses resources rather than evicting tasks, evicting both offline tasks 3 still results in significant damage. Therefore, considering matching conditions one, two, and three, evicting offline task 2 is the optimal choice.

[0431] Specifically, the target offline task is offline task 2, which consumes CPU and GPU resources in the target dimension, including non-elastic resources. Therefore, by evicting offline task 2, one CPU core and two GPU cards can be directly freed up. This satisfies the resource compression requirements while minimizing interference with the system.

[0432] Optionally, the above process can be implemented by the resource detection controller within the hybrid component on the worker node. For example... Figure 10 As shown, it is a logical diagram of resource release in an embodiment of this application. Wherein, Figure 10 express:

[0433] The resource detection controller receives conflict resource information from the offline resource manager and schedulable resource information from the interference detection controller. After analysis, it finds that there are resource conflicts / over-issuance situations and confirms that it is necessary to reduce the resources occupied by offline tasks to protect the quality of online tasks. Therefore, the resource detection controller selects a suitable target offline task from the current working nodes for resource release based on the algorithm.

[0434] When releasing resources, referring to the above ideas, for elastic resources, release strategy one can be adopted to reduce the resource usage of the task in the target dimension; while for non-elastic resources, all resources occupied by the task can be released.

[0435] It should be noted that the specific implementation method of this process can be found in the above embodiments, and repeated parts will not be described again.

[0436] After releasing resources, the resource detection controller needs to update the schedulable resource information of the current working node and report it to the upper-level scheduler. For example, the report can be made through the mixed deployment control layer. For specific implementation methods, please refer to the following text, which will not be repeated here.

[0437] In summary, the hybrid deployment component on the working node of this application adopts a novel multi-dimensional hybrid deployment technology, which is responsible for multi-dimensional offline resource mining and reporting, multi-dimensional offline resource conflict and interference detection, and multi-dimensional offline resource selection and compression, to ensure the limited scheduling resource view of offline training GPU jobs and the quality of service guarantee of online GPU jobs.

[0438] Optionally, a notification mechanism can be established on the basis of the above to send notifications to relevant users or systems, informing them of the reasons for the task being migrated, removed, or resources being compressed, as well as the expected recovery time, so as to help users understand the situation in a timely manner and make corresponding adjustments.

[0439] In the above implementation, specific implementation steps further refine how to execute the release strategy based on the resource type of the target dimension. When a target offline task occupies non-elastic resources, all resources are released by evicting the task. This method can quickly alleviate resource bottlenecks and avoid negative impacts on online tasks. For tasks that only occupy elastic resources, some resources are released by reducing resource usage, which can optimize resource utilization while ensuring the normal operation of the task. This method not only improves the operability and controllability of resource release operations, but also reduces the risk of system performance fluctuations caused by resource adjustments, enhancing the stability and reliability of the system. Through this multi-layered and refined resource management mechanism, the resource utilization efficiency and response speed of the entire cluster can be effectively improved.

[0440] It should be noted that the release strategies for elastic and inelastic resources listed above are just simple examples. In addition, other target offline tasks and their release strategies are also applicable to the embodiments of this application. Several examples are listed below:

[0441] (1) Proportional release refers to reducing the resource usage of the target offline task by a certain percentage based on the degree of resource scarcity. Specific implementation details are as follows:

[0442] First, calculate the required reduction percentage based on current resource usage and the set security threshold. For example, if CPU usage reaches 95% and the security threshold is 80%, it might be decided to reduce the CPU usage of certain deployed offline tasks (i.e., the target offline task) on a worker node by 20%. To avoid reducing too many resources at once, which could cause offline task failures or significant performance degradation, adjustments can be made in stages, and the response of the worker node can be observed after each adjustment. This approach balances resource release with offline task execution, minimizing the impact on offline task performance.

[0443] For example, when a worker node's CPU usage is detected to be close to 95%, a proportional release strategy is initiated, identifying several non-critical offline data analysis tasks as target offline tasks, and reducing the CPU usage of these tasks by 20%, that is, reducing the number of cores from the CPU usage of each task by 20%.

[0444] (2) Gradual release refers to reducing resource usage in stages, observing the response of work nodes, and avoiding releasing too many resources at once, which could lead to offline task failures or a significant performance drop. Specific implementation details are as follows:

[0445] First, assess the current resource usage and resilience of the offline task. Then, reduce resources by a small amount each time (e.g., 5%–10%), monitoring the worker node's response after each adjustment. Based on the monitoring data, decide whether to continue reducing resources or revert to the previous configuration. This approach reduces the risk of offline task failure due to sudden resource reduction, improving the stability of the worker nodes.

[0446] For example, in a machine learning training environment, when it is detected that the memory usage of a certain worker node is too high, the memory usage of each offline training task on that worker node is gradually reduced by 5% each time, and its performance changes are continuously monitored to ensure that resource pressure is effectively alleviated without affecting the model training effect.

[0447] (3) Priority-based tiered release strategy: This strategy releases resources based on the priority of offline tasks, prioritizing the release of resources from lower-priority offline tasks. Specific implementation details are as follows:

[0448] By categorizing deployed offline tasks on worker nodes by priority, low-priority offline tasks are prioritized for resource release, ensuring that critical business operations are not affected. Offline tasks are released in a tiered manner based on their priority, with priority given to releasing resources from low-priority tasks. First, deployed offline tasks are categorized by priority, and a priority threshold (e.g., less than or equal to 3) is set. Offline tasks below this threshold are selected as target offline tasks, and their performance impact is further evaluated, prioritizing the release of resources from those tasks with minimal performance impact. Resources for these low-priority offline tasks are gradually reduced or completely released to ensure that critical business operations are not affected.

[0449] For example, if CPU resource shortage is detected on a working node, several log processing offline tasks with a priority of 2 are selected based on priority. Their CPU usage is reduced from 2 cores per offline task to 1 core, freeing up 2 CPU cores for high-priority online tasks and ensuring the resource needs of critical business operations are met.

[0450] (4) Predictive release strategy, which refers to using historical data and machine learning models to predict future resource demand and release resources in advance. Specific implementation details are as follows:

[0451] Historical resource usage information (such as CPU and memory) is collected and preprocessed, and a predictive model is trained using time series analysis or regression analysis. Based on the prediction results, periods of potential resource scarcity are identified in a specific future timeframe, and offline tasks expected to consume significant resources are selected as target offline tasks. Subsequently, some or all resources for these target offline tasks are released in advance to prevent resource congestion.

[0452] For example, by analyzing historical data from the past month and using machine learning models, it was predicted that CPU resources would be strained for a certain work node within the next hour. Therefore, the CPU usage of three offline data analysis tasks that were expected to consume a large amount of CPU resources was reduced in advance, from 4 cores per task to 2 cores, ensuring that there were sufficient resources to support online processing during peak periods and avoiding resource shortages.

[0453] The different resource release strategies listed above are suitable for different scenarios and needs. In practical applications, these strategies can be flexibly adjusted, rationally selected, and combined according to different application scenarios and requirements to improve the system's flexibility and adaptability. This ensures that resource management anomalies are effectively resolved under limited resource conditions, guarantees the service quality of online tasks, and optimizes the resource utilization efficiency of the entire cluster. Furthermore, resource release strategies can be adaptively adjusted based on real-time monitoring data to ensure system stability and efficiency.

[0454] Of course, the scenarios listed above all apply to a single worker node. In addition, this application can also achieve load balancing while releasing some resources by dynamically adjusting the distribution of offline tasks across different worker nodes. This strategy is suitable when some worker nodes in the cluster are resource-constrained while others have surplus resources. Specific implementation details are as follows:

[0455] By dynamically adjusting the distribution of offline tasks across different worker nodes, load balancing is achieved while releasing some resources. The resource usage (such as CPU and memory) of each worker node is periodically assessed to identify those with scarce and abundant resources. Offline tasks with higher tolerance for resource changes are selected from those on resource-scarce nodes and migrated to resource-abundant worker nodes to release resources on the scarce nodes and improve the overall cluster resource utilization.

[0456] For example, in a cluster with two nodes A and B, node A has 9 CPU cores in use, while node B has only 4. The system migrates two more resilient offline data analysis tasks from node A to node B, reducing node A's CPU usage to 7 cores and increasing node B's usage to 6 cores, thus achieving load balancing and alleviating the pressure on node A.

[0457] It should be noted that the release strategies listed above are just simple examples, and other release strategies are also applicable to the embodiments of this application, which will not be described in detail here.

[0458] In the above implementation, by analyzing whether there are any resource management anomalies based on schedulable resource information and conflicting resource information, and selectively releasing resources for target offline tasks that match the resource scarcity dimension, resource overload and contention issues can be effectively alleviated, ensuring the service quality of online tasks. This method not only improves system stability and response speed but also optimizes the resource utilization of the entire cluster, ensuring that critical business operations are not affected under limited resource conditions, while maximizing the use of available resources and improving overall efficiency and performance. This dynamic adjustment mechanism enables the system to react quickly to changes, enhancing its flexibility and adaptability.

[0459] S42: Schedule offline tasks to be scheduled based on the latest schedulable resource information of each working node.

[0460] In this step, the scheduler can allocate offline tasks to be scheduled based on the latest schedulable resource information of each worker node. During allocation, the scheduler can intelligently match and allocate tasks to the most suitable nodes by combining the resource requirements (such as CPU and memory), priority, and node load of the offline tasks, so as to ensure efficient resource utilization, avoid overload, and maintain system stability and performance.

[0461] To better achieve stable operation and reasonable resource allocation for online tasks, while improving the scheduling efficiency of offline tasks, this application proposes a three-layer hybrid architecture, including a bottom layer (composed of multiple hybrid clusters), a middle layer (also known as a hybrid management layer), and an upper layer (composed of at least one hybrid cluster).

[0462] For the above-mentioned hybrid architecture, one optional implementation is to perform step S42 according to the following process, including the following S421 to S423 ( Figure 4 (Not shown):

[0463] S421: For each working node, the corresponding schedulable resource information is reported to the mixed-department control layer.

[0464] In step S421, for each worker node, the current schedulable resource information is collected periodically or through an event trigger.

[0465] In one optional implementation, worker nodes can periodically collect resource information at fixed time intervals and report it periodically to the hybrid deployment control layer. This allows the hybrid deployment control layer to aggregate and send the information to the upper-level scheduler, enabling the scheduler to continuously monitor the global status of cluster resources. Specifically, the periodic collection interval can be short (e.g., 1-5 seconds), reporting once per second to ensure the system responds quickly to changes in online task load. The periodic collection interval can also be medium (e.g., 10-30 seconds), reporting every 10 or 30 seconds to reduce communication overhead while maintaining system stability. Alternatively, the periodic collection interval can be long (e.g., 1-5 minutes), reporting every 1 or 5 minutes to reduce additional overhead while ensuring that the scheduling nodes obtain the latest schedulable resource information.

[0466] In another alternative implementation, the worker nodes can also collect resource information at variable time intervals, dynamically adjusting the reporting interval based on the task load characteristics at different times. For example, during peak daytime hours, the task load is heavy and resource usage fluctuates significantly, so the reporting interval is shorter (e.g., 5-10 seconds) to ensure the scheduler can quickly detect changes in resource status and adjust task scheduling strategies. Conversely, during off-peak nighttime hours, the task load is lighter and resource status is more stable, so the reporting interval is longer (e.g., 1-5 minutes) to reduce unnecessary communication overhead and optimize resource utilization efficiency. Specifically, worker nodes can combine historical resource usage data and adopt a time-adaptive reporting strategy. For example, during the day (08:00-20:00), due to active online tasks and large fluctuations in resource demand, short intervals (5-30 seconds) are used for reporting. During peak periods (10:00-14:00, 17:00-19:00), when GPU load is high, the interval can be shortened to 5-10 seconds to more accurately adjust task scheduling. At night (20:00-08:00), offline tasks dominate and resource status changes are small, so long intervals (1-5 minutes) are used for reporting. For example, during 00:00-06:00, when task load is low, the interval can be adjusted to 3-5 minutes to reduce unnecessary requests and improve overall system efficiency.

[0467] It should be noted that the time intervals mentioned above are just simple examples, and the specific reporting intervals can be flexibly configured according to actual business needs and system load. For example, in some computationally intensive task scenarios, a shorter reporting cycle (such as 13 seconds) can be used to ensure the real-time nature of resource monitoring, while for batch tasks that run stably for a long time, a longer reporting cycle (such as 10 to 30 minutes) can be used to reduce communication overhead.

[0468] Optionally, by combining dynamic adjustment strategies, the reporting interval can be adaptively adjusted based on the historical operating data and prediction models of the working nodes, so that the collection and reporting of resource information can be more intelligently adapted to different load conditions, improving the accuracy of scheduling while reducing communication overhead.

[0469] Specifically, the dynamic adjustment strategy can analyze the resource usage patterns of worker nodes in different time periods based on historical operational data, and combine this with machine learning prediction models to predict future changes in resource demand, thereby adaptively adjusting the reporting frequency of resource information. For example, in historical data, if a GPU node has high GPU and CPU usage between 10:00-12:00 and 18:00-20:00 daily, this worker node can automatically shorten the reporting interval (e.g., 5-10 seconds) during these periods to ensure that the scheduler can promptly detect resource changes. Conversely, during the period from 1:00 to 6:00 AM, when task load is low, the system can automatically extend the reporting interval (e.g., 3-5 minutes) to reduce unnecessary data transmission and optimize system communication overhead.

[0470] Furthermore, by combining real-time monitoring and predictive models, adaptive analysis of resource trends can be performed. When a worker node's GPU usage is detected to be continuously increasing, and historical data predicts that it may reach a high load state within the next 5 minutes, the worker node can proactively shorten the reporting interval to adjust task scheduling earlier and prevent resource overload. Similarly, if it is predicted that GPU usage will decrease in the future, meaning a reduction in task load, the reporting interval can be increased accordingly, reducing data transmission frequency.

[0471] In the embodiments of this application, the prediction model can be a time series prediction model, such as ARIMA or LSTM, which can be used to analyze historical resource usage data and predict future resource usage trends. For example, historical data on GPU usage shows that the load is high from 10:00 to 12:00 and from 18:00 to 20:00 every day. ARIMA can predict the increase in resource demand during these periods and shorten the reporting interval in advance. LSTM can capture the resource usage patterns of the training task at different stages (initial stage, stable stage, and final stage) to optimize the reporting interval. For example, the reporting frequency is high in the early stage of training (e.g., 10 seconds), and the reporting interval is automatically extended (e.g., 2 to 5 minutes) after the training is stable.

[0472] Predictive models can also be regression analysis (such as multiple linear regression and decision tree regression). Regression analysis can be used to establish the relationship between resource usage and system variables. Decision tree regression is suitable for scenarios where resource usage changes significantly in stages, such as when the task load varies greatly in different time periods. Decision trees can help determine the optimal reporting interval for different time periods. For example, under high load (GPU usage is higher than 80% of total GPU resources and CPU usage is higher than 70% of total CPU resources), the reporting interval is automatically shortened, while under low load (GPU usage is lower than 50% of total GPU resources), the reporting interval is appropriately extended.

[0473] Predictive models can also be deep learning models based on transformers and reinforcement learning. Deep learning models can be used for complex resource scheduling optimization. Transformers can analyze the relationship between different resource metrics through self-attention mechanisms, such as the interaction between GPU usage and I / O throughput, to predict resource conflicts in advance and optimize the reporting interval. Reinforcement learning can intelligently adjust the reporting strategy, such as extending the reporting interval when GPU usage is low and stable, shortening the reporting interval when GPU usage fluctuates drastically, and triggering reporting immediately when online tasks suddenly increase.

[0474] Predictive models can also be statistical analysis methods (such as sliding window averaging and dynamic threshold adjustment). Statistical analysis can be used for lightweight dynamic reporting optimization. For example, sliding window averaging calculates the average GPU / CPU usage over the past 10 minutes. If the change exceeds a set threshold, the reporting interval is shortened; otherwise, it is extended appropriately. Dynamic threshold adjustment can be dynamically adjusted based on the current system load and historical data. For example, the reporting threshold can be set to 75% during the day and automatically adjusted to 85% at night due to reduced tasks.

[0475] By combining the monitoring mechanism and the predictive model described above, resource demand can be predicted more accurately, ensuring rapid response to scheduling needs under high load conditions while reducing communication costs under low load conditions, thereby achieving an optimal balance between scheduling efficiency and system overhead. It should also be noted that the above predictive model is just a simple example; specific prediction methods, algorithm selection, and parameter adjustments can be flexibly configured according to actual business needs, computing cluster size, task type, and resource usage patterns.

[0476] In another alternative implementation, worker nodes can also employ an event-triggered mechanism for resource information collection. This means that resource information collection and reporting are triggered when specific system events or significant changes in resource status occur, rather than relying on fixed time intervals. For example, when key resource indicators such as GPU usage, CPU load, memory consumption, and I / O bandwidth experience sudden changes or exceed set thresholds, worker nodes can immediately trigger resource information collection and report the updated schedulable resource information to the hybrid deployment control layer. This allows the system to quickly detect changes in resource status and adjust task scheduling strategies accordingly.

[0477] It should be noted that the above triggering conditions can be flexibly configured according to actual business needs, system load and cluster scheduling strategies to better adapt to different computing task scenarios. For example, in high-load, high-concurrency inference task environments, the threshold can be more sensitive, while in offline task environments, it can be appropriately relaxed to reduce the system overhead caused by frequent reporting.

[0478] In addition, worker nodes can trigger events based on the task lifecycle. For example, when a new task starts, a task is completed, or a task terminates abnormally, the resource status is immediately collected and reported so that the scheduler can make more accurate scheduling decisions based on the task execution status.

[0479] It should be noted that the conditions and strategies for triggering events can be flexibly configured according to specific business needs and system characteristics. For example, in a high-priority online task environment, a more sensitive trigger threshold can be set to ensure real-time response to online task resource requirements. In an offline task scenario, the trigger conditions can be appropriately relaxed to reduce unnecessary reporting frequency, thereby optimizing the overall communication overhead and scheduling efficiency of the system while ensuring scheduling accuracy.

[0480] Through the above process, the working nodes can dynamically report the latest schedulable resource information when the resource status changes significantly or when a task lifecycle event is triggered. This ensures that the mixed deployment control layer can grasp the resource status of each working node in real time and optimize the task scheduling strategy accordingly, thereby improving the overall resource utilization of the system.

[0481] like Figure 11 As shown, this is a schematic diagram of the application logic for schedulable resource information in an embodiment of this application. Each working node at the bottom layer reports multi-dimensional schedulable resource information to the middle layer (i.e., the mixed deployment control layer), such as... Figure 11 The list includes CPU, memory, ..., disk, GPU, etc. The mixed deployment management layer also includes a mixed deployment component, which is responsible for aggregating multi-dimensional offline resources, reporting them to the upper-layer scheduler, and distributing and synchronizing offline tasks between upper and lower layers.

[0482] The implementation details of the mixed-department control layer will be introduced below.

[0483] S422: The mixed-department control layer summarizes the schedulable resource information reported by each working node and reports it to the upper layer.

[0484] In this embodiment of the application, the mixed deployment control layer receives schedulable resource information from each working node and performs summary processing to ensure that the data reported to the upper layer can accurately reflect the global resource status of the cluster.

[0485] Optionally, during the process of aggregating the schedulable resource information of each work node in the mixed-deployment control layer, the schedulable resource information data is cleaned, deduplicated, standardized, and outlier detected to ensure the consistency and accuracy of the information.

[0486] (I) Data Cleaning: The purpose of data cleaning is to remove incomplete, erroneous, or redundant data to ensure that scheduling decisions are based on accurate information. For example, some worker nodes may experience data loss due to network jitter (e.g., an empty GPU usage field). The mixed deployment control layer can use methods such as mean filling or interpolation to repair the data. Alternatively, if certain field values ​​are invalid (e.g., negative numbers or CPU usage exceeding a reasonable range), these erroneous data can be directly filtered out. Furthermore, if the data format reported by a node does not match the standard format (e.g., missing timestamps or inconsistent units), data format conversion is required to make it conform to the system standard.

[0487] (ii) Deduplication: When multiple worker nodes report data, duplicate resource information may exist. This could be due to repeated reporting by worker nodes caused by short-term fluctuations, or multiple writes caused by the data synchronization mechanism. When aggregating data, the mixed-deployment management layer can use methods such as hash verification and data fingerprint comparison to detect duplicate data and retain the latest or most complete record. For example, if a worker node reports the same GPU usage data multiple times within 5 seconds, the system can use a time window deduplication mechanism to retain only the latest report, thereby reducing redundant computation and storage consumption.

[0488] (III) Standardization: Different work nodes may use different units, formats, or precisions to report resource information. For example, some nodes may use MB / s for I / O throughput, while others may use GB / s. The mixed-deployment management layer needs to perform unified unit conversion on these data, such as converting all I / O throughput to MB / s and ensuring that all values ​​have consistent precision (e.g., uniformly retaining two decimal places) so that the scheduler can seamlessly process the data. In addition, standardization also includes time alignment. When there are discrepancies in the reporting timestamps of different nodes, the timestamps can be standardized based on the server clock or synchronization mechanism to ensure that all resource information is analyzed according to a unified time dimension.

[0489] (iv) Outlier Detection: Outlier detection is used to identify and filter out unreasonable resource usage data to prevent abnormal data from affecting task scheduling decisions. For example, the GPU usage of a node usually fluctuates between 5 and 8 cores, but if the GPU usage suddenly drops to 20 cores or some other sudden change, it is obviously unreasonable. The mixed deployment control layer can use statistical methods, such as Z-score and Interquartile Range (IQR), or machine learning methods, such as Isolation Forest and Local Outlier Factor (LOF), to detect and process this abnormal data. Outliers may also be caused by short-term resource contention or system failures. For example, a worker node may suddenly report that its CPU usage drops to 0, but at the same time the GPU is still under high load. This may indicate that the monitoring system of that node has failed or that the data reporting is incorrect. Anomalies can be identified through cross-dimensional correlation analysis (e.g., checking whether CPU, GPU, memory and other indicators are consistent) and automatic correction or alarms can be performed.

[0490] Through the above-mentioned cleaning, deduplication, standardization, and outlier detection, the mixed deployment control layer can ensure that the resource information reported by each working node has higher accuracy, stability, and consistency, providing reliable data support for upper-layer task scheduling, thereby improving the overall resource utilization and task execution efficiency of the GPU mixed deployment cluster.

[0491] Still with Figure 11 As shown in the example, this mixed-distribution control layer also includes a mixed-distribution component. When summarizing the above-mentioned multi-dimensional data, it can be achieved by plugins for multiple dimensions respectively. Figure 11 The GPU, memory, disk, and CPU plugins listed in section 11-1 each correspond to one plugin for each dimension. These plugins are responsible for aggregating the schedulable resource information for that dimension reported by each worker node, and performing the aforementioned cleaning, deduplication, standardization, and outlier detection operations during aggregation. Finally, the hybrid deployment component in section 11-1 further reports the aggregated multi-dimensional schedulable resource information to the scheduler in section 11-2.

[0492] Optionally, before reporting the aggregated schedulable resource information to the upper layer, the mixed-deployment control layer can also convert the aggregated data into a format supported by the upper layer.

[0493] For example, in a Kubernetes cluster environment, information can be reported to the upper-layer scheduler in a native Kubernetes manner. This means that the hybrid deployment component of the hybrid deployment management layer uses the standard APIs and mechanisms provided by Kubernetes to report multi-cluster, multi-dimensional schedulable resource information collected from the underlying GPU cluster hybrid deployment component to the upper-layer scheduler. This approach ensures compatibility and consistency with the Kubernetes ecosystem, enabling the upper-layer scheduler to seamlessly receive and process resource information. The standard APIs provided by Kubernetes can be Metrics APIs, Custom Metrics APIs, etc., which are not specifically limited in this article.

[0494] Optionally, to further improve data processing efficiency, the mixed-deployment management layer can also be combined with streaming computing frameworks, such as Apache Flink (a distributed stream processing framework focused on high performance and accurate computation) or Spark Streaming (a component of Apache Spark used for real-time processing of streaming data, processing data streams in a micro-batch manner), to process and aggregate the resource data of worker nodes in real time. This ensures that the upper-layer scheduler always obtains the latest global resource information and can make quick adjustment decisions when resources are scarce, such as releasing resources for low-priority offline tasks or dynamically adjusting task scheduling strategies to ensure the service quality of online tasks and improve the execution efficiency of offline tasks.

[0495] S423: The upper layer schedules offline tasks based on the aggregated schedulable resource information.

[0496] In this embodiment, the upper layer, specifically the upper-layer scheduler, makes task scheduling decisions based on the aggregated schedulable resource information and the resource requirements of the offline tasks to be scheduled (such as the required number of CPU cores, memory size, etc.).

[0497] Optionally, the upper-layer scheduler can obtain the latest global resource status through the aforementioned standard API interfaces or message queues, so as to be able to grasp the load status of each GPU node in real time during task scheduling and formulate the optimal task allocation strategy.

[0498] like Figure 11 The upper-middle layer, section 11-2, contains multiple containers used to manage user-submitted offline tasks awaiting scheduling. The scheduler makes task scheduling decisions based on multi-dimensional schedulable resource information reported by the mixed deployment control layer, as well as the resource requirements of the offline tasks awaiting scheduling, such as CPU, memory, ..., disk, GPU, etc.

[0499] Taking the aforementioned offline task as an example of deep learning model training, this task requires high GPU memory support and significant computational resources during training. The upper-layer scheduler, based on resource information provided by the distributed computing management layer, prioritizes worker nodes with sufficient GPUs and low computational load. It also considers CPU and memory usage to ensure the task can execute in the optimal computing environment. If a node has sufficient GPU resources but limited CPU resources, the upper-layer scheduler may select another node with ample CPU resources but moderate GPU load to achieve efficient utilization of computational resources and optimal task scheduling, thereby improving the execution efficiency of offline tasks and reducing interference with online tasks.

[0500] Through the above process, a multi-layered resource management and scheduling mechanism significantly improves resource utilization efficiency and scalability. First, each worker node periodically reports the latest schedulable resource information, enabling the mixed-deployment control layer to monitor the resource status of the entire cluster in real time, thus making more rational scheduling decisions. Second, the aggregated resource information provides comprehensive data support to the upper layers, helping to optimize task allocation strategies, reduce resource waste, and improve overall performance. Finally, this method enhances flexibility and scalability, enabling efficient task scheduling in multi-cluster environments and ensuring that the service quality of critical businesses remains unaffected.

[0501] To further optimize the task scheduling process, ensure the rational allocation of resources, and improve the stable operation of the system, the specific task scheduling process can be further clarified based on the multi-level resource scheduling mechanisms listed above. One possible approach is as follows:

[0502] Based on the aggregated schedulable resource information, the upper layer determines the target working node for deploying the offline tasks to be scheduled from among the working nodes; the mixed deployment control layer then distributes the offline tasks to be scheduled to the target working node for deployment.

[0503] In this embodiment, the upper layer selects the most suitable task deployment node from all working nodes at the lower layer based on the aggregated schedulable resource information and the requirements of the offline task to be scheduled. This node is the target working node.

[0504] In one optional implementation, the selection criteria include, but are not limited to, node resource availability, load balancing, and historical performance. These will be described in detail below:

[0505] Selection Criterion 1: Resource Availability. Evaluate whether the resources of each working node, such as GPU, CPU, memory, and I / O bandwidth, meet the task requirements. For example, if an offline task requires a large amount of video memory, and the GPU video memory of some nodes is already largely occupied by online tasks, then these nodes will be excluded, and nodes with sufficient video memory will be selected first.

[0506] Selection Criterion Two: Load Balancing. Consider the overall system load balancing to prevent certain worker nodes from becoming bottlenecks due to excessive task concentration. For example, if one worker node is running multiple high-load tasks while another worker node has relatively idle resources, prioritize scheduling to the idle worker node to maintain overall system load balance.

[0507] Selection Criterion 3: Historical Performance. Based on historical operational data, analyze the task completion efficiency, stability, and error rate of work nodes. If certain nodes have experienced multiple failures or performance instability in past task executions, their priority can be lowered to avoid affecting the stability of task execution.

[0508] Selection Criterion 4: Task Characteristics Matching. Different types of tasks may have specific hardware requirements. For example, some AI training tasks rely on specific models of GPUs for acceleration, while some working nodes may only have older GPUs. Therefore, it is necessary to select the most suitable computing node based on the task requirements.

[0509] It should be noted that the above filtering conditions are just simple examples. The specific filtering criteria can be flexibly configured according to actual business needs, computing cluster size, task type, resource distribution status and scheduling strategy.

[0510] After determining the target work node, a scheduling plan can be generated. The scheduling plan includes, but is not limited to, information such as the task's deployment location, expected execution time, and resource requirements. This will be explained in detail below:

[0511] (i) The deployment location of the task (the unique identifier of the target working node, node group information, etc.).

[0512] The deployment location of a task determines which worker nodes it will ultimately execute on. The scheduler needs to specify the unique identifier (ID) of the target worker node, its compute node group, physical location, and topology to ensure the rational allocation of computing resources. For example, in a large-scale GPU cluster, worker nodes may be grouped according to different hardware configurations. Furthermore, the network connection topology between different compute nodes, such as Peripheral Component Interconnect Express (PCIe) and NVIDIA Link, will also affect the communication efficiency of computing tasks. Therefore, the scheduling plan needs to include node topology information to ensure that computing tasks are optimally distributed across nodes with lower communication latency. In addition, if some tasks require access to specific data storage systems, such as local SSD cache, Non-Volatile Memory Express (NVMe) storage, or remote S3 storage, the scheduling plan also needs to consider data affinity, prioritizing scheduling to nodes closer to the data storage location to reduce I / O overhead.

[0513] (ii) Expected execution time (task priority, estimated completion time, whether resources need to be reserved, etc.).

[0514] In the embodiments of this application, the priorities of different tasks may affect their scheduling strategies. For example, high-priority tasks (such as online inference tasks) need to be executed as soon as possible and may preempt the resources of low-priority tasks, while low-priority tasks (such as offline training tasks) can be scheduled to be executed during periods of low resource utilization. The scheduling plan needs to clearly define the priority of the task in order to make the correct scheduling decision when resource contention occurs.

[0515] The estimated completion time is used to determine whether a task can be completed within a set time window. For example, if a deep learning training task is estimated to take 5 hours to run, but a worker node only has 3 hours of resource window remaining, then the task may need to be split or scheduled to another node that can provide long-term running resources. Furthermore, some tasks may involve reserved resources. For instance, some real-time inference tasks require a certain amount of GPU / CPU resources to ensure low-latency inference. In this case, the scheduling plan needs to clarify whether the task has fixed resource reservation requirements and whether it is permissible for other tasks to preempt it.

[0516] (III) Resource requirements (allocated GPU, CPU, memory, storage, etc.).

[0517] Optionally, the task's resource requirements describe the computational resources needed to execute the task, including:

[0518] GPU requirements refer to the number of GPUs needed for the task, the amount of video memory required, and whether a specific type of GPU is required. CPU requirements refer to the number of CPU cores and threads required for the task, and whether hyper-threading is supported. Memory requirements refer to the amount of RAM required for the task, whether non-uniform memory access (NUMA) affinity is required (i.e., whether memory needs to be bound to a specific CPU). Storage requirements refer to whether the task needs to use local SSDs, remote network file systems (NFS) storage, or object storage (such as S3), and the amount of storage space required. I / O bandwidth requirements refer to whether the task requires high-throughput disk or network I / O during execution; for example, video processing tasks require high-bandwidth storage read / write, while distributed training tasks may require high-speed network communication.

[0519] The scheduler needs to allocate resources reasonably on the target working nodes based on these resource requirements of the tasks, so as to ensure that the tasks will not fail due to resource shortages during operation, while avoiding over-allocation that would lead to resource waste.

[0520] (iv) Quality of Service (QoS) guarantee strategy (whether it is necessary to guarantee the minimum resource usage of the task to avoid performance degradation caused by task preemption).

[0521] QoS guarantee policies are used to ensure that tasks have sufficient computing resources during execution and to prevent resource contention from causing a decrease in task execution efficiency. For example, in a multi-tenant environment, tasks submitted by different users may compete for the same GPU / CPU resources. If high-priority tasks suddenly increase, low-priority tasks may be dequeued by the system or have their resource allocation reduced. Therefore, the scheduling plan needs to clearly define whether the task has a minimum resource guarantee.

[0522] In online tasks, QoS guarantees are particularly important because the response time of inference tasks directly affects the user experience, while offline tasks can adopt more lenient QoS policies, such as allowing tasks to automatically reduce priority or adjust computing resource allocation when resources are scarce.

[0523] (v) Task Dependencies (If a task needs to be executed in parallel or sequentially with other tasks, the tasks it depends on and the execution order should be indicated in the scheduling plan).

[0524] Some complex computational tasks may depend on multiple preceding tasks; that is, they cannot run independently but must wait for the dependent tasks to complete before they can begin execution. Therefore, the scheduling plan needs to explicitly define whether task dependencies exist. Common types of task dependencies include: sequential dependencies, parallel execution, and cross-node dependencies.

[0525] Sequential dependency refers to tasks that need to be executed in a specific order. For example, AI model training can only begin after data preprocessing is completed, and model inference and evaluation can only be performed after training is complete. Parallel execution refers to tasks that can be executed simultaneously. For example, multiple data processing tasks can run in parallel to improve overall throughput. Cross-node dependency refers to tasks that require collaboration among multiple worker nodes. For example, distributed deep learning training tasks require ensuring that tasks on all computing nodes start simultaneously to guarantee the synchronization of the training process.

[0526] The scheduler needs to optimize the scheduling order of tasks based on these dependencies to ensure that all tasks can be executed in the correct order, while avoiding tasks from being suspended for a long time while waiting for dependent resources.

[0527] By generating detailed scheduling plans, the system can precisely manage task deployment locations, expected execution times, resource requirements, QoS guarantee policies, and task dependencies, ensuring efficient task execution and full utilization of cluster resources, thereby improving overall computing power and task scheduling efficiency. Furthermore, the fine-grained management of scheduling plans supports advanced features such as task preemption, task migration, and dynamic resource adjustment, enabling the system to adapt more flexibly to different task requirements and computing environments.

[0528] After the scheduling plan is generated at the upper layer, it is sent to the mixed-department control layer.

[0529] Then, the mixed-deployment control layer distributes the offline tasks to be scheduled to the selected target work nodes according to the scheduling plan generated by the upper layer.

[0530] In this embodiment, the distribution process involves task packaging, transmission, and deployment to ensure that the task can be successfully started and run on the target worker node.

[0531] Before a task is issued, the mixed deployment control layer packages the computing resources, environmental dependencies, execution parameters, etc. required for the task. For example, some AI training tasks require specific Docker images or model files, and the system will prepare these dependencies in advance to ensure that the task can be executed smoothly on the target working node.

[0532] If the task involves large-scale data (such as deep learning training datasets or large-scale computation input files), an efficient distributed storage or data transfer mechanism is required. The mixed-distribution management layer may use remote file synchronization, such as NFS or S3, or efficient data replication strategies, such as Remote Direct Memory Access (RDMA) or data prefetching, to synchronize task data to the target worker node in advance, thereby reducing data transfer latency when the task starts.

[0533] After the target worker node receives the task, it starts the task container or virtual environment and loads the corresponding execution script to ensure the task can run normally. For example, in a Kubernetes cluster, a Pod or Job will be generated for the task, and corresponding computing resources will be allocated to it.

[0534] After the task is distributed, the mixed deployment control layer will monitor the execution status of the task and adjust resource allocation or reschedule the task as needed.

[0535] The execution status of a task refers to its performance and condition at each stage of its operation, including startup, running, paused, completion, or failure. For example... Figure 12 As shown, several offline tasks and their execution statuses are listed, such as offline task 1 in the startup stage, offline task 2 in the running stage, offline task 3 in the paused stage, ..., offline task x failed to execute.

[0536] Optionally, Kubernetes' event and state synchronization mechanisms can be leveraged to update the status of offline tasks in real time, ensuring that the upper-layer scheduler can obtain the latest job execution information. This includes information such as whether the task started successfully, current CPU and memory usage, any errors and their causes, and task completion progress. This information is updated and reported to the upper-layer scheduler in real time, allowing for adjustments to resource allocation or rescheduling of tasks based on the latest execution status, ensuring efficient and stable system operation.

[0537] In the above implementation, the upper layer determines the target worker nodes for deploying offline tasks based on the aggregated schedulable resource information, and then distributes the tasks to the target worker nodes for deployment. This method not only improves the accuracy and efficiency of task scheduling but also ensures the rational allocation of resources and the stable operation of the system. In this way, a more efficient and reliable resource management and task scheduling scheme is proposed, thereby guaranteeing the service quality of critical businesses.

[0538] In summary, taking AI training and inference scenarios as an example, the task scheduling method in the embodiments of this application will be briefly summarized below:

[0539] like Figure 13 As shown, this is a schematic diagram of another hybrid architecture in an embodiment of this application, and... Figure 2 Similarly, Figure 13 The internal composition of each layer has been further improved, and the scenarios for the task submission platform have been further refined.

[0540] During the scheduling of online tasks in this scenario, users can submit tasks through an online task submission platform, such as... Figure 13 The AI / inference online platform 13-1 in the middle submits online inference tasks. Online inference tasks typically have high real-time requirements, so it is necessary to prioritize the allocation of computing resources. The GPU nodes in the underlying layer 13-2 provide computing services for the online tasks, ensuring that the online inference tasks can obtain sufficient GPU, CPU, memory, and I / O resources to guarantee the quality of service for the inference tasks. Specifically, a hybrid deployment component, denoted as a hybrid agent, is deployed on each GPU node. The specific implementation logic of this hybrid agent can be found in the above embodiments and will not be repeated here.

[0541] When there are offline tasks to be performed (such as AI training), users can submit offline tasks to the platform, such as... Figure 13 In the training platform 13-5, offline training tasks are submitted. These tasks first enter the GPU cluster, specifically the upper layer 13-4 of the hybrid deployment architecture, and are stored in containers within 13-4. The scheduler in 13-4 schedules the tasks and distributes them to the GPU nodes in the underlying GPU cluster 13-2 via the hybrid deployment control layer 13-3. The scheduler in 13-4 schedules offline tasks based on the schedulable resource information reported by 13-3, which in turn summarizes the schedulable resource information from each GPU node in the underlying 13-2. Specifically, 13-3 also deploys hybrid deployment components and some plugins. The specific implementation logic of these components and plugins can be found in the above embodiments and will not be elaborated further here.

[0542] Based on the aforementioned hybrid deployment architecture, this application supports multi-dimensional resource mining of GPUs, CPUs, and disks within a GPU cluster, and enables multi-dimensional online service quality assurance. Compared to traditional GPU-based hybrid deployment technology that only considers the single resource dimension of GPUs, this addresses and avoids issues such as insufficient online inference service assurance capabilities, unreasonable offline task scheduling, poor data processing performance, and frequent offline task eviction when handling complex online and offline hybrid workloads.

[0543] Based on the same inventive concept, embodiments of this application also provide a task scheduling device. For example... Figure 14 As shown, this is a structural schematic diagram of the task scheduling device 1400, which may include:

[0544] Resource management unit 1401 is used to perform the following resource management operations for each worker node in the mixed deployment cluster:

[0545] Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node.

[0546] Resource interference detection is performed based on the specified dimensions to obtain conflicting resource information; wherein...

[0547] The conflict resource information represents the details of resources that interfere with online tasks due to insufficient resources;

[0548] Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated.

[0549] The task scheduling unit 1402 is used to schedule offline tasks to be scheduled based on the latest schedulable resource information of each working node.

[0550] Optionally, the resource management unit 1401 is specifically used for:

[0551] Based on the schedulable resource information and the conflicting resource information, analyze whether there is a resource management anomaly; wherein, the resource management anomaly indicates that the amount of resources occupied by offline tasks needs to be reduced;

[0552] If a resource management anomaly exists, at least one target offline task matching each target dimension is selected from the deployed tasks; wherein each target dimension is the dimension with a resource management anomaly among the plurality of specified dimensions.

[0553] Based on the release strategy corresponding to each of the target dimensions, the corresponding resources occupied by the at least one target offline task are released.

[0554] Optionally, if the resource in the target dimension is an elastic resource, then the release strategy corresponding to the target dimension is: reduce the resource usage of the task in the target dimension;

[0555] If the resource in the target dimension is an inelastic resource, then the release strategy corresponding to the target dimension is: release all resources occupied by the task.

[0556] Optionally, the resource management unit 1401 is specifically used for:

[0557] For each target offline task, if the resources occupied by the target offline task in each target dimension include non-elastic resources, then by scheduling the target offline task, all resources occupied by the target offline task are released.

[0558] For each target offline task, if the resources occupied by the target offline task in each target dimension do not include non-elastic resources, then the resources occupied by the target offline task are released by reducing the resource usage of the target offline task in each target dimension.

[0559] Optionally, the resource management unit 1401 is specifically used for:

[0560] Tasks from target task combinations selected from each task combination are designated as target offline tasks; wherein each task combination consists of at least one offline task from each deployed task that occupies resources in the target dimension; the target task combination satisfies at least one of the following matching conditions:

[0561] Matching condition 1: Under each target dimension, the total amount of resources occupied by the combination of target tasks reaches the expected reduction in resources;

[0562] Matching condition two: The sum of the sacrifice levels of each task in the target task combination is the minimum; the sacrifice level of each task represents the intensity of the impact of executing the release strategy on the execution state of the corresponding task.

[0563] Matching condition three: The total number of tasks in the target task combination is the minimum.

[0564] Optionally, when applying matching condition two or matching condition three for filtering, the target task combination must simultaneously satisfy matching condition one.

[0565] Optionally, there are multiple hybrid clusters, which are located at the bottom layer of the hybrid architecture, which also includes an upper layer and a hybrid management and control layer; the device is applied to the hybrid architecture.

[0566] The task scheduling unit 1402 is specifically used for:

[0567] For each working node, the working node reports the corresponding schedulable resource information to the mixed-distribution management layer;

[0568] The mixed-deployment control layer aggregates the schedulable resource information reported by each working node and reports it to the upper layer.

[0569] The upper layer schedules the offline tasks to be scheduled based on the aggregated schedulable resource information.

[0570] Optionally, the task scheduling unit 1402 is specifically used for:

[0571] The upper layer determines the target working node for deploying the offline task to be scheduled from among the working nodes based on the aggregated schedulable resource information.

[0572] The mixed deployment control layer distributes the offline tasks to be scheduled to the target working nodes for deployment.

[0573] Optionally, each worker node includes a hybrid component; the hybrid component is used to periodically perform resource management operations for the worker node.

[0574] In summary, the task scheduling method based on mixed-deployment clusters in this application effectively solves the common problems of task quality and workload performance degradation in traditional GPU offline mixed-deployment schemes through refined resource management operations, significantly improves system resource utilization and task execution efficiency, and at the same time ensures the performance of online tasks.

[0575] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0576] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0577] Having introduced the task scheduling method and apparatus according to exemplary embodiments of this application, we will now introduce an electronic device according to another exemplary embodiment of this application.

[0578] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0579] Based on the same inventive concept as the above-described method embodiments, this application also provides an electronic device. In one embodiment, the electronic device may be a server, such as... Figure 1The server 120 is shown. In this embodiment, the structure of the electronic device can be as follows: Figure 15 As shown, it includes a memory 1501, a communication module 1503, and one or more processors 1502.

[0580] The memory 1501 is used to store computer programs executed by the processor 1502. The memory 1501 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0581] Memory 1501 may be volatile memory, such as random-access memory (RAM); memory 1501 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or SSD; or memory 1501 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1501 may be a combination of the above-described memories.

[0582] Processor 1502 may include one or more CPUs or digital processing units, etc. Processor 1502 is used to implement the above-described task scheduling method when calling computer programs stored in memory 1501.

[0583] The communication module 1503 is used to communicate with terminal devices and other servers.

[0584] This application embodiment does not limit the specific connection medium between the memory 1501, communication module 1503, and processor 1502. This application embodiment... Figure 15 The memory 1501 and the processor 1502 are connected via a bus 1504, and the bus 1504 is in Figure 15 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 1504 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 15 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0585] The memory 1501 stores a computer storage medium, which stores computer-executable instructions for implementing the task scheduling method of the embodiments of this application. The processor 1502 is used to execute the steps of any of the above-described task scheduling methods.

[0586] As one embodiment, processor 1502 is specifically used to perform the following resource management operations on each worker node in the mixed-distribution cluster:

[0587] Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node.

[0588] Resource interference detection is performed based on the specified dimensions to obtain conflicting resource information; wherein...

[0589] The conflict resource information represents the details of resources that interfere with online tasks due to insufficient resources;

[0590] Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated.

[0591] Based on the latest schedulable resource information of each working node, the offline tasks to be scheduled are scheduled.

[0592] In some possible implementations, various aspects of the task scheduling method provided in this application can also be implemented as a program product, which includes a computer program. When the program product is run on an electronic device, the computer program causes the electronic device to perform the steps of the task scheduling method according to the various exemplary embodiments of this application described above. For example, the electronic device can perform actions such as... Figure 4 The steps are shown in the figure.

[0593] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0594] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0595] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0596] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0597] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's electronic device, partially on the user's electronic device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0598] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0599] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0600] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing a computer-usable computer program.

[0601] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0602] These computer program commands may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the commands stored in the computer-readable storage medium produce an article of manufacture including command means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0603] These computer program commands can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing the commands executed on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0604] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0605] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A task scheduling method based on mixed-deployment clusters, characterized in that, The method includes: For each worker node in the mixed-deployment cluster, perform the following resource management operations: Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node. Resource interference detection is performed based on the specified dimensions to obtain conflict resource information; wherein, the conflict resource information represents the resource details that interfere with online tasks due to insufficient resources; Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated. Based on the latest schedulable resource information of each working node, the offline tasks to be scheduled are scheduled.

2. The method as described in claim 1, characterized in that, Based on the schedulable resource information and the conflicting resource information, resources are released for at least one target offline task among the deployed tasks, including: Based on the schedulable resource information and the conflicting resource information, analyze whether there is a resource management anomaly; wherein, the resource management anomaly indicates that the amount of resources occupied by offline tasks needs to be reduced; If a resource management anomaly exists, at least one target offline task matching each target dimension is selected from the deployed tasks; wherein each target dimension is the dimension with a resource management anomaly among the plurality of specified dimensions. Based on the release strategy corresponding to each of the target dimensions, the corresponding resources occupied by the at least one target offline task are released.

3. The method as described in claim 2, characterized in that, If the resource in the target dimension is an elastic resource, then the release strategy corresponding to the target dimension is: reduce the resource usage of the task in the target dimension; If the resource in the target dimension is an inelastic resource, then the release strategy corresponding to the target dimension is: release all resources occupied by the task.

4. The method as described in claim 2 or 3, characterized in that, The step of releasing the corresponding resources occupied by the at least one target offline task according to the release strategy corresponding to each target dimension includes: For each target offline task, if the resources occupied by the target offline task in each target dimension include non-elastic resources, then by scheduling the target offline task, all resources occupied by the target offline task are released. For each target offline task, if the resources occupied by the target offline task in each target dimension do not include non-elastic resources, then the resources occupied by the target offline task are released by reducing the resource usage of the target offline task in each target dimension.

5. The method as described in claim 2 or 3, characterized in that, The step of selecting at least one target offline task that matches each target dimension from the deployed tasks includes: Tasks from target task combinations selected from each task combination are designated as target offline tasks; wherein each task combination consists of at least one offline task from each deployed task that occupies resources in the target dimension; the target task combination satisfies at least one of the following matching conditions: Matching condition 1: Under each target dimension, the total amount of resources occupied by the combination of target tasks reaches the expected reduction in resources; Matching condition two: The sum of the sacrifice levels of each task in the target task combination is the minimum; the sacrifice level of each task represents the intensity of the impact of executing the release strategy on the execution state of the corresponding task. Matching condition three: The total number of tasks in the target task combination is the minimum.

6. The method as described in claim 5, characterized in that, When applying matching condition two or matching condition three for filtering, the target task combination must simultaneously satisfy matching condition one.

7. The method according to any one of claims 1 to 3, characterized in that, There are multiple hybrid clusters, which are located at the bottom layer of the hybrid architecture. The hybrid architecture also includes an upper layer and a hybrid management and control layer. The method is applied to the hybrid architecture. The step of scheduling offline tasks based on the current schedulable resource information of each working node includes: For each working node, the working node reports the corresponding schedulable resource information to the mixed-distribution management layer; The mixed-deployment control layer aggregates the schedulable resource information reported by each working node and reports it to the upper layer. The upper layer schedules the offline tasks to be scheduled based on the aggregated schedulable resource information.

8. The method as described in claim 7, characterized in that, The scheduling of the offline tasks to be scheduled by the upper layer based on the aggregated schedulable resource information includes: The upper layer determines the target working node for deploying the offline task to be scheduled from among the working nodes based on the aggregated schedulable resource information. The mixed deployment control layer distributes the offline tasks to be scheduled to the target working nodes for deployment.

9. The method according to any one of claims 1 to 3, characterized in that, Each worker node includes a hybrid component; the hybrid component is used to periodically perform resource management operations for the worker node.

10. A task scheduling device based on a hybrid cluster, characterized in that, The device includes: The resource management unit is used to perform the following resource management operations for each worker node in the hybrid deployment cluster: Based on the resource usage information of each deployed task under multiple specified dimensions, schedulable resource information for offline tasks is obtained; wherein each deployed task is an online task or an offline task deployed on the working node. Resource interference detection is performed based on the specified dimensions to obtain conflict resource information; wherein, the conflict resource information represents the resource details that interfere with online tasks due to insufficient resources; Based on the schedulable resource information and the conflicting resource information, after releasing resources for at least one target offline task among the deployed tasks, the schedulable resource information is updated. The task scheduling unit is used to schedule offline tasks based on the latest schedulable resource information of each working node.

11. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of any one of the methods described in claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the methods described in claims 1 to 9.

13. A computer program product, characterized in that, The method includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any one of claims 1 to 9.

Images