Task deployment method, system and device, storage medium and program product

Abstract

The invention provides a task deployment method, system and device, a storage medium and a program product, relates to the technical field of computers, and is used for improving task deployment efficiency and a host resource utilization rate. The method is applied to a scheduler and comprises the following steps: acquiring configuration information of a to-be-scheduled task; the configuration information is used for reflecting the demand of the to-be-scheduled task; based on the configuration information, determining an architecture type of a host required by the to-be-scheduled task; and determining a deployment host of the to-be-scheduled task based on the architecture type and a preset scheduling strategy.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task deployment method, system, device, storage medium, and program product. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have entered the stage of large-scale commercial deployment. Although task processing based on graphics processing units (GPUs) has superior performance, the high hardware cost has prompted the industry to actively explore task deployment solutions based on general-purpose central processing units (CPUs) to reduce costs.

[0003] Currently, in heterogeneous cluster environments that use a mix of hosts with different architectures such as x86 and ARM, task deployment is achieved by operations and maintenance personnel identifying and recording the specific architecture type of each host, then manually selecting the appropriate software deployment media (such as container images or binary packages for specific architectures) based on the architecture type, and writing and configuring differentiated resource management strategies for hosts with different architectures.

[0004] However, the above methods have low task deployment efficiency and low host resource utilization. Summary of the Invention

[0005] This application provides a task deployment method, system, device, storage medium, and program product to improve task deployment efficiency and host resource utilization.

[0006] In a first aspect, this application provides a task deployment method applied to a scheduler, comprising: obtaining configuration information of a task to be scheduled; the configuration information being used to reflect the requirements of the task to be scheduled; determining the architecture type of the host required by the task to be scheduled based on the configuration information; and determining the deployment host of the task to be scheduled based on the architecture type and a preset scheduling strategy.

[0007] The technical solution provided in this application offers at least the following benefits: By obtaining the configuration information of the task to be scheduled and determining the architecture type of the host required for the task, the tedious process of manually selecting an appropriate architecture for the task can be avoided. Furthermore, based on the architecture type and preset scheduling strategy, a suitable deployment host can be matched and determined for the task, improving the efficiency of task deployment and enhancing the overall resource utilization in a heterogeneous host cluster.

[0008] One possible implementation is that the preset scheduling strategy includes at least one of the following: when the task to be scheduled is a large model task, select a host with isolation capability for the task to be scheduled; select a host with remaining resources greater than the resource requirements of the task to be scheduled based on the resource requirements of the task to be scheduled; select a host for the task to be scheduled based on the priority of the task to be scheduled.

[0009] Another possible implementation involves configuring information including at least one of the following: task name, task type, resource requirements, task priority, and host architecture type.

[0010] Another possible implementation method includes: writing the identifier of the deployment host into the configuration information of the task to be scheduled, and adjusting the task status of the task to be scheduled to a task to be deployed.

[0011] Secondly, this application provides a task deployment method applied to an executor of a host, comprising: reading at least one task to be deployed stored in a memory; obtaining configuration information of the target task when a target task is detected; wherein the target task is a task deployed on the local machine among at least one task to be deployed; and deploying the target task on the local machine based on the configuration information of the target task.

[0012] The technical solution provided in this application offers at least the following advantages: The host's executor reads the list of tasks to be deployed, which has already been decided by the scheduler, from the memory, and identifies the target task whose deployment host is the local machine. Then, it obtains the configuration information of the target task and deploys the task on the local machine. This process, by identifying the target task through configuration information, improves the reliability of task deployment on the local machine.

[0013] One possible implementation is that the configuration information includes at least one of the following: task name, task type, resource requirements, task priority, host identifier, and host architecture type; wherein the host identifier is consistent with the local machine identifier.

[0014] Another possible implementation is to deploy the target task on the local machine based on the target task's configuration information. This includes: when the target task is a large model task, determining the resource isolation strategy for the target task based on the local machine's architecture type and the target task's configuration information; wherein the resource isolation strategy is used to indicate that the target task is bound to a specified CPU node.

[0015] Another possible implementation involves, after determining the resource isolation strategy for the target task, the method further including: during the execution of the target task, detecting the resource usage and performance indicators of the target task, and adjusting the resource isolation strategy of the target task when resource contention or performance interference is detected.

[0016] Thirdly, this application provides a task deployment system, comprising: a memory, a scheduler, and at least one host; wherein the memory is connected to the scheduler and at least one host respectively; the host includes an executor; the memory is used to store configuration information of tasks; the configuration information is used to reflect the requirements of the tasks; the scheduler is used to read the configuration information of the tasks to be scheduled from the memory; and determine the deployment host of the tasks to be scheduled based on the configuration information; the scheduler is also used to write the identifier of the deployment host into the configuration information of the tasks to be scheduled, update the configuration information of the tasks to be scheduled stored in the memory, and adjust the task status of the tasks to be scheduled to be deployed; the executor of the host is used to read at least one task to be deployed stored in the memory; and, when a target task is detected, obtain the configuration information of the target task; wherein the target task is a task among at least one task to be deployed that is deployed on the local machine; and deploy the target task on the local machine based on the configuration information of the target task.

[0017] One possible implementation is a scheduler, specifically used to determine the architecture type of the host required for the task to be scheduled based on configuration information; and to determine the deployment host of the task to be scheduled based on the architecture type and a preset scheduling policy.

[0018] Another possible implementation involves the host executor also determining the resource isolation strategy for the target task based on the host's architecture type and the target task's configuration information, when the target task is a large-model task; wherein the resource isolation strategy is used to indicate that the target task should be bound to a specified CPU node.

[0019] Another possible implementation is that the host executor is also used to detect the resource usage and performance indicators of the target task during its execution, and adjust the resource isolation strategy of the target task when resource contention or performance interference is detected.

[0020] Another possible implementation includes an API gateway connected to the storage; the API gateway receives user-submitted tasks, writes the task configuration information to the storage, and configures the task status as a task to be scheduled.

[0021] Fourthly, this application provides a task deployment apparatus applied to a scheduler, comprising: an acquisition unit and a processing unit; the acquisition unit is used to acquire configuration information of a task to be scheduled; the configuration information is used to reflect the requirements of the task to be scheduled; the processing unit is used to determine the architecture type of the host required by the task to be scheduled based on the configuration information; the processing unit is used to determine the deployment host of the task to be scheduled based on the architecture type and a preset scheduling strategy.

[0022] One possible implementation is a processing unit, specifically used to select a host with isolation capabilities for a large-scale task to be scheduled; select a host with remaining resources greater than the resource requirements of the task to be scheduled based on the resource requirements of the task to be scheduled; and select a host for the task to be scheduled based on the priority of the task to be scheduled.

[0023] Another possible implementation is a processing unit, specifically used to write the identifier of the deployment host into the configuration information of the task to be scheduled, and to adjust the task status of the task to be scheduled to a task to be deployed.

[0024] Fifthly, this application provides a task deployment apparatus applied to an executor of a host machine, comprising: an acquisition unit and a deployment unit; the acquisition unit is configured to read at least one task to be deployed stored in a memory; the acquisition unit is configured to acquire configuration information of the target task when a target task is detected; wherein the target task is a task among the at least one task to be deployed that is deployed on the local machine; the deployment unit is configured to deploy the target task on the local machine based on the configuration information of the target task.

[0025] One possible implementation is a deployment unit, specifically configured to determine a resource isolation strategy for the target task based on the local architecture type and the configuration information of the target task when the target task is a large-model task; wherein the resource isolation strategy is used to indicate that the target task is bound to a specified CPU node.

[0026] Another possible implementation is a deployment unit, which is specifically used to detect the resource usage and performance indicators of the target task during its operation, and adjust the resource isolation strategy of the target task when resource contention or performance interference is detected.

[0027] In a sixth aspect, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the methods of the first or second aspect described above.

[0028] In a seventh aspect, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the methods of the first or second aspect described above.

[0029] Eighthly, this application provides a computer program product comprising a computer program; when the computer program is run in an electronic device, the electronic device performs the methods described in the first or second aspect.

[0030] The beneficial effects of the third to eighth aspects mentioned above are described in the corresponding descriptions of the first or second aspects, and will not be repeated here. Attached Figure Description

[0031] Figure 1 A system architecture diagram of a task deployment method provided in this application; Figure 2 A flowchart of a task deployment method provided in this application; Figure 3 A flowchart of another task deployment method provided in this application; Figure 4 A schematic diagram illustrating a resource isolation method provided in this application; Figure 5 A schematic diagram illustrating yet another resource isolation method provided in this application; Figure 6 A flowchart of yet another task deployment method provided in this application; Figure 7 A schematic diagram of the composition of a mission deployment device provided in this application; Figure 8 A schematic diagram illustrating the composition of yet another mission deployment device provided in this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0032] The following is a detailed description of a task deployment method, system, device, storage medium, and program product provided in this application, with reference to the accompanying drawings.

[0033] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0034] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0035] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0036] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0037] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0038] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0039] As data centers and computing infrastructure evolve towards heterogeneity and scale, task deployment technologies increasingly face the challenge of complex environments involving the mixed deployment of hosts with various CPU architectures such as x86 and ARM. This is especially true in cloud computing platforms or distributed edge computing, where it is crucial to efficiently handle large-scale model inference tasks alongside traditional web applications, microservices, and other common applications.

[0040] Currently, in heterogeneous clusters containing hosts with various architectures such as x86 and ARM, large-scale applications and general applications are deployed in a mixed manner. Task deployment is often handled manually or semi-automatically: operations personnel need to pre-identify and record the specific host architecture type of each node in the cluster using scripting tools, creating a static configuration list. During task deployment, for applications with specific architecture requirements (such as certain tasks that only provide x86 compiled versions), the host architecture type must be manually specified in the deployment description file (such as a Kubernetes YAML file).

[0041] However, the above methods have the following drawbacks: First, they rely heavily on manual identification and static configuration, making the task deployment process cumbersome and prone to errors when facing large-scale clusters with hundreds or thousands of heterogeneous nodes. Second, the x86 and ARM architectures use almost the same cgroup isolation strategy, failing to fully utilize the NUMA hardware characteristics of the x86 architecture to achieve deep isolation of memory bandwidth and access latency, and also failing to design a refined control group quota strategy for the ARM architecture, resulting in limited isolation effectiveness. Third, due to the lack of awareness of task characteristics and corresponding preventive scheduling, large-scale tasks are prone to fierce resource competition with ordinary applications on shared hosts, causing a significant increase in response latency and a decrease in throughput for ordinary applications. Fourth, the resource allocation strategy cannot be dynamically adjusted according to the actual load characteristics of the task during runtime, resulting in hardware resources (such as the high memory bandwidth of x86 and the high energy efficiency core of ARM) not being utilized by the most suitable task, causing resource idleness and low resource utilization and cost-effectiveness.

[0042] To address the aforementioned technical issues, this application provides a task deployment method. The method's approach involves obtaining the configuration information of the task to be scheduled and determining the required host architecture type, thus avoiding the tedious process of manually selecting a suitable architecture for the task. Based on this, and according to the architecture type and preset scheduling strategy, a suitable deployment host is matched and determined for the task, improving task deployment efficiency and enhancing overall resource utilization in heterogeneous host clusters.

[0043] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0044] This application provides a task deployment method that can be applied to, for example... Figure 1 In the system architecture shown. For example... Figure 1 As shown, the system architecture includes: API gateway 10, memory 20, scheduler 30, and host 40. API gateway 10 is interconnected with memory 20, memory 20 is interconnected with scheduler 30, and memory 20 is interconnected with host 40.

[0045] In some embodiments, the Application Programming Interface (API) gateway 10 is used to write the configuration information of the task submitted by the user into the storage 20 and to configure the task status as a task to be scheduled.

[0046] For example, configuration information is used to reflect the requirements of a task.

[0047] For example, API Gateway 10 serves as the unified northbound access point for the system, used to receive and process task deployment requests submitted by users through standard interfaces (such as RESTful APIs).

[0048] In some embodiments, the API gateway 10 also has an authentication function, which can identify the permissions of different users.

[0049] For example, API Gateway 10 performs identity authentication and permission authorization on the source of task deployment requests to ensure that only authorized users or services can submit deployment operations; it also verifies the format, range, and compliance of input parameters to prevent abnormal configurations from flowing into the system.

[0050] For example, for a verified task, the API gateway 10 serializes the complete configuration information related to the task (such as task identifier, resource requirements, architecture constraints, priority, etc.) into a common structured data format such as JSON or YAML, marks it as a task to be scheduled, and writes it into the storage 20.

[0051] In some embodiments, the memory 20 is used to store configuration information for the task.

[0052] In some embodiments, the scheduler 30 is used to read configuration information of the task to be scheduled from the memory; and based on the configuration information, to determine the deployment host of the task to be scheduled.

[0053] In some embodiments, the scheduler 30 is specifically used to determine the architecture type of the host required for the task to be scheduled based on configuration information; and to determine the deployment host of the task to be scheduled based on the architecture type and a preset scheduling policy.

[0054] In some embodiments, the scheduler 30 is further configured to write the identifier of the deployment host into the configuration information of the task to be scheduled, update the configuration information of the task to be scheduled stored in the memory, and adjust the task status of the task to be scheduled to the task to be deployed.

[0055] In some embodiments, the host 40 includes an actuator 401, which is used to read at least one task to be deployed stored in the memory 20; and, if a target task is detected, to obtain configuration information of the target task; wherein the target task is a task deployed on the local machine among at least one task to be deployed; and to deploy the target task on the local machine based on the configuration information of the target task.

[0056] In some embodiments, the executor 401 of the host 40 is further configured to determine a resource isolation policy for the target task based on the host's architecture type and the target task's configuration information when the target task is a large model task; wherein the resource isolation policy is used to indicate that the target task is bound to a specified CPU node.

[0057] In some embodiments, the executor 401 of the host 40 is also used to detect the resource usage and performance indicators of the target task during the execution of the target task, and adjust the resource isolation strategy of the target task when resource contention or performance interference is detected.

[0058] It should be noted that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0059] See Figure 2 This is a flowchart illustrating a task deployment method provided in an embodiment of this application. Figure 2 As shown, the task deployment method provided in this application can be implemented by the scheduler described above, and specifically includes the following steps S201~S203.

[0060] S201. Obtain the configuration information of the task to be scheduled.

[0061] The configuration information is used to reflect the requirements of the tasks to be scheduled.

[0062] In some embodiments, a task to be scheduled refers to a task instance that has been submitted through the API gateway and completed preliminary verification, but has not yet been assigned to a specific host for execution by the scheduler.

[0063] In some embodiments, the memory is used to store configuration information for different tasks, and step S201 above can be implemented as: obtaining the configuration information of the task to be scheduled from the memory.

[0064] In some embodiments, the configuration information includes at least one of the following: task name, task type, resource requirements, task priority, and host architecture type.

[0065] For example, the task name is a unique identifier for the task, used to track and manage the task's lifecycle. For instance: llm-inference-20240327-001.

[0066] For example, task type is used to identify the computational characteristics and behavioral patterns of a task, and may include: large model tasks (such as large model inference tasks) and general applications (such as web service tasks, real-time stream processing tasks, etc.).

[0067] For example, resource requirements are defined in a quantitative form as the minimum or expected computing resources required to run a task. Such as: the number of CPU cores required by the task (e.g., cpu_cores: 8), the memory capacity required by the task (e.g., memory_gb: 32), the storage space required by the task (e.g., storage_gb: 100), etc.

[0068] For example, task priority defines the importance of a task, typically represented by an enumeration value (such as high, medium, low) or a numerical level (such as 0-9). High-priority tasks have priority in scheduling resource allocation and are given priority to ensure the service quality of critical business operations.

[0069] For example, the host architecture type is the hardware architecture constraint of the runtime environment on which the task operates. The host architecture type can be: x86, ARM, any, RISC-V, MIPS, or PowerPC, etc.

[0070] As one possible approach, the scheduler periodically scans data tables or queues in storage where tasks are in a pending scheduling state, and extracts task identifiers and their associated configuration information in batches.

[0071] As another possible approach, once the API gateway writes a new task to storage, it can send an event notification to the scheduler (e.g., via a message broker), and the scheduler can retrieve the configuration information of the task to be scheduled from storage.

[0072] S202. Based on the configuration information, determine the architecture type of the host required for the task to be scheduled.

[0073] In some embodiments, the scheduler determines the architecture type (such as x86 or ARM) of the host required (or preferred) for the task to be scheduled by analyzing one or more key elements in the configuration information.

[0074] As an alternative approach, if the configuration information includes a field specifying the host architecture type (such as the value of architecture or cpu_arch being "x86", "ARM64", or "any"), the scheduler will directly adopt this user-specified architecture type.

[0075] As another possible approach, the configuration information includes a task type (task_type) field, which the scheduler will infer based on predefined "task type-architecture preference" mapping rules.

[0076] For example, when task_type is "large model task", it can be mapped to "x86" by default. When task_type is "normal application", it can be mapped to "ARM" by default, which can strike a balance between cost and performance.

[0077] As another possible approach, for repetitive tasks, the scheduler can query historical execution records. If the task or similar tasks have historically achieved better performance metrics (such as lower latency and higher throughput) on a certain architecture host, then that historically chosen architecture type can be recommended.

[0078] As another possible approach, configuration information may implicitly include architecture dependencies through container images (such as the `image` field) or binary programs (such as the `binary` field). The scheduler can infer the architecture type required for task execution by parsing image tags (such as `image: myapp:latest-ARM64`) or the metadata of the binary file.

[0079] S203. Based on the architecture type and preset scheduling policy, determine the deployment host of the task to be scheduled.

[0080] As one possible approach, when the task to be scheduled is a large-scale task, a host with isolation capabilities can be selected for the task to be scheduled.

[0081] For example, for a host cluster with an x86 architecture, the scheduler further checks whether each host has an available independent NUMA node with sufficient CPU cores and memory capacity to meet the requirements of the large model task, so as to ensure that the large model task can be bound to a dedicated NUMA node for execution.

[0082] As another possible approach, a host with more remaining resources than the resource requirements of the task to be scheduled can be selected based on the resource requirements of the task to be scheduled.

[0083] For example, the scheduler obtains the resource requirements such as cpu_cores and memory_gb explicitly stated in the configuration information, and queries the real-time resource usage and remaining available resources of all hosts in the cluster. It then selects the deployment host from among the hosts whose available CPU cores are greater than or equal to the number of CPU cores required by the task, and whose available memory capacity is greater than or equal to the memory capacity required by the task.

[0084] As another possible approach, a host is selected for a task based on its priority.

[0085] For example, for high-priority tasks to be scheduled, the scheduler will prioritize the host with the lightest current load, the largest performance margin, and the lowest network latency to ensure that high-priority tasks get the best running environment.

[0086] In some embodiments, the identifier of the deployment host is written into the configuration information of the task to be scheduled, and the task status of the task to be scheduled is adjusted to the task to be deployed.

[0087] For example, after making a host selection decision, the scheduler initiates an update operation to the shared storage. In the data record of the corresponding task, the value of the assigned_node field is set to the unique identifier of the selected host (such as hostname or IP address), and the status field is updated from a task to be scheduled to a task to be deployed. Then, the executor component on the target host discovers and pulls the task configuration to complete the deployment and execution of the task.

[0088] Based on the above embodiments, by obtaining the configuration information of the task to be scheduled and determining the architecture type of the host required for the task, the tedious process of manually selecting an appropriate architecture for the task can be avoided. Furthermore, by matching and determining a suitable deployment host for the task according to the architecture type and preset scheduling strategy, the efficiency of task deployment can be improved, and the overall resource utilization in the heterogeneous host cluster can also be enhanced.

[0089] In some embodiments, such as Figure 3 As shown, the task deployment method provided in this application can be implemented through the executor of the aforementioned host, specifically including the following steps S301~S303.

[0090] S301, Read at least one task to be deployed stored in the memory.

[0091] In some embodiments, a task to be deployed refers to a task that has been explicitly assigned to a specific target host in the scheduler and whose task status has been updated to a task to be deployed, but has not yet been deployed or run on the target host.

[0092] As one possible approach, the executor proactively initiates query requests to the memory at preset time intervals (e.g., every 5 seconds). Query criteria typically include: the task status is "task to be deployed" and the identifier of the deployment host matches the identifier of the local machine (e.g., hostname, host ID, or IP address). The executor retrieves all task records that meet these criteria, forming a list of tasks to be deployed for the current round.

[0093] As another possible approach, when the scheduler updates a task's status to "to be deployed" and writes it to memory, it can simultaneously publish an event notification. The local executor subscribes to this event, and upon receiving an event notification related to the local machine, it immediately retrieves the complete configuration information of the task from memory based on the task identifier carried in the notification.

[0094] S302. If the target task is detected, obtain the configuration information of the target task.

[0095] The target task is at least one task to be deployed that is deployed on the local machine.

[0096] In some embodiments, the configuration information includes at least one of the following: task name, task type, resource requirements, task priority, host identifier, and host architecture type. The host identifier is identical to the identifier of the local machine.

[0097] S303. Based on the configuration information of the target task, deploy the target task on the local machine.

[0098] In some embodiments, when the target task is a large model task, the resource isolation strategy for the target task is determined based on the local architecture type and the configuration information of the target task.

[0099] The resource isolation policy is used to instruct the target task to be bound to a specified CPU node.

[0100] For example, the executor detects the local host's architecture type (such as x86, ARM, etc.) by reading and parsing key fields in the standard system file / proc / cpuinfo provided by the host operating system.

[0101] In some embodiments, for x86 hosts, the executor selects the most idle NUMA node and binds the target task to the CPU on the NUMA node. If a host has multiple NUMA nodes, multiple large model tasks can be deployed if resources are sufficient.

[0102] For example, such as Figure 4 As shown, on an x86 host, multiple large model tasks (such as large model instance 1, large model instance 2) and / or ordinary applications (such as ordinary application 1, ordinary application 2, ordinary application 3) can be deployed simultaneously. Large model instance 1 is bound to CPU0 and CPU1 of the numa node0, large model instance 2 is bound to CPU2 and CPU3 of the numa node1, and ordinary applications can run on any CPU with available resources.

[0103] For example, the implementation of the resource isolation strategy based on NUMA binding is as follows: First, the executor runs the numactl command in the local environment, binding the task process to a specific NUMA node by specifying the --cpunodebind parameter. Second, when starting or managing the task process, the executor calls programming interfaces such as sched_setaffinity(0, sizeof(cpu_set_t)) in the code to explicitly specify the CPUs that the process can run on by constructing and setting the cpu_set_t bitmap, thereby restricting its execution to the CPUs contained in the target NUMA node.

[0104] In some embodiments, for an ARM host, the executor creates one or more CGroups based on the task demand, selects idle CPUs and puts them into the CGroups. If resources are sufficient, multiple large model tasks can be deployed.

[0105] For example, such as Figure 5 As shown, on an ARM host, multiple large model tasks (such as large model instance 1, large model instance 2) and / or ordinary applications (such as ordinary application 1, ordinary application 2, ordinary application 3) can be deployed simultaneously. Large model instance 1 is bound to CPU0 and CPU1 in CGroup 0, large model instance 2 is bound to CPU2 and CPU3 in CGroup 1, and ordinary applications can run on any CPU with available resources.

[0106] For example, resource isolation strategies using CGroup can be implemented in two ways: First, CGroups can be created and configured using `cpuset`, assigning specific CPUs to CGroups and writing process PIDs to the `tasks` file for binding. Alternatively, management tools such as `systemd` can be used to dynamically set CGroup control parameters such as CPU and memory limits for tasks via unit files or the `systemctl set-property` command. Second, fine-grained binding of CPU cores can be achieved by calling system calls such as `sched_setaffinity()`. For instance, in a C / C++ program, a CPU core bitmap can be constructed using `CPU_ZERO(&cpuset)` and `CPU_SET(core_id,&cpuset)`, and then `sched_setaffinity(pid, sizeof(cpu_set_t),&cpuset)` can be called to bind tasks to the specified CPUs.

[0107] For example, after the call, the return value should usually be checked. If the binding fails (e.g., returning -1), error handling should be performed, such as logging and returning an error code: if (sched_setaffinity(0, sizeof(cpu_set_t),&cpuset) ==-1) { perror("sched_setaffinity failed"); return 1;}.

[0108] In some embodiments, during the execution of the target task, the resource usage and performance indicators of the target task are detected, and when resource contention or performance interference is detected, the resource isolation strategy of the target task is adjusted.

[0109] For example, a lightweight monitoring agent is deployed on each host to continuously monitor the resource usage and performance metrics of the target task, including but not limited to: memory bandwidth utilization, CPU core load distribution, cache miss rate, cross-node memory access latency, and service quality metrics such as response time and throughput of multiple tasks running on the same host. When resource contention or performance interference is detected (such as the memory bandwidth saturation of the NUMA node where a large model task resides consistently exceeding 85%), the executor on the local machine will adjust the resource isolation strategy.

[0110] For example, in a NUMA isolation scenario, if the host load continues to increase and affects the target task, the executor can dynamically adjust the number of CPU cores on the bound NUMA node. In a CGroup isolation scenario, the executor can dynamically adjust the number of CPU cores specified in the CGroup.

[0111] Based on the above embodiments, the host's executor reads the list of tasks to be deployed that has been decided by the scheduler in the memory, and identifies the target task whose deployment host is the local machine. Then, it obtains the configuration information of the target task and deploys the task on the local machine. This process, by identifying the target task through configuration information, improves the reliability of task deployment on the local machine.

[0112] The task deployment method of this application embodiment is described below with reference to a specific example. The specific implementation process of this method is as follows: Figure 6 As shown.

[0113] S601. Users submit tasks through the API gateway, and the task configuration information is recorded as a task to be scheduled and written into the memory.

[0114] S602, The scheduler retrieves the configuration information of the task to be scheduled from the memory.

[0115] S603. The scheduler determines the architecture type of the host required for the task to be scheduled based on the configuration information.

[0116] S604. The scheduler determines the deployment host of the task to be scheduled based on the architecture type and the preset scheduling policy, and changes the task to the task to be deployed and writes it into the memory.

[0117] S605, The host's executor reads at least one task to be deployed stored in the memory.

[0118] S606. When the executor detects the target task, it obtains the configuration information of the target task.

[0119] S607. The executor deploys the target task on the local machine according to the configuration information of the target task.

[0120] S608: When the target task is a large model task, the executor determines the resource isolation strategy of the target task based on the local architecture type and the configuration information of the target task.

[0121] S609. During the execution of the target task, the actuator detects the resource usage and performance indicators of the target task.

[0122] S610. When the actuator detects resource contention or performance interference, it adjusts the resource isolation strategy of the target task.

[0123] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0124] This application embodiment can divide the task deployment device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0125] In some embodiments, this application also provides a task deployment apparatus. The task deployment apparatus may include one or more functional units for implementing the task deployment method of the above method embodiments.

[0126] For example, Figure 7 This is a schematic diagram illustrating the composition of a task deployment device provided in an embodiment of this application. Figure 7 As shown, the task deployment device 700 is applied to the scheduler and includes: an acquisition unit 701 and a processing unit 702.

[0127] The acquisition unit 701 is used to acquire the configuration information of the task to be scheduled; the configuration information is used to reflect the requirements of the task to be scheduled; the processing unit 702 is used to determine the architecture type of the host required by the task to be scheduled based on the configuration information; the processing unit 702 is used to determine the deployment host of the task to be scheduled based on the architecture type and the preset scheduling policy.

[0128] In some embodiments, the processing unit 702 is specifically configured to: select a host with isolation capability for the task to be scheduled when the task to be scheduled is a large model task; select a host with remaining resources greater than the resource requirements of the task to be scheduled based on the resource requirements of the task to be scheduled; and select a host for the task to be scheduled based on the priority of the task to be scheduled.

[0129] In other embodiments, the processing unit 702 is specifically used to write the identifier of the deployment host into the configuration information of the task to be scheduled, and to adjust the task status of the task to be scheduled to the task to be deployed.

[0130] For example, Figure 8 This is a schematic diagram illustrating the composition of a task deployment device provided in an embodiment of this application. Figure 8 As shown, the task deployment device 800 is applied to the executor of the host and includes: an acquisition unit 801 and a deployment unit 802.

[0131] The acquisition unit 801 is used to read at least one task to be deployed stored in the memory; the acquisition unit 801 is used to acquire the configuration information of the target task when the target task is detected; wherein the target task is the task deployed on the local machine among the at least one task to be deployed; the deployment unit 802 is used to deploy the target task on the local machine based on the configuration information of the target task.

[0132] In some embodiments, deployment unit 802 is specifically configured to determine a resource isolation strategy for the target task based on the architecture type of the local machine and the configuration information of the target task when the target task is a large model task; wherein the resource isolation strategy is used to indicate that the target task is bound to a specified CPU node.

[0133] In other embodiments, the deployment unit 802 is specifically used to detect the resource usage and performance indicators of the target task during the operation of the target task, and adjust the resource isolation strategy of the target task when resource contention or performance interference is detected.

[0134] In the case of implementing the functions of the integrated modules described above in hardware, this embodiment of the invention provides a possible structural schematic diagram of the electronic device involved in the above embodiments. For example... Figure 9 As shown, the electronic device 900 includes: a processor 902, a communication interface 903, and a bus 904. Optionally, the electronic device 900 may also include a memory 901.

[0135] Processor 902 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0136] The communication interface 903 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0137] The memory 901 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0138] In one possible implementation, the memory 901 can exist independently of the processor 902. The memory 901 can be connected to the processor 902 via a bus 904 and is used to store instructions or program code. When the processor 902 calls and executes the instructions or program code stored in the memory 901, it can implement the task deployment method provided in this embodiment of the invention.

[0139] In another possible implementation, the memory 901 can also be integrated with the processor 902.

[0140] The 904 bus can be an extended industry standard architecture (EISA) bus, etc. The 904 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0141] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0142] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the aforementioned computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The aforementioned computer-readable storage medium can also be an external storage device of the aforementioned service invocation device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the aforementioned service invocation device. Further, the aforementioned computer-readable storage medium can include both internal storage units of the aforementioned service invocation device and external storage devices. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned service invocation device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0143] This application also provides a computer program product comprising a computer program that, when run on a computer, causes the computer to perform any of the task deployment methods provided in the above embodiments.

[0144] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A task deployment method, characterized in that, Applied to a scheduler, the method includes: Obtain the configuration information of the task to be scheduled; the configuration information is used to reflect the requirements of the task to be scheduled. Based on the configuration information, determine the architecture type of the host required for the task to be scheduled; Based on the architecture type and preset scheduling policy, the deployment host of the task to be scheduled is determined.

2. The method according to claim 1, characterized in that, The preset scheduling strategy includes at least one of the following: When the task to be scheduled is a large model task, a host with isolation capabilities is selected for the task to be scheduled. Based on the resource requirements of the task to be scheduled, select a host with remaining resources greater than the resource requirements of the task to be scheduled. Based on the priority of the task to be scheduled, a host is selected for the task to be scheduled.

3. The method according to claim 1, characterized in that, The configuration information includes at least one of the following: Task name, task type, resource requirements, task priority, and host architecture type.

4. The method according to claim 1, characterized in that, The method further includes: Write the identifier of the deployment host into the configuration information of the task to be scheduled, and adjust the task status of the task to be scheduled to a task to be deployed.

5. A task deployment method, characterized in that, An actuator applied to a host computer, the method comprising: Read at least one task to be deployed stored in memory; Upon detecting a target task, the configuration information of the target task is obtained; wherein the target task is a task deployed on the local machine among the at least one task to be deployed; Based on the configuration information of the target task, the target task is deployed on the local machine.

6. The method according to claim 5, characterized in that, The configuration information includes at least one of the following: The task name, task type, resource requirements, task priority, host identifier, and host architecture type are specified; wherein the host identifier is consistent with the identifier of the local machine.

7. The method according to claim 5, characterized in that, The step of deploying the target task on the local machine based on the configuration information of the target task includes: When the target task is a large-scale task, a resource isolation strategy for the target task is determined based on the local architecture type and the configuration information of the target task; wherein, the resource isolation strategy is used to indicate that the target task is bound to a specified CPU node.

8. The method according to claim 7, characterized in that, After determining the resource isolation strategy for the target task, the method further includes: During the execution of the target task, the resource usage and performance indicators of the target task are monitored. When resource contention or performance interference is detected, the resource isolation strategy of the target task is adjusted.

9. A task deployment system, characterized in that, The system includes: a memory, a scheduler, and at least one host; wherein the memory is connected to the scheduler and the at least one host; the host includes an executor; The memory is used to store the task's configuration information; the configuration information is used to reflect the task's requirements. The scheduler is configured to read configuration information of the task to be scheduled from the memory; and determine the deployment host of the task to be scheduled based on the configuration information. The scheduler is also used to write the identifier of the deployment host into the configuration information of the task to be scheduled, update the configuration information of the task to be scheduled stored in the memory, and adjust the task status of the task to be scheduled to a task to be deployed. The executor of the host is configured to read at least one task to be deployed stored in the memory; and, upon detecting a target task, obtain the configuration information of the target task; wherein the target task is a task among the at least one task to be deployed that is deployed on the local machine; and, based on the configuration information of the target task, deploy the target task on the local machine.

10. The system according to claim 9, characterized in that, The scheduler is specifically used to determine the architecture type of the host required for the task to be scheduled based on the configuration information. Based on the architecture type and preset scheduling policy, the deployment host of the task to be scheduled is determined.

11. The system according to claim 9, characterized in that, The executor of the host is further configured to determine a resource isolation policy for the target task based on the host's architecture type and the configuration information of the target task when the target task is a large model task; wherein the resource isolation policy is used to indicate that the target task is bound to a specified CPU node.

12. The system according to claim 11, characterized in that, The executor of the host is also used to detect the resource usage and performance indicators of the target task during the execution of the target task, and adjust the resource isolation strategy of the target task when resource contention or performance interference is detected.

13. The system according to claim 9, characterized in that, The system also includes an API gateway, which is connected to the memory; The API gateway is used to receive tasks submitted by users, write the configuration information of the tasks into the memory, and configure the task status of the tasks as tasks to be scheduled.

14. An electronic device, characterized in that, It includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the task deployment method as described in any one of claims 1 to 8.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the task deployment method as described in any one of claims 1 to 8.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the task deployment method as described in any one of claims 1 to 8.

Images