Task scheduling method and related equipment

Abstract

The present disclosure provides a task scheduling method and related equipment, which relate to the field of computer cloud processing technology. First, the amount of resources of the edge node and the Z tasks to be scheduled are obtained, and then the allocation relationship is determined based on the amount of resources of the edge node, and the primary population is calculated based on the genetic algorithm, and the primary population is iterated based on the target fitness function and the allocation relationship to obtain the target code, and the Z tasks are scheduled according to the target code and the amount of resources of the edge node. The present disclosure determines the allocation relationship through the amount of resources of the edge node and the demand for the task, and iteratively trains the weights corresponding to the task based on the genetic algorithm, the allocation relationship and the target fitness function to obtain the target code, and schedules the Z tasks according to the target code and the amount of resources of the edge node, thereby improving the utilization of resources, reducing the network load, and improving the overall performance and service quality of the AI ​​large model reasoning business.

Description

Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, especially large models, the demand for computing resources is increasing. However, the traditional centralized cloud computing model is far away from the data source, resulting in long communication links, large data transmission volume, and concentrated service requests, which leads to problems such as delayed response time, increased bandwidth pressure and network congestion, and ultimately affects the quality of service. To solve this problem, distributed cloud, as an evolutionary solution for cloud computing infrastructure, sinks cloud resources to the edge of the network through the collaboration of central cloud and edge cloud, alleviates the backbone network congestion caused by centralized cloud computing, and effectively meets the high real-time requirements of AI large model inference business.

[0003] In a distributed cloud environment, how to efficiently schedule (offload) tasks is an urgent problem to be solved.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a task scheduling method and related equipment, which improve the response speed of the system, improve resource utilization, and reduce network load to at least a certain extent, thereby improving the overall performance and service quality of AI large model reasoning business.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a task scheduling method is provided, including: obtaining the amount of resources of an edge node and Z tasks to be scheduled, where Z is a positive integer; determining an allocation relationship based on the amount of resources of the edge node, where the allocation relationship is used to indicate the allocation of the Z tasks on the edge node and the central node; obtaining an initial population based on a genetic algorithm, where the initial population includes M initial codes, each initial code includes Z weights, and the Z weights correspond one-to-one to the Z tasks, where M is a positive integer; performing population iteration on the initial population based on a target fitness function and the allocation relationship to obtain a target code; and scheduling the Z tasks based on the target code and the amount of resources of the edge node.

[0008] In some embodiments, before obtaining the resource amount of the edge node and the Z tasks to be scheduled, the method also includes: obtaining the performance requirements of each task to be scheduled; if the performance requirement of the task to be scheduled is greater than a performance requirement threshold, allocating the task to be scheduled to the central node; wherein the performance requirements of the Z tasks to be scheduled are all less than or equal to the performance requirement threshold.

[0009] In some embodiments, the allocation relationship is determined based on the resource amount of the edge node, including: pre-scheduling the first P tasks of the Z tasks to be scheduled to the edge node, P is an integer less than Z; obtaining the remaining resources of the edge node after the P tasks are pre-scheduled to the edge node; pre-scheduling tasks among the ZP tasks to the edge node in sequence until the remaining resources of the edge node cannot meet the resource requirements of the tasks to be pre-scheduled, and pre-scheduling the tasks whose resource requirements cannot be met by the edge node to the central node; wherein the resource allocation principle of the tasks pre-scheduled to the edge node is the maximum-minimum fair allocation principle.

[0010] In some embodiments, the performing population iteration on the initial population based on the target fitness function and the allocation relationship to obtain the target code includes: sorting the target fitness function values ​​of the iterative codes in each iterative population in ascending order based on the target fitness function and the allocation relationship; selecting the iterative codes ranked in the top H to enter the next generation of population iteration, performing crossover and / or mutation operations on the remaining iterative codes, and normalizing the iterative codes after the crossover and / or mutation operations to generate a new generation of population for population iteration; and obtaining the target code when the iteration result meets the preset conditions.

[0011] In some embodiments, the preset conditions include: the number of population iterations is equal to a preset number; or the value of the target fitness function reaches a minimum.

[0012] In some embodiments, the Z tasks are scheduled according to the target code and the resource amount of the edge node, including: sorting the Z tasks according to the size of the weight in the target code; scheduling the first J tasks of the sorted Z tasks to the edge node, J is an integer less than Z; after the J tasks are scheduled to the edge node, obtaining the remaining resource amount of the edge node; scheduling the tasks of the ZJ tasks to the edge node in turn until the remaining resource amount of the edge node cannot meet the resource requirements of the tasks to be scheduled, and scheduling the tasks whose resource requirements cannot be met by the edge node to the central node; wherein, the resource allocation principle of the tasks scheduled to the edge node is the maximum-minimum fair allocation principle.

[0013] In some embodiments, the target fitness function is constructed in the following manner: constructing an edge node load balancing function based on the number of GPU cards of the edge node, the single-card computing power of the edge node, the multi-card parallel efficiency of the edge node, the computing power requirements of the task and the task scheduling decision; constructing a delay function based on the task queuing delay, task execution delay, task transmission delay and the maximum delay acceptable to the user of each task; determining the target fitness function based on the edge node load balancing function and / or the delay function.

[0014] In some embodiments, the constraints of the target fitness function are as follows: the sum of the task queuing delay, the task execution delay and the task transmission delay is less than the maximum delay acceptable to the user; the performance requirement of the task scheduled to the edge node is less than or equal to the inference accuracy supported by the edge node deployment model; the performance requirement of the task scheduled to the central node is less than or equal to the inference accuracy supported by the central node deployment model; the sum of the computing power requirements of all tasks scheduled to the edge node in the same time period is less than or equal to the resource amount of the edge node.

[0015] According to another aspect of the present disclosure, a task scheduling device is also provided, including: an acquisition module, used to acquire the resource amount of an edge node and Z tasks to be scheduled, where Z is a positive integer; a determination module, used to determine an allocation relationship based on the resource amount of the edge node, where the allocation relationship is used to indicate the pre-allocation status of the Z tasks on the edge node and the central node; a calculation module, used to calculate and obtain an initial population based on a genetic algorithm, where the initial population includes M initial codes, each initial code includes Z weights, and the Z weights correspond one-to-one to the Z tasks, where M is a positive integer; an iteration module, used to perform population iteration on the initial population based on a target fitness function and the allocation relationship to obtain a target code; and a scheduling module, used to schedule the Z tasks based on the target code and the resource amount of the edge node.

[0016] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned task scheduling methods by executing the executable instructions.

[0017] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the task scheduling method described in any one of the above is implemented.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including: a computer program or instructions, wherein when the computer program or instructions are executed by a processor, any one of the above-mentioned task scheduling methods is implemented.

[0019] In the embodiment of the present disclosure, the resource amount of the edge node and the Z tasks to be scheduled are first obtained, where Z is a positive integer. Then, according to the resource amount of the edge node, the allocation relationship is determined. The allocation relationship is used to indicate the pre-allocation of the Z tasks on the edge node and the central node, and the initial population is calculated based on the genetic algorithm. The initial population includes M initial codes, each initial code includes Z weights, and the Z weights correspond to the Z tasks one by one, where M is a positive integer. The initial population is iterated based on the target fitness function and the allocation relationship to obtain the target code, and the Z tasks are scheduled according to the target code and the resource amount of the edge node. The present disclosure determines the allocation relationship through the resource amount of the edge node and the demand for the task, and iteratively trains the weights corresponding to the task based on the genetic algorithm, the allocation relationship and the target fitness function to obtain the target code, and schedules the Z tasks according to the target code and the resource amount of the edge node, thereby improving the utilization of resources, reducing the network load, and improving the overall performance and service quality of the AI ​​large model reasoning business.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0022] Figure 1 A schematic diagram showing the structure of a task scheduling system in an embodiment of the present disclosure.

[0023] Figure 2 A schematic diagram showing the structure of a task scheduling system in another embodiment of the present disclosure.

[0024] Figure 3 A flow chart of a task scheduling method in an embodiment of the present disclosure is shown.

[0025] Figure 4 A flowchart of a task scheduling method in another embodiment of the present disclosure is shown.

[0026] Figure 5 A schematic diagram of a task scheduling device in an embodiment of the present disclosure is shown.

[0027] Figure 6 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown.

[0028] Figure 7 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0030] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0031] For ease of understanding, before introducing the embodiments of the present disclosure, several terms involved in the embodiments of the present disclosure are first explained as follows:

[0032] QoS (Quality of Service): Quality of Service is a measure of network service capabilities, used to describe the performance of a network when transmitting data. QoS indicators include bandwidth, delay, jitter, and packet loss rate. These indicators can ensure the priority and processing order of different types of traffic in the network, thereby meeting the performance requirements of different applications and services.

[0033] AI (Artificial Intelligence): AI is a technology that simulates human intelligence through computer systems, including learning, reasoning, planning, perception, and language understanding. AI technology is widely used in various fields, such as natural language processing, image recognition, speech recognition, autonomous driving, and intelligent recommendation.

[0034] Cloud-Edge Collaboration: A mode in which the central cloud (also known as the central node) and the edge cloud (also known as the edge node) jointly process tasks to improve system performance through resource allocation and scheduling.

[0035] Task Offloading: The process of transferring computing tasks from one node (such as an edge node) to another node (such as a central node) for execution.

[0036] Among the related technologies, cloud task offloading and resource scheduling issues have received extensive attention. Most studies have focused on offloading decisions and resource allocation in a cloud-edge collaborative environment from the aspects of task response delay and power consumption. The characteristics of AI reasoning tasks and the impact of the difference in capabilities between central nodes and edge nodes on reasoning accuracy have not been fully considered. Central nodes have powerful computing and storage capabilities and are suitable for deploying complex and high-precision AI models; while edge nodes provide faster response speeds in resource-constrained environments through model compression and pruning. How to effectively utilize these two resources and coordinate processing tasks in combination with model reasoning requirements remains a challenge.

[0037] Based on this, the present disclosure provides a task scheduling method, which is applied to edge computing or distributed cloud scenarios to realize large model reasoning task deployment and resource collaborative scheduling. For example, for application scenarios such as video surveillance, traffic management and environmental monitoring in smart cities, the present disclosure can efficiently perform real-time data analysis through edge computing and central node collaborative processing to meet the needs of low latency and high precision. The present disclosure determines the allocation relationship by the amount of resources of the edge node and the demand for the task, and iteratively trains the weights corresponding to the task based on the genetic algorithm, the allocation relationship and the target fitness function to obtain the target code, and schedules Z tasks according to the target code and the amount of resources of the edge node, thereby improving resource utilization, reducing network load, and improving the overall performance and service quality of AI large model reasoning business.

[0038] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0039] Figure 1 A schematic diagram of a task scheduling system structure in an embodiment of the present disclosure is shown. Figure 1 As shown, the system architecture includes a terminal device 101 , a network 102 and a network side device 103 .

[0040] The network 102 is a medium for providing a communication link between the terminal device 101 and the network-side device 103 , and may be a wired network or a wireless network.

[0041] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocols. The network is typically the Internet, but may also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec), etc. may be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.

[0042] Optionally, the terminal device in the embodiment of the present disclosure may also be referred to as UE (User Equipment). In a specific implementation, the terminal device may be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a personal digital assistant (Personal Digital Assistant, PDA), a mobile Internet device (Mobile Internet Device, MID), a wearable device (Wearable Device) or a vehicle-mounted device, etc. It should be noted that the specific type of the terminal device is not limited in the embodiment of the present invention.

[0043] The network side device may be a base station, a relay or an access point, etc. The base station may be a base station of 5G or later versions (for example, 5G NR NB), or a base station in other communication systems (for example, an eNB base station). It should be noted that the specific type of the network side device is not limited in the embodiments of the present disclosure.

[0044] For example, Figure 2As shown, the system architecture can include a cloud-edge collaborative network consisting of multiple wireless access points (APs) and one base station. Each wireless AP is equipped with an edge cloud server (edge ​​node) and is connected to the central cloud server (central node) through the base station.

[0045] Users can access the network through IoT devices and choose to offload tasks to local edge servers or central cloud servers based on their requirements for resources and performance.

[0046] Those skilled in the art will know that Figure 1 The number of terminals, networks, and network-side devices in the example is only illustrative, and any number of terminals, networks, and network-side devices may be provided according to actual needs. The embodiments of the present disclosure are not limited to this.

[0047] Under the above system architecture, a task scheduling method is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0048] In some embodiments, the task scheduling method provided in the embodiments of the present disclosure can be executed by a terminal device of the above-mentioned system architecture; in other embodiments, the task scheduling method provided in the embodiments of the present disclosure can be executed by a server in the above-mentioned system architecture; in other embodiments, the task scheduling method provided in the embodiments of the present disclosure can be implemented by the terminal device and the server in the above-mentioned system architecture through interaction.

[0049] Figure 3 A flowchart of a task scheduling method in an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the task scheduling method provided in the embodiment of the present disclosure may include the following S301 to S305.

[0050] S301, obtaining the resource amount of the edge node and Z tasks to be scheduled, where Z is a positive integer.

[0051] In the embodiment of the present disclosure, the resource amount of the edge node is used to indicate the size of the edge node resources. The task is a task to be offloaded to the edge node or the central node.

[0052] S302, determining an allocation relationship according to the resource amount of the edge node, where the allocation relationship is used to indicate the allocation status of the Z tasks on the edge node and the central node.

[0053] In the disclosed embodiment, Z tasks are pre-allocated according to the resource amount of the edge node. If the resource amount of the edge node meets the resource requirements of the tasks, as many tasks as possible are pre-allocated to the edge node until the edge node cannot meet the resource requirements, and the remaining tasks are pre-allocated to the central node.

[0054] The pre-allocation can provide a basis standard for the population iteration in S304, and the iteration is performed according to the allocation relationship, so as to facilitate the determination of the target code.

[0055] S303, obtain an initial population based on genetic algorithm calculation, the initial population includes M initial codes, each initial code includes Z weights, the Z weights correspond to the Z tasks one by one, and M is a positive integer.

[0056] In the disclosed embodiment, each task corresponds to a weight, and the weight is used to indicate the priority of task processing. In order to quickly train and find the optimal solution, the disclosed embodiment can set multiple codes in the initial population, and each code represents a priority ranking of Z tasks.

[0057] The disclosed embodiments do not limit how to obtain the initial population. Exemplarily, the tasks are sorted according to the arrival time, and the weight value is set for each task in order, and the sum of the weight values ​​of all task queues is 1. For example, the edge node has 5 tasks in total, and the weight value of the first task is a random number between [0, 1], and the value is 0.2; the weight value of the second task is a random number between [0, (1-0, 2)], and so on, until all tasks are assigned weights, and the weight sequence of each task is used as an initial code, and a total of M initial codes are generated as the initial population.

[0058] S304, performing population iteration on the initial population based on the target fitness function and the allocation relationship to obtain the target code.

[0059] In the disclosed embodiment, the target fitness function is a function that can take into account resource utilization and service quality while improving task processing efficiency. The target fitness function calculates the function value through the allocation relationship and the weight (coding) of each task. The population is iterated through the target fitness function, and the obtained target coding makes the result of unloading the task to the edge node or the central node optimal, while improving the task processing efficiency, taking into account resource utilization and service quality.

[0060] S305 , scheduling Z tasks according to the target coding and the resource amount of the edge node.

[0061] The disclosed implementation determines the allocation relationship through the resource amount of the edge node and the demand of the task, and iteratively trains the weights corresponding to the task based on the genetic algorithm, the allocation relationship and the target fitness function to obtain the target code, and schedules Z tasks according to the target code and the resource amount of the edge node, thereby improving resource utilization, reducing network load, and improving the overall performance and service quality of AI large model inference business.

[0062] The present disclosure comprehensively considers the computing power requirements, reasoning performance, latency and other multi-dimensional factors of the task to improve the task processing efficiency and service quality, and improve the resource utilization of the cloud-edge system. In order to better illustrate the present disclosure, the tasks, edge nodes and central nodes are described below.

[0063] like Figure 2 As shown in the figure, there is a cloud-edge collaborative network consisting of several wireless access points AP and one base station. Each AP is equipped with an edge node and is connected to the central node through the base station. The central node is represented by , Indicates a total of n edge nodes.

[0064] Central Node Can be represented as a four-tuple ,in Indicates the number of GPU (Graphics Processing Unit) cards. Indicates the computing power of a single card. Indicates the multi-card parallel efficiency, Indicates the inference accuracy supported by the model deployed on the central node.

[0065] Edge Nodes e It can also be represented as a four-tuple , where n eg Indicates the number of GPU cards, c eg Indicates the computing power of a single card, d eg Indicates the multi-card parallel efficiency, Indicates the inference accuracy supported by the model deployed on the edge node.

[0066] The interconnection transmission bandwidth between the central node and the edge node is B ce , the interconnection transmission bandwidth between the user access point and its edge node is B ue .

[0067] Task Can be recorded as a four-tuple in, Indicates the computing power required for the task, represents the amount of tasks that need to be processed by task offloading, represents the user's performance requirements for the reasoning task, represents the maximum delay acceptable to the user, i represents the i-th task, and j represents the j-th edge node. It should be noted that the task set in the network at each moment can be recorded as: .

[0068] Task The uninstall decision is expressed as ,when When Uninstall on the edge node, when When Unloading to the central node, the current moment all task unloading decisions can be expressed as .

[0069] Users access the network through IoT devices and can choose to offload tasks to local cloud edge nodes or central nodes based on their requirements for resources and performance.

[0070] The above describes the tasks, edge nodes, and central nodes. The following describes how the present disclosure performs scheduling through several exemplary embodiments.

[0071] In an exemplary embodiment, before obtaining the resource amount of the edge node and the Z tasks to be scheduled, the method provided by the embodiment of the present disclosure may further include the following steps A1 and A2.

[0072] Step A1: Obtain the performance requirements of each task to be scheduled.

[0073] In the disclosed embodiment, the performance requirement may be one or more of the user's performance requirement for the reasoning task and the computing power requirement. The computing power requirement may include one or more of the GPU card number requirement, the single card computing power requirement, and the multi-card parallel efficiency requirement.

[0074] Step A2: if the performance requirement of the task to be scheduled is greater than the performance requirement threshold, the task to be scheduled is allocated to the central node; wherein the performance requirements of the Z tasks to be scheduled are all less than or equal to the performance requirement threshold.

[0075] In the embodiment of the present disclosure, the performance requirement threshold can be set according to the actual application scenario and specific application experience, which is used to determine the tasks that must be executed through the central node. For example, when the performance requirement of the task is greater than the performance requirement threshold, the task is scheduled to the central node for execution.

[0076] Exemplarily, the edge node performs an initial screening of the collected tasks and divides them according to the performance requirements of the tasks (for example, the user's performance requirements for reasoning tasks). If the task has high requirements for reasoning performance and the edge node cannot meet the task performance requirements, the task is offloaded to the central node for deployment. The task is not added to the task queue of the edge node, that is, the task does not need to be prioritized in the queue, that is, it does not participate in the task scheduling of S301 to S305.

[0077] The present disclosure schedules tasks with high performance requirements to the central node, and schedules the remaining tasks (Z tasks to be scheduled) through the methods of S301 to S305, thereby achieving reasonable unified scheduling of tasks.

[0078] It should be noted that, based on the capability difference between central nodes and edge nodes, the present invention deploys models in a resource-constrained edge cloud environment by reducing the amount of parameters, which loses a certain performance but achieves a faster response speed. By deploying high-precision models in a central cloud environment with sufficient resources but far away from users, more accurate calculation results can be obtained at the cost of sacrificing transmission, and AI reasoning tasks can be offloaded by pooling the resources of the central node and edge nodes, collaborating with each other, and scheduling in a unified manner.

[0079] In another exemplary embodiment, determining the allocation relationship according to the resource amount of the edge node may include the following steps B1 to B3.

[0080] Step B1, pre-schedule the first P tasks among the Z tasks to be scheduled to the edge nodes, where P is an integer less than Z.

[0081] In the embodiment of the present disclosure, the value of P can be set according to the actual application scenario and application experience, for example, the value of P is any value between 2 and 50. For example, P=3.

[0082] Step B2, obtaining the remaining resources of the edge node after the P tasks are pre-scheduled to the edge node.

[0083] In the embodiments of the present disclosure, the embodiments of the present disclosure do not limit how to calculate the amount of resources of the edge node. For example, the amount of resources of the edge node can be the number of GPU cards of the edge node, the computing power of a single card of the edge node, and the multi-card parallel efficiency of the edge node. For another example, the amount of resources of the edge node can be the product of the number of GPU cards of the edge node, the computing power of a single card of the edge node, and the multi-card parallel efficiency of the edge node.

[0084] In the disclosed embodiment, the remaining resources of the edge node are obtained by subtracting the resources pre-occupied by P tasks from the total resources of the edge node.

[0085] Step B3, pre-schedule tasks among ZP tasks to edge nodes in turn until the remaining resources of the edge nodes cannot meet the resource requirements of the tasks to be pre-scheduled, and pre-schedule the tasks whose resource requirements cannot be met by the edge nodes to the central nodes; wherein the resource allocation principle for tasks pre-scheduled to edge nodes is the maximum-minimum fair allocation principle.

[0086] In the disclosed embodiment, the remaining ZP tasks are pre-scheduled to the edge nodes in sequence. After each task is pre-scheduled, it is calculated whether the remaining resources of the edge node meet the resource demand of the next task to be pre-scheduled. If so, the next task is pre-scheduled to the edge node. If not, all the remaining tasks are pre-scheduled to the central node, thereby obtaining tasks pre-scheduled to the edge nodes and tasks pre-scheduled to the central node.

[0087] It should be noted that pre-scheduling is not the implementation of task offloading, but a scheduling strategy assumed in order to calculate the optimal scheduling decision.

[0088] The disclosed embodiment provides a basis for population iteration by determining the allocation relationship, thereby facilitating rapid determination of the target code.

[0089] In yet another exemplary embodiment, population iteration is performed on the initial population based on the target fitness function and the allocation relationship to obtain the target code, including the following steps C1 to C3.

[0090] Step C1, based on the target fitness function and the allocation relationship, sort the target fitness function values ​​of the iterative codes in each iterative population in ascending order.

[0091] In the disclosed embodiment, the iterative coding is the coding after the population iteration of the initial coding. Each iterative coding corresponds to the priority of a group of tasks (task unloading order). It should be noted that the disclosed embodiment can re-determine the allocation relationship according to the resource amount of the edge node and the priority of each task after each population iteration, and calculate the target fitness function value according to the re-confirmed allocation relationship and the target fitness function.

[0092] Step C2, select the iterative codes ranked in the top H to enter the next generation of population iteration, perform crossover and / or mutation operations on the remaining iterative codes, and normalize the iterative codes after the crossover and / or mutation operations to generate a new generation of population for population iteration.

[0093] In the disclosed embodiment, there are M initial codes, and there are M corresponding target fitness function values. The iterative codes are sorted according to the target fitness function values, and the first H iterative codes with the smallest target fitness function values ​​are selected to enter the next generation of population iteration. The remaining iterative codes are crossover and / or mutation operations are performed, and the iterative codes after the crossover and / or mutation operations are normalized to generate a new generation of populations for population iteration. It should be noted that the smaller the target fitness function value, the better the scheduling strategy.

[0094] Step C3, when the iteration result meets the preset conditions, the target code is obtained.

[0095] In one embodiment, the preset condition may be that the number of population iterations is equal to the preset number. The specific value of the preset number is not limited in the embodiment of the disclosure and can be set according to requirements. For example, the number of population iterations is 50.

[0096] It should be noted that in step C2, whether it is the iterative coding entering the next generation population iteration or the iterative coding in the new generation population generated after crossover and / or mutation processing, the iteration ends after the population iteration number equals the preset number. The iterative coding with the smallest target fitness function value is selected as the target coding.

[0097] In another embodiment, the preset condition may be that the value of the target fitness function reaches a minimum. In the disclosed embodiment, the population is iterated multiple times until the target fitness function converges, so that the value of the target fitness function reaches a minimum, and the iteration is terminated. The iterative code with the minimum value of the target fitness function is selected as the target code.

[0098] The disclosed embodiment performs population iteration on the initial population based on the target fitness function and the allocation relationship to obtain the target code, and performs task scheduling based on the target code, which can improve resource utilization, reduce network load, and improve the overall performance and service quality of AI large model inference business.

[0099] In yet another exemplary embodiment, scheduling Z tasks according to target coding and the amount of resources of edge nodes may include the following steps D1 to D4.

[0100] Step D1, sort the Z tasks according to the weights in the target encoding.

[0101] In the disclosed embodiment, the weight is used to indicate the order of task offloading. The larger the weight, the higher the priority of offloading the corresponding task.

[0102] Step D2, scheduling the first J tasks among the sorted Z tasks to the edge node, where J is an integer less than Z.

[0103] The specific value of J can be set according to the actual application scenario and specific application experience. For example, the value of J is any value between 2 and 50. For example, J=2.

[0104] Step D3, after J tasks are scheduled to the edge node, the remaining resources of the edge node are obtained.

[0105] Step D4, schedule the tasks among ZJ tasks to the edge nodes in turn until the remaining resources of the edge nodes cannot meet the resource requirements of the tasks to be scheduled, and schedule the tasks whose resource requirements cannot be met by the edge nodes to the central nodes; wherein the resource allocation principle for the tasks scheduled to the edge nodes is the maximum-minimum fair allocation principle.

[0106] It should be noted that the scheduling method in step D4 is similar to the pre-scheduling method in step B3, which will not be described in detail here.

[0107] The disclosed embodiment schedules Z tasks according to the target coding and the resource amount of the edge node, thereby improving resource utilization, reducing network load, and improving the overall performance and service quality of the AI ​​large model inference business.

[0108] The target fitness function is described below through exemplary embodiments.

[0109] In an exemplary embodiment, the target fitness function is constructed in the following manner: constructing an edge node load balancing function based on the number of GPU cards of the edge node, the computing power of a single card of the edge node, the multi-card parallel efficiency of the edge node, the computing power requirements of the task and the task scheduling decision; constructing a delay function based on the task queuing delay, task execution delay, task transmission delay and the maximum delay acceptable to the user of each task; determining the target fitness function based on the edge node load balancing function and / or the delay function.

[0110] In the disclosed embodiment, the target fitness function may be an edge node load balancing function or a delay function. In addition, the target fitness function may be a function that minimizes the sum of the edge node load balancing function and the delay function.

[0111] In the disclosed embodiment, the target fitness function needs to meet the following constraints: the sum of the task queuing delay, the task execution delay and the task transmission delay is less than the maximum delay acceptable to the user; the performance requirement of the task scheduled to the edge node is less than or equal to the inference accuracy supported by the edge node deployment model; the performance requirement of the task scheduled to the central node is less than or equal to the inference accuracy supported by the central node deployment model; the sum of the computing power requirements of all tasks scheduled to the edge node in the same time period is less than or equal to the resource amount of the edge node.

[0112] In one embodiment, when the target fitness function is an edge node load balancing function, it is expressed by the following formula 1.

[0113]

[0114] in, represents the number of GPU cards of the jth edge node, Represents the computing power of a single card at the jth edge node, represents the multi-card parallel efficiency of the jth edge node, represents the computing power requirement of the task on the jth edge node, Represents a task scheduling decision (task offloading decision), when When , the task is offloaded to the jth edge node, When , the task is offloaded to the central node. i represents the i-th task.

[0115] In another embodiment, when the target fitness function is a delay function, it is expressed by the following formula 2.

[0116]

[0117] in, Indicates the maximum delay acceptable to the user. Indicates the task queuing delay, Indicates the task execution delay. represents the task transmission delay, T represents the time period, N represents the total number of tasks, and i represents the i-th task.

[0118] In yet another embodiment, when the target fitness function is a function that minimizes the sum of the edge node load balancing function and the delay function, it is expressed by the following formula 3.

[0119]

[0120] Among them, w1 and w2 represent coefficient parameter values. represents the number of GPU cards of the jth edge node, Represents the computing power of a single card at the jth edge node, represents the multi-card parallel efficiency of the jth edge node, represents the computing power requirement of the task on the jth edge node, represents the task scheduling decision, Indicates the maximum delay acceptable to the user. Indicates the task queuing delay, Indicates the task execution delay. represents the task transmission delay, T represents the time period, N represents the total number of tasks, and i represents the i-th task.

[0121] The following describes how to calculate task queuing delay, task execution delay, and task transmission delay.

[0122] The task processing process can be divided into three parts: task queuing, task execution and task transmission, and the task delay is calculated based on these three parts.

[0123] (1) Task queuing

[0124] Task The queuing time indicates that after the user publishes the task and accesses the edge node, the decision task is offloaded to the central cloud server or the local edge cloud server and enters the corresponding waiting queue Q c or Q e Task queuing delay It can be expressed by the following formula 4.

[0125]

[0126] Among them, t0 represents the task Unloading decision time, T0 represents the scheduling cycle time of the scheduling system, Indicates the task The number of scheduling rounds waiting in the queue.

[0127] (2) Task execution

[0128] The task execution time refers to the time it takes for a task to be processed at a node after it enters the queue at a central node or an edge node. The task execution delay can be expressed by the following formula 5.

[0129]

[0130] in, Indicates the computing power required for the task, Indicates the number of GPU cards allocated to the server (central node / edge node) when the task is offloaded. Indicates the computing power of a single card in the central node / edge node. Indicates the multi-card parallel efficiency of the central node / edge node, The representative task is offloaded to the central node. Represents the task offloaded to the edge node.

[0131] (3) Task communication

[0132] The task communication process consists of two parts: the process of user-published tasks being transmitted to the local edge node for execution and result return, and the transmission of data between the local edge node and the central node. The data transmission speed of user-published tasks being transmitted to the edge node is expressed by the following formula 6.

[0133]

[0134] Among them, B ue Indicates the interconnection transmission bandwidth between the user access point and its edge node; Indicates the i-th task at time t Transmitted to edge nodes via access points The transmission power of Indicates the i-th task at time t Transmitted to edge nodes via access points The channel gain between is the noise power; Indicates that at time t, except for the task The transmission power of other tasks accessing the edge node; Indicates that at time t, except for the task The channel gain of other tasks accessing the edge node.

[0135] Similarly, the transmission speed between the edge node and the central node is expressed by the following formula 7.

[0136]

[0137] Among them, B ce Indicates the interconnection transmission bandwidth between edge nodes and central nodes; Indicates the i-th task at time t Transmitted to the central node s through the edge node c The transmission power of Indicates the i-th task at time t Channel gain from edge nodes to central nodes; is the noise power; Indicates that at time t, except for the task The transmission power of other tasks transmitted from edge nodes to central nodes; Indicates that at time t, except for the task The channel gain of other tasks transmitted from edge nodes to central nodes.

[0138] Task transmission delay It can be expressed by the following formula 8.

[0139]

[0140] in, Indicates the amount of tasks that need to be processed by task offloading; Indicates the transmission speed between edge nodes and central nodes; Indicates the transmission speed of tasks published by users to edge nodes; The representative task is offloaded to the central node; Represents the task offloaded to the edge node.

[0141] The above describes how to calculate the task queuing delay, task execution delay, and task transmission delay. The following describes a type of target fitness function.

[0142] The embodiment of the present disclosure is to offload tasks to edge nodes or central nodes to reduce the user task feedback delay and improve the network resource utilization rate, wherein the user publishes the task The ratio of the feedback delay after task execution to the maximum delay acceptable to the user and the load balancing of edge node resources after task deployment in the system are taken as the objective function, then:

[0143]

[0144] Among them, w1 and w2 represent coefficient parameter values. represents the number of GPU cards of the jth edge node, Represents the computing power of a single card at the jth edge node, represents the multi-card parallel efficiency of the jth edge node, represents the computing power requirement of the task on the jth edge node, represents the task scheduling decision, Indicates the maximum delay acceptable to the user. Indicates the task queuing delay, Indicates the task execution delay. represents the task transmission delay, T represents the time period, N represents the total number of tasks, and i represents the i-th task. When , the i-th task is unloaded to the j-th edge node; when When , the i-th task is offloaded to the central node.

[0145] The constraints of P3 are as follows: ; ; ; ; .

[0146] Among them, C1 means that the execution time of the task is less than the maximum acceptable task delay when the user publishes it, C2 means that the task can be offloaded to the edge node or the central node; C3 and C4 respectively indicate that the reasoning performance of the edge node and the central node for the task offloading decision must meet the reasoning performance required by the task; C5 means that the computing demand offloaded to the edge node in the same time period cannot exceed the maximum available resources of the edge node.

[0147] The disclosed embodiment can not only prioritize tasks according to their acceptable latency, but also ensure that high-priority tasks can obtain the required resources in a timely manner, thereby improving the quality of user service. In addition, technologies such as resource recycling are considered to effectively utilize idle resources to improve the resource utilization of the entire cluster, providing solutions for the deployment and scheduling of AI reasoning tasks in the current distributed cloud.

[0148] The effects that can be achieved by the present disclosure are further described below through embodiments.

[0149] To achieve the above effects, the present disclosure proposes a task scheduling method based on the combination of the maximum-min fairness idea and a genetic algorithm. The steps of solving the above modeling problem are divided into three parts: edge node task queue sorting, task scheduling strategy iterative evolution, and maximum-minimum fair resource allocation.

[0150] In one embodiment, if Figure 4 As shown, the task scheduling method may include S401 to S407.

[0151] S401, define a task set and a node set.

[0152] In the embodiment of the present disclosure, the task information of the tasks in the task set includes the computing power requirements of each task, the amount of tasks to be processed by task offloading, the performance requirements of the user for the reasoning task, and the maximum latency acceptable to the user. The node set may include multiple edge nodes and at least one central node. , , Calculate each edge node For example, each edge node The amount of resources is How to define edge nodes and central nodes has been described in the above embodiments and will not be repeated here.

[0153] S402: Determine the allocation relationship according to the resource amount of the edge node. How to specifically determine the allocation relationship has been described in the above embodiment and will not be repeated here.

[0154] S403, initializing the first generation population based on the genetic algorithm.

[0155] In the disclosed embodiment, the edge node will initially screen the collected tasks and divide them according to the performance requirements of the tasks (such as task reasoning performance requirements). If the task has high requirements for reasoning performance and the edge node cannot meet the task performance requirements, the task needs to be offloaded to the central node for deployment. The task is not added to the task queue of the edge node, that is, the task does not need to be prioritized in the queue.

[0156] It should be noted that when the performance supported by the model deployment on the edge node can meet the performance requirements of the task reasoning, the task can be temporarily placed on the local edge node to calculate the queue order, and the task can be offloaded to the edge node or the central node according to the queue priority. At this edge node The priorities are sorted as task weights.

[0157] For example, tasks are sorted according to arrival time, weight values ​​are set for each task in order, and the sum of weight values ​​of all task queues is 1. There are 5 tasks in total. The weight value of the first task is a random number between [0, 1], which is 0.2; the weight value of the second task is a random number between [0, (1-0.2)], and so on, until all tasks are assigned weights. The weight sequence of each task is used as a code, and a total of p codes are generated as the initial population (generating multiple weight combinations).

[0158] S404, calculate the target fitness function value according to the target fitness function and the allocation relationship. Determine the first P tasks in the local server queue to be unloaded to the edge node according to the task requirements to implement the maximum-minimum fair distribution principle, unload the subsequent tasks of the queue to the central node, and calculate the target fitness function value of each code according to the allocation relationship.

[0159] S405, determine whether to end the iteration based on the target fitness function value. If the iteration ends, execute S406; if the iteration does not end, perform population selection, crossover and mutation, generate a new generation of population, and execute S404. Exemplarily, sort the target fitness function values ​​of each generation in ascending order, select the first H items to directly enter the next generation, and perform crossover and mutation operations on the remaining (pH) items. The crossover operation randomly selects a position in the code, and crosses two adjacent codes at the selected position. The mutation operation randomly selects a task in the code, and its weight value is randomly set to [0,1]. If the sum of the weight values ​​of a chromosome code after crossover and mutation is not 1, normalization is performed to generate a new generation of population. After the iteration meets the preset conditions, the iteration ends.

[0160] S406, selecting a code with the smallest target fitness function value from the population at the end of the iteration as the target code, that is, the priority sorting order of the tasks.

[0161] S407, performing task scheduling according to the target code and the resource amount of the edge node.

[0162] In the disclosed embodiment, for each edge node, task priority is sorted according to the above steps, the first J tasks are offloaded to the edge node, the remaining resources are recovered, the remaining resources of the edge node after the first J tasks are offloaded are calculated, and the remaining resources are compared in queue order to see whether they can support the requirements of subsequent tasks. If the subsequent tasks can be met, the task will continue to be offloaded to the local edge node, and the remaining tasks will be offloaded to the central cloud server.

[0163] The disclosed embodiment prioritizes tasks according to their acceptable latency, ensuring that high-priority tasks can obtain required resources in a timely manner to improve user service quality. It also considers technologies such as resource recycling to effectively utilize idle resources to improve resource utilization of the entire cluster, providing a solution for the deployment and scheduling of AI reasoning tasks in the current distributed cloud.

[0164] In another embodiment, a task scheduling method based on the combination of the Max-min fairness concept and a genetic algorithm is proposed, which can be applied to the AI ​​reasoning task deployment process. The specific embodiments are as follows.

[0165] In the scenario of smart cities, cameras and sensors in the city collect data in real time, and complex AI reasoning tasks such as video analysis, anomaly detection, and intelligent traffic management are required. These tasks require high computing resources and low-latency responses, and have different requirements for the accuracy of reasoning, so the tasks are offloaded to the cloud server for processing. Assume that the system includes 5 edge nodes and 1 central node. The edge nodes are distributed in different areas of the city, and the central node is located in the data center. Users access the network through IoT devices and offload and schedule tasks according to the needs of the tasks and resource conditions.

[0166] (1) Initialization: In the initialization phase, define the task set and cloud server set. Record the computing power requirements, transmission power, channel gain and other information of each task; record the number of GPU cards, single-card computing power, multi-card parallel efficiency and supported inference accuracy of each node.

[0167] The task set includes several AI reasoning tasks, such as video analysis tasks tsk1, tsk2, tsk3, tsk4, tsk5. Each task contains a real-time video stream and requires target recognition and behavior analysis.

[0168] The node set includes multiple edge nodes and one central node. The edge nodes are: se1, se2, se3, se4, se5. The central node is: sc1.

[0169] (2) Initial population setting: Each task queued on the edge node is sorted according to the arrival time, and the task weight is set. The sum of the weights of all tasks is 1. The initial population setting is as follows: Assume that there are 5 tasks on the edge node se1, and their weights are 0.2 according to the weight of the first task, which is a random number between [0, 1]; the weight of the second task is a random number between [0, (1-0, 2)], and so on, until all tasks are assigned weights, and the weight sequence of each task is used as a code. For example, the weight of tsk1 is 0.2, the weight of tsk2 is 0.3, the weight of tsk3 is 0.1, the weight of tsk4 is 0.25, and the weight of tsk5 is 0.15. The sum of the weights of all tasks is 1, and the generated weight sequence [0.2, 0.3, 0.1, 0.25, 0.15] is 1 code. Assume that a total of 10 such codes are generated.

[0170] (3) Fitness calculation: Determine the first few tasks to be unloaded to the edge node according to the task requirements to implement the maximum-minimum fair allocation scheme, calculate the remaining resources of the edge node after the first few tasks are unloaded, and compare the remaining resources in the queue order to see if they can support the subsequent task requirements. If the edge node resources are full, then unload the subsequent tasks in the queue to the central cloud. For example, in the queue of se1, the first three tasks are unloaded to the edge node, and the last two tasks are unloaded to the central node. And calculate the target fitness function value of each code based on the task unloading situation.

[0171] (4) Population selection: Sort the target fitness function values ​​of each generation in ascending order. Here, the first five items are selected to enter the next generation directly, and the remaining items are subjected to crossover and mutation operations to generate a new generation of population. Among them, the crossover operation example: the crossover operation randomly selects a position in the encoding, crosses two adjacent encodings at the selected position, and the mutation operation randomly selects a task in the encoding, and randomly takes its weight value as [0,1]. After the crossover and mutation, there is a situation where the sum of the weight values ​​of a chromosome encoding is not 1, and normalization is performed to generate a new generation of population. It should be noted that the maximum number of population evolution generations can be set to 50 generations, and the population iteration is repeated until the algorithm converges.

[0172] (5) Priority sorting: Select the code with the smallest target fitness function value in the final population as the target code. For example, select the code with the smallest target fitness function value as the priority sorting order. Assume that the final priority sorting is: [tsk2, tsk4, tsk1, tsk5, tsk3].

[0173] (6) Task scheduling: For each edge node, the task priority represented by the target code is sorted, and the maximum-minimum fairness idea is used to allocate resources, and the first h tasks are unloaded to the edge node. The remaining resources of the edge node after the first h tasks are unloaded are calculated, and the remaining resources are compared in queue order to see whether they can support the requirements of subsequent tasks. If they can be met, the task is unloaded to the edge node, and the remaining tasks are unloaded to the central node.

[0174] The specific task offloading process is as follows: offload h tasks to the edge node and calculate the remaining resources. Compare with the requirements of subsequent tasks. If the remaining resources of the edge node are sufficient, continue to offload tasks. If the resources of the edge node are insufficient, unload the remaining tasks to the central node. For example, set h to 2, then tsk2 and tsk4 are offloaded to the edge node se1. Calculate the remaining resources of se1. If the remaining resources are sufficient to support the needs of tsk1, unload tsk1 to the edge node se1, otherwise unload to the central node sc1, and so on.

[0175] (7) Task scheduling: Generate the final task scheduling decision based on step (6), execute task scheduling, and complete the task optimization scheduling process.

[0176] In the disclosed embodiment, a method for scheduling AI reasoning tasks that combines the maximum-minimum fairness concept with a genetic algorithm is proposed, which aims to optimize the allocation of resources of central nodes and edge nodes by combining multi-dimensional factors such as task computing power requirements, reasoning performance, and maximum latency. While improving task processing efficiency, it also takes into account resource utilization and service quality, achieving high efficiency and fairness in task scheduling.

[0177] In addition, the intelligent scheduling mechanism based on the maximum-minimum fair allocation starts from both the task and the cloud server, comprehensively considers the task requirements and server capabilities, and matches them while fully considering the task priority and resource load. Through the maximum-minimum fair allocation principle, the resource requirements of high-priority tasks are met first, while ensuring the balance of overall resource utilization, significantly improving the efficiency and fairness of task scheduling.

[0178] In addition, computation-guided scheduling based on the target fitness function can fully optimize the task offloading and resource allocation scheme. By comprehensively considering task delay, computing resource consumption and network load, the overall benefit of the system can be maximized. This method not only improves the user's task processing experience, but also optimizes the resource utilization of the cloud service provider to ensure maximum economic benefits.

[0179] Based on the same inventive concept, the present disclosure also provides a task scheduling device, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0180] Figure 5 A schematic diagram of a task scheduling device in an embodiment of the present disclosure is shown. Figure 5 As shown, the device includes: an acquisition module 51, a determination module 52, a calculation module 53, an iteration module 54 and a scheduling module 55. The acquisition module 51 can be used to acquire the resource amount of the edge node and the Z tasks to be scheduled, where Z is a positive integer; the determination module 52 can be used to determine the allocation relationship according to the resource amount of the edge node, and the allocation relationship is used to indicate the allocation of the Z tasks on the edge node and the central node; the calculation module 53 can be used to calculate the initial population based on the genetic algorithm, and the initial population includes M initial codes, each initial code includes Z weights, and the Z weights correspond to the Z tasks one by one, where M is a positive integer; the iteration module 54 can be used to perform population iteration on the initial population based on the target fitness function and the allocation relationship to obtain the target code; the scheduling module 55 can be used to schedule the Z tasks according to the target code and the resource amount of the edge node.

[0181] In one embodiment, before obtaining the resource amount of the edge node and the Z tasks to be scheduled, the acquisition module 51 can also be used to obtain the performance requirements of each task to be scheduled; if the performance requirement of the task to be scheduled is greater than the performance requirement threshold, the task to be scheduled is allocated to the central node; wherein the performance requirements of the Z tasks to be scheduled are all less than or equal to the performance requirement threshold.

[0182] In one embodiment, the determination module 52 can also be used to pre-schedule the first P tasks among the Z tasks to be scheduled to the edge node, where P is an integer less than Z; obtain the remaining resources of the edge node after the P tasks are pre-scheduled to the edge node; pre-schedule the tasks among the ZP tasks to the edge node in sequence until the remaining resources of the edge node cannot meet the resource requirements of the tasks to be pre-scheduled, and pre-schedule the tasks whose resource requirements cannot be met by the edge node to the central node; wherein the resource allocation principle for the tasks pre-scheduled to the edge node is the maximum-minimum fair allocation principle.

[0183] In one embodiment, the iteration module 54 can also be used to sort the target fitness function values ​​of the iterative codes in each iteration population in ascending order based on the target fitness function and the allocation relationship; select the iterative codes ranked in the top H to enter the next generation of population iteration, and perform crossover and / or mutation operations on the remaining iterative codes, and normalize the iterative codes after the crossover and / or mutation operations to generate a new generation of population for population iteration; when the iteration result meets the preset conditions, the target code is obtained.

[0184] In one embodiment, the preset conditions include: the number of population iterations is equal to the preset number; or the value of the target fitness function reaches the minimum.

[0185] In one embodiment, the scheduling module 55 can also be used to sort Z tasks according to the size of the weight in the target coding; schedule the first J tasks of the sorted Z tasks to the edge node, where J is an integer less than Z; after the J tasks are scheduled to the edge node, obtain the remaining resources of the edge node; schedule the tasks of the ZJ tasks to the edge node in turn until the remaining resources of the edge node cannot meet the resource requirements of the tasks to be scheduled, and schedule the tasks whose resource requirements cannot be met by the edge node to the central node; wherein the resource allocation principle of the tasks scheduled to the edge node is the maximum-minimum fair allocation principle.

[0186] In one embodiment, the target fitness function is constructed in the following manner: constructing an edge node load balancing function based on the number of GPU cards of the edge node, the computing power of a single card of the edge node, the multi-card parallel efficiency of the edge node, the computing power requirements of the task and the task scheduling decision; constructing a delay function based on the task queuing delay, task execution delay, task transmission delay and the maximum delay acceptable to the user of each task; determining the target fitness function based on the edge node load balancing function and / or the delay function.

[0187] In one embodiment, the constraints of the target fitness function are as follows: the sum of the task queuing delay, the task execution delay and the task transmission delay is less than the maximum delay acceptable to the user; the performance requirement of the task scheduled to the edge node is less than or equal to the inference accuracy supported by the edge node deployment model; the performance requirement of the task scheduled to the central node is less than or equal to the inference accuracy supported by the central node deployment model; the sum of the computing power requirements of all tasks scheduled to the edge node in the same time period is less than or equal to the resource amount of the edge node.

[0188] The task scheduling device disclosed in the embodiment of the present disclosure determines the allocation relationship through the resource amount of the edge node and the demand amount of the task, and iteratively trains the weights corresponding to the task based on the genetic algorithm, the allocation relationship and the target fitness function to obtain the target code, and schedules Z tasks according to the target code and the resource amount of the edge node, thereby improving resource utilization, reducing network load, and improving the overall performance and service quality of AI large model inference business.

[0189] It should be noted that the examples and application scenarios implemented by the modules in the above-mentioned device embodiment are the same as those of the corresponding steps in the method embodiment, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0190] Those skilled in the art will appreciate that various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0191] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present disclosure, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned task scheduling methods by executing the executable instructions. Since the principle of solving the problem in the electronic device embodiment is similar to that in the above-mentioned method embodiment, the implementation of the electronic device embodiment can refer to the implementation of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0192] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present disclosure is described. Figure 6 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0193] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).

[0194] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps described in the above “Exemplary Method” section of this specification according to various exemplary embodiments of the present disclosure.

[0195] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0196] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0197] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0198] The electronic device 600 may also communicate with one or more external devices 640 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0199] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0200] In the disclosed exemplary embodiments, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. Figure 7 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. Figure 7 As shown, the computer-readable storage medium 700 stores a program product capable of implementing the above method of the present disclosure.

[0201] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0202] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0203] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0204] In specific implementation, the program code for performing the operation of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0205] Based on the same inventive concept, a computer program product is also provided in the embodiment of the present disclosure, including a computer program product, including: a computer program or an instruction, wherein when the computer program or the instruction is executed by a processor, the task scheduling method of any one of the above method embodiments is implemented. Since the principle of solving the problem in the computer program product embodiment is similar to that in the above method embodiment, the implementation of the computer program product embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0206] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0207] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0208] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the example implementation modes described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation mode of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation mode of the present disclosure.

[0209] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A task scheduling method, characterized in that: include: Obtaining the performance requirements of each task to be scheduled, wherein the performance requirements refer to the requirements of the task to be scheduled for the inference accuracy supported by the model deployed on the node, and the node is an edge node or a central node; if the performance requirement of the task to be scheduled is greater than the performance requirement threshold, the task to be scheduled is allocated to the central node; wherein the performance requirements of the Z tasks to be scheduled are all less than or equal to the performance requirement threshold; Obtain the resource amount of the edge node and the Z tasks to be scheduled, where Z is a positive integer; Determine an allocation relationship according to the amount of resources of the edge node, wherein the allocation relationship is used to indicate the allocation of the Z tasks on the edge node and the central node; A primary population is obtained based on genetic algorithm calculation, wherein the primary population includes M initial codes, each initial code includes Z weights, the Z weights correspond to the Z tasks one by one, and M is a positive integer; Performing population iteration on the initial population based on the target fitness function and the allocation relationship to obtain a target code, wherein the target code represents the priority order of the Z tasks; Scheduling the Z tasks according to the target code and the amount of resources of the edge node; Wherein, the target fitness function is as follows: ； Among them, w1 and w2 represent coefficient parameter values. represents the number of GPU cards of the jth edge node, Represents the computing power of a single card at the jth edge node, represents the multi-card parallel efficiency of the jth edge node, represents the computing power requirement of the task on the jth edge node, represents the task scheduling decision, Indicates the maximum delay acceptable to the user. Indicates the task queuing delay, Indicates the task execution delay. represents the task transmission delay, T represents the time period, N represents the total number of tasks, and i represents the i-th task.

2. The task scheduling method according to claim 1, characterized in that: Determining an allocation relationship according to the resource amount of the edge node includes: Pre-scheduling the first P tasks of the Z tasks to be scheduled to the edge node, where P is an integer less than Z; Obtaining the remaining resources of the edge node after P tasks are pre-scheduled to the edge node; Pre-schedule tasks among ZP tasks to the edge node in sequence until the remaining resources of the edge node cannot meet the resource requirements of the tasks to be pre-scheduled, and pre-schedule the tasks whose resource requirements cannot be met by the edge node to the central node; wherein the resource allocation principle of the tasks pre-scheduled to the edge node is the maximum-minimum fair allocation principle.

3. The task scheduling method according to claim 1, characterized in that: The performing population iteration on the initial population based on the target fitness function and the allocation relationship to obtain the target code includes: Based on the target fitness function and the allocation relationship, the target fitness function values ​​of the iterative codes in each iterative population are sorted in ascending order; The iterative codes ranked in the top H are selected to enter the next generation of population iteration, and the remaining iterative codes are crossover and / or mutation operations are performed, and the iterative codes after the crossover and / or mutation operations are normalized to generate a new generation of population for population iteration; When the iteration result meets the preset conditions, the target code is obtained.

4. The task scheduling method according to claim 3, characterized in that: The preset conditions include: The number of population iterations is equal to a preset number; or, the value of the target fitness function reaches a minimum.

5. The task scheduling method according to claim 1, characterized in that: Scheduling the Z tasks according to the target code and the amount of resources of the edge node includes: Sort the Z tasks according to the weights in the target encoding; Schedule the first J tasks among the sorted Z tasks to the edge node, where J is an integer less than Z; After J tasks are scheduled to the edge node, obtaining the remaining resources of the edge node; Schedule tasks among ZJ tasks to the edge node in turn until the remaining resources of the edge node cannot meet the resource requirements of the tasks to be scheduled, and schedule the tasks whose resource requirements cannot be met by the edge node to the central node; wherein the resource allocation principle for the tasks scheduled to the edge node is the maximum-minimum fair allocation principle.

6. The task scheduling method according to claim 1, characterized in that: The target fitness function is constructed in the following way: Construct an edge node load balancing function based on the number of GPU cards of the edge node, the computing power of a single card of the edge node, the multi-card parallel efficiency of the edge node, the computing power requirements of the task, and the task scheduling decision; Construct a delay function based on the task queuing delay, task execution delay, task transmission delay and the maximum delay acceptable to the user for each task; Based on the edge node load balancing function and / or the delay function, a target fitness function is determined.

7. The task scheduling method according to claim 6, characterized in that: The constraints of the objective fitness function are as follows: The sum of the task queuing delay, the task execution delay and the task transmission delay is less than the maximum delay acceptable to the user; The performance requirement of the task scheduled to the edge node is less than or equal to the reasoning accuracy supported by the edge node deployment model; The performance requirement of the task scheduled to the central node is less than or equal to the reasoning accuracy supported by the deployment model of the central node; The sum of computing power requirements of all tasks scheduled to the edge node in the same time period is less than or equal to the resource amount of the edge node.

8. A task scheduling device, characterized in that: include: An acquisition module is used to acquire the performance requirements of each task to be scheduled, wherein the performance requirement refers to the requirement of the task to be scheduled for the inference accuracy supported by the model deployed on the node, and the node is an edge node or a central node; if the performance requirement of the task to be scheduled is greater than the performance requirement threshold, the task to be scheduled is allocated to the central node; wherein the performance requirements of the Z tasks to be scheduled are all less than or equal to the performance requirement threshold; and the resource amount of the edge node and the Z tasks to be scheduled are acquired, where Z is a positive integer; A determination module, used to determine an allocation relationship according to the amount of resources of the edge node, wherein the allocation relationship is used to indicate the allocation of the Z tasks on the edge node and the central node; A calculation module, used for calculating an initial population based on a genetic algorithm, wherein the initial population includes M initial codes, each initial code includes Z weights, the Z weights correspond to the Z tasks one by one, and M is a positive integer; An iteration module, configured to perform population iteration on the initial population based on a target fitness function and the allocation relationship to obtain a target code, wherein the target code represents a priority order of the Z tasks; A scheduling module, used for scheduling the Z tasks according to the target code and the resource amount of the edge node; Wherein, the target fitness function is as follows: ； Among them, w1 and w2 represent coefficient parameter values. represents the number of GPU cards of the jth edge node, Represents the computing power of a single card at the jth edge node, represents the multi-card parallel efficiency of the jth edge node, represents the computing power requirement of the task on the jth edge node, represents the task scheduling decision, Indicates the maximum delay acceptable to the user. Indicates the task queuing delay, Indicates the task execution delay. represents the task transmission delay, T represents the time period, N represents the total number of tasks, and i represents the i-th task.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the task scheduling method described in any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the task scheduling method described in any one of claims 1 to 7 is implemented.

11. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the task scheduling method described in any one of claims 1 to 7.

Images