Task execution method, device, storage medium and electronic device

By collecting performance data and total load information on multiple AI hardware units and determining the target AI hardware units, the problem of unreasonable allocation and scheduling of AI computing resources is solved, and the rational allocation of resources and rapid execution of models is achieved.

CN111831452BActive Publication Date: 2024-12-06ZHEJIANG DAHUA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010713701.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-22
Publication Date
2024-12-06
Estimated Expiration
2040-07-22

AI Technical Summary

Technical Problem

In the prior art, the allocation and scheduling of AI computing resources is unreasonable, and the static model parameters cannot reflect the model performance in the actual operating environment, resulting in unreasonable resource allocation.

Method used

By running the target model on multiple AI hardware units, collecting performance data and total load information, determining the target AI hardware unit that is most suitable for performing tasks, and scheduling and allocating resources based on the operating performance parameters in the actual environment.

Benefits of technology

It realizes reasonable allocation of AI computing resources, improves the utilization rate of computing resources, shortens the model execution time, and meets the needs of different algorithm businesses.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111831452B_ABST
    Figure CN111831452B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a task execution method, device, storage medium and electronic device, wherein the method comprises: based on the performance data of the target model when it runs on at least two artificial intelligence AI hardware units and the total load of the at least two AI hardware units, determining a target AI hardware unit for executing the target task from the at least two AI hardware units; and executing the target task using the target AI hardware unit. Through the present invention, the problem of unreasonable allocation and scheduling of AI computing resources existing in the related art is solved, and the effect of reasonable allocation and scheduling of AI computing resources is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communications, and in particular, to a task execution method, device, storage medium and electronic device. Background Art

[0002] There are multiple AI computing units in edge computing devices. Different models can be assigned to designated computing units for execution according to the complexity of the models. Models with high complexity are executed on high-performance hardware, and models with low complexity are executed on low-performance hardware. Different models can be automatically and flexibly assigned to different computing units.

[0003] In the related art, the input of the scheduling strategy is a static model parameter, rather than a real model performance parameter in an actual operating environment.

[0004] From this, we can see that there are problems with unreasonable allocation and scheduling of AI computing resources in related technologies.

[0005] With respect to the above-mentioned problems existing in the related technologies, no effective solution has been proposed yet. Summary of the invention

[0006] Embodiments of the present invention provide a task execution method, device, storage medium, and electronic device to at least solve the problem of unreasonable allocation and scheduling of AI computing resources existing in the related art.

[0007] According to one embodiment of the present invention, there is provided a task execution method, comprising: determining a target AI hardware unit for executing a target task from the at least two AI hardware units based on performance data of a target model when running on at least two artificial intelligence (AI) hardware units and the total load of the at least two AI hardware units; and executing the target task using the target AI hardware unit.

[0008] According to another embodiment of the present invention, there is provided a task execution device, comprising: a first determination module, used to determine a target AI hardware unit for executing a target task from at least two AI hardware units based on performance data of a target model when running on the at least two AI hardware units and the total load of the at least two AI hardware units; and an execution module, used to execute the target task using the target AI hardware unit.

[0009] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0010] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0011] Through the present invention, according to the performance data and the total load of the target model when running on multiple AI hardware units, the target AI hardware unit for executing the target task is determined among the multiple AI hardware units, and the target AI hardware unit is used to execute the target task. Since different computing resources are scheduled and allocated for the model according to the running performance parameters of the target model in the actual environment and the total load of each AI resource, it is possible to solve the problem of unreasonable allocation and scheduling of AI computing resources existing in the related art, and achieve the effect of reasonable allocation and scheduling of AI computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a hardware structure block diagram of a mobile terminal of a task execution method according to an embodiment of the present invention;

[0013] Figure 2 is a flowchart of a task execution method according to an embodiment of the present invention;

[0014] Figure 3 is a flow chart of the initial stage of the task execution method model according to a specific embodiment of the present invention;

[0015] Figure 4 is a flow chart of the actual operation phase of the task execution method model according to a specific embodiment of the present invention;

[0016] Figure 5 is a structural frame of a task execution device according to an embodiment of the present invention Figure 1 ;

[0017] Figure 6 is a structural frame of a task execution device according to an embodiment of the present invention Figure 2 . DETAILED DESCRIPTION

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0020] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1FIG. 1 is a hardware structure diagram of a mobile terminal of a task execution method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0021] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the task execution method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0023] In this embodiment, a task execution method is provided. Figure 2 is a flowchart of a task execution method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0024] Step S202, determining a target AI hardware unit for executing a target task from the at least two AI hardware units based on performance data of the target model when running on the at least two AI hardware units and the total load of the at least two AI hardware units;

[0025] Step S204: Utilize the target AI hardware unit to execute the target task.

[0026] In the above embodiment, the target model may be an algorithm model, etc., and the performance data may include the number of cycles and time consumed actually executed by the AI ​​hardware unit. In order to ensure the correctness and robustness of the strategy input, multiple tests may be repeated to obtain the average value.

[0027] Optionally, the execution subject of the above steps can be a processor, or other devices with similar processing capabilities, or a machine that at least integrates a data processing device, wherein the data processing device may include terminals such as computers and mobile phones, but is not limited to this.

[0028] Through the present invention, according to the performance data and the total load of the target model when running on multiple AI hardware units, the target AI hardware unit for executing the target task is determined among the multiple AI hardware units, and the target AI hardware unit is used to execute the target task. Since different computing resources are scheduled and allocated for the model according to the running performance parameters of the target model in the actual environment and the total load of each AI resource, it is possible to solve the problem of unreasonable allocation and scheduling of AI computing resources existing in the related art, and achieve the effect of reasonable allocation and scheduling of AI computing resources.

[0029] In an exemplary embodiment, before determining the target AI hardware unit for executing the target task from the at least two AI hardware units based on the performance data of the target model when running on at least two artificial intelligence AI hardware units and the total load of the at least two AI hardware units, the method further includes: running the target model on the at least two AI hardware units in sequence; and determining the performance data of the target model when running on the at least two AI hardware units respectively based on the running results. In this embodiment, assuming that there are three AI hardware acceleration units A, B, and C, the target model is run when the AI ​​hardware acceleration units A, B, and C respectively, and the performance data of the target model when running on the AI ​​hardware acceleration units A, B, and C is determined.

[0030] In an exemplary embodiment, based on the performance data of the target model when it runs on at least two artificial intelligence AI hardware units and the total load of the at least two AI hardware units, determining the target AI hardware unit for executing the target task from the at least two AI hardware units includes: determining the load sum corresponding to each AI hardware unit included in the at least two AI hardware units based on the following method: determining the performance data of the target model when it runs on the first AI hardware unit and the total load of the first AI hardware unit, and obtaining the load sum of the first AI hardware unit by summing the performance data of the target model when it runs on the first AI hardware unit and the total load of the first AI hardware unit, wherein the first AI hardware unit is any AI hardware unit included in the at least two AI hardware units; determining the AI ​​hardware unit with the smallest load sum in the at least two AI hardware units as the target AI hardware unit. In this embodiment, the model performance data can be added to the total load of each corresponding hardware unit to obtain the hardware unit with the smallest total load after addition, and the model reasoning task can be placed in the task list to wait for execution completion. Allocating and scheduling all AI computing units according to the load balancing strategy can maximize the utilization of each computing unit, and make the model execute as quickly as possible, shortening the waiting time for model execution.

[0031] In an exemplary embodiment, determining the total load of the first AI hardware unit includes: determining the priority of the pre-configured target model; determining the total load of the model tasks to be executed of the first AI hardware unit corresponding to the priority of the target model as the total load of the first AI hardware unit. In this embodiment, the priority can be configured and issued during the model initialization phase, and according to the model priority, the total load (for example, total time consumption or total number of cycles) of the model tasks to be executed of all different hardware units at this priority is counted, and the total load is determined as the total load of the hardware unit. Among them, the priority of executing the model task can be a manually configured priority. Using the model priority strategy, according to the algorithm business, the algorithms with important real-time requirements are executed quickly and have high priority, while the algorithms with low real-time requirements are executed slowly and have low priority.

[0032] In an exemplary embodiment, the performance data includes the number of cycles and / or the time taken; the total load includes the number of cycles and / or the time taken. In this embodiment, the performance data of the hardware unit may be the number of cycles and / or the time taken when the target model is run, and the total load may be the sum of the number of cycles and / or the time taken when multiple target models are run on the hardware unit.

[0033] The following describes how to perform the task in conjunction with a specific implementation method:

[0034] Figure 3is a flowchart of the initial stage of the task execution method model according to a specific embodiment of the present invention, such as Figure 3 As shown, the process includes:

[0035] Step S302, during the algorithm model initialization phase, all existing different AI computing unit models are loaded.

[0036] Step S304, actually run and obtain the performance data of the model on each different AI computing unit and save it as the input of the scheduling strategy when the subsequent algorithm model is actually run. When obtaining the model performance data, it is best to obtain the number of cycles or time-consuming data actually executed by the hardware to ensure the correctness of the input of the strategy; in addition, for robustness, the test can be repeated multiple times to obtain the average value.

[0037] Step S306, save the corresponding hardware data and priority, the specific storage location is not limited, as long as it can be obtained. Among them, the priority is a user-defined priority, which refers to the priority of the task.

[0038] Step S308, determine whether all hardware units have been traversed. If the determination result is yes, execute step S310; if the determination result is no, execute step S302.

[0039] Step S310, end the initialization process.

[0040] Figure 4 is a flowchart of the actual operation phase of the task execution method model according to a specific embodiment of the present invention, such as Figure 4 As shown, the process includes:

[0041] Step S402, obtaining the algorithm model to be executed.

[0042] Step S404, obtaining the priority of the algorithm model to be executed, wherein the priority is configured and issued during the model initialization phase.

[0043] Step S406: According to the model priority, the total load (total time or total number of cycles) of the model tasks to be executed of all different hardware units at the priority level is counted.

[0044] Step S408: Add the model performance data to the total load of each corresponding hardware unit to obtain the hardware unit with the smallest total load after the addition.

[0045] Step S410, putting the model reasoning task into the task list and waiting for completion of execution.

[0046] Step S412, simultaneously updating the added load data to the hardware unit.

[0047] Step S414, wait for the execution to complete and obtain the result.

[0048] In the aforementioned embodiment, the computing units for model execution are allocated according to statistics based on the performance data of the model during actual runtime, and statistics based on the performance parameters of the model actually running are used as the input basis of the strategy. Scheduling is based more on actual conditions rather than pre-setting scheduling based on static data, so as to achieve the effect of reasonable allocation and scheduling of AI computing resources; all AI computing resources are allocated according to the load balancing strategy, and all AI computing units are allocated and scheduled according to the load balancing strategy, so that each computing unit can be utilized to the greatest extent, and the model can be executed as quickly as possible, shortening the waiting time for model execution; there is a priority strategy for model execution to meet the needs of different algorithm services. The model priority strategy can be used to determine the priority of the algorithm service, and the priority of the algorithm with important real-time requirements can be high. The priority of the algorithm with low real-time requirements can be low.

[0049] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0050] In this embodiment, a task execution device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0051] Figure 5 is a structural frame of a task execution device according to an embodiment of the present invention Figure 1 ,like Figure 5 As shown, the device comprises:

[0052] A first determination module 52 is used to determine a target AI hardware unit for executing a target task from the at least two AI hardware units based on performance data of the target model when running on the at least two AI hardware units and the total load of the at least two AI hardware units;

[0053] The execution module 54 is used to execute the target task using the target AI hardware unit.

[0054] Figure 6 is a structural frame of a task execution device according to an embodiment of the present invention Figure 2 ,like Figure 6 As shown, the device includes Figure 5 All modules shown, plus:

[0055] An operating module 62 is configured to, based on performance data of the target model when running on at least two artificial intelligence AI hardware units and a total load of the at least two AI hardware units, sequentially operate the target model on the at least two AI hardware units before determining a target AI hardware unit for executing a target task from the at least two AI hardware units;

[0056] The second determination module 64 is used to determine the performance data of the target model when it is run on the at least two AI hardware units respectively based on the running results.

[0057] In an exemplary embodiment, the first determination module 52 includes: a first determination unit, used to determine the load sum corresponding to each AI hardware unit included in the at least two AI hardware units based on the following method: determining the performance data when the target model is running on the first AI hardware unit and the total load of the first AI hardware unit, and obtaining the load sum of the first AI hardware unit by summing the performance data when the target model is running on the first AI hardware unit and the total load of the first AI hardware unit, wherein the first AI hardware unit is any AI hardware unit included in the at least two AI hardware units; a second determination unit, used to determine the AI ​​hardware unit with the smallest load sum among the at least two AI hardware units as the target AI hardware unit.

[0058] In an exemplary embodiment, the first determination unit may determine the total load of the first AI hardware unit in the following manner: determine the priority of the pre-configured target model; and determine the total load of the model tasks to be executed of the first AI hardware unit corresponding to the priority of the target model as the total load of the first AI hardware unit.

[0059] In an exemplary embodiment, the performance data includes the number of cycles and / or the number of time consumption; the total load includes the number of cycles and / or the number of time consumption.

[0060] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0061] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0062] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0063] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0064] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0065] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0066] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A task execution method, characterized in that: include: Determine a target AI hardware unit for executing the target task from the at least two AI hardware units based on performance data of the target model when running on the at least two AI hardware units and a total load of each AI hardware unit included in the at least two AI hardware units; Utilizing the target AI hardware unit to perform the target task; Among them, the total load of each of the AI ​​hardware units is determined by: determining the priority of the pre-configured target model; and determining the total load of the model tasks to be executed corresponding to the priority of the target model of each of the AI ​​hardware units as the total load of each of the AI ​​hardware units.

2. The method according to claim 1, characterized in that: Before determining a target AI hardware unit for executing the target task from the at least two AI hardware units based on performance data of the target model when running on the at least two AI hardware units and the total load of the at least two AI hardware units, the method further includes: Running the target model on the at least two AI hardware units in sequence; Based on the running results, performance data of the target model when running on the at least two AI hardware units respectively is determined.

3. The method according to claim 1, characterized in that: Based on performance data of the target model when running on at least two artificial intelligence AI hardware units and the total load of the at least two AI hardware units, determining a target AI hardware unit for executing the target task from the at least two AI hardware units includes: Determining the sum of loads corresponding to each AI hardware unit included in the at least two AI hardware units based on the following method: determining performance data when the target model is running on a first AI hardware unit and a total load of the first AI hardware unit, and obtaining the sum of loads of the first AI hardware unit by summing the performance data when the target model is running on the first AI hardware unit and the total load of the first AI hardware unit, wherein the first AI hardware unit is any AI hardware unit included in the at least two AI hardware units; The AI ​​hardware unit with the smallest load among the at least two AI hardware units is determined as the target AI hardware unit.

4. The method according to any one of claims 1 to 3, characterized in that: The performance data includes the number of cycles and / or the time taken; The total load includes the number of cycles and / or the time consumed.

5. A task execution device, characterized in that: include: A first determination module is used to determine a target AI hardware unit for executing a target task from the at least two AI hardware units based on performance data of a target model when running on the at least two AI hardware units and a total load of each AI hardware unit included in the at least two AI hardware units; An execution module, configured to execute the target task using the target AI hardware unit; Among them, the first determination module is used to determine the total load of each of the AI ​​hardware units in the following manner: determine the priority of the pre-configured target model; and determine the total load of the model tasks to be executed corresponding to the priority of the target model of each of the AI ​​hardware units as the total load of each of the AI ​​hardware units.

6. The device according to claim 5, characterized in that Also includes: A running module, configured to run the target model on at least two artificial intelligence (AI) hardware units in sequence before determining a target AI hardware unit for executing a target task from the at least two AI hardware units based on performance data of the target model when the target model is run on the at least two AI hardware units and a total load of the at least two AI hardware units; The second determination module is used to determine the performance data of the target model when it is run on the at least two AI hardware units respectively based on the running results.

7. The device according to claim 5, characterized in that The first determining module comprises: A first determining unit is configured to determine a load sum corresponding to each AI hardware unit included in the at least two AI hardware units based on the following method: determining performance data when the target model is running on the first AI hardware unit and a total load of the first AI hardware unit, and obtaining a load sum of the first AI hardware unit by summing the performance data when the target model is running on the first AI hardware unit and the total load of the first AI hardware unit, wherein the first AI hardware unit is any AI hardware unit included in the at least two AI hardware units; The second determining unit is configured to determine the AI ​​hardware unit with the smallest load among the at least two AI hardware units as the target AI hardware unit.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 4 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.