Task scheduling method, device and computer equipment based on multi-operator
By obtaining the processing time and dependencies of operators in the target program and generating a schedule table, the problem of poor generalization of task scheduling in the existing technology is solved, and load balancing and task scheduling adapted to different application scenarios is realized.
Patent Information
- Application Number
- CN202411027246.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-30
AI Technical Summary
In the prior art, the lock-free centralized task scheduling mechanism based on the processor architecture cannot adapt to different application scenarios, resulting in poor generalization and difficulty in achieving load balancing.
By obtaining multiple processing times for each operator in the target program, determining the processing unit for parallel operation, and generating an operator schedule based on the number of threads, number of operators and operator dependency tables, optimizing task scheduling, and avoiding utilization Processor characteristics, adapted to different application scenarios.
It improves the universality of task scheduling, realizes load balancing, and ensures the effectiveness and efficiency of task scheduling in different application scenarios.
Smart Images

Figure CN118550674B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a multi-operator based task scheduling method, apparatus and computer equipment. Background Art
[0002] With the rise of modular programming, new solutions that meet current business needs can be obtained by adding different operators for combination. However, the modularization and generalization of operators not only help improve the efficiency of creating new solutions, but also change the basic unit of program operation from functions to more complex operators, increasing the difficulty of parallel and refined performance tuning. During program operation, it is difficult to achieve load balancing between various computing resources.
[0003] In order to solve the above problems, in the existing technical solutions, the characteristics of specific processors are analyzed and obtained, and based on the unique characteristics of the processor architecture, a lock-free centralized task scheduling mechanism is designed to optimize the scheduling overhead. However, the above solutions are strongly related to the characteristics of hardware devices and cannot be adapted to different application scenarios, resulting in poor versatility.
[0004] Currently, no effective solution has been proposed to address the problem that related technologies cannot adapt to different application scenarios, resulting in poor versatility. Summary of the invention
[0005] In this embodiment, a multi-operator based task scheduling method, apparatus and computer device are provided to solve the problem that the related technology cannot adapt to different application scenarios, resulting in poor versatility.
[0006] In a first aspect, a multi-operator-based task scheduling method is provided in this embodiment, and the method includes:
[0007] Based on the preliminary running result of the target program, a plurality of first processing times corresponding to each operator in the target program are obtained; the first processing time is the time taken by different processing units to run each part of the task in the operator;
[0008] Determining a target processing unit among the processing units for executing the operators in parallel;
[0009] An operator scheduling table corresponding to each of the operators is generated according to the number of threads corresponding to the target processing unit, the number of each of the operators, a plurality of first processing times corresponding to each of the operators, and a preset operator dependency table.
[0010] In some embodiments, generating an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of each operator, a plurality of first processing times corresponding to each operator, and a preset operator dependency table includes:
[0011] Each time, according to the number of threads corresponding to the target processing unit and the number of each operator, a number of target operators are selected from each operator;
[0012] Based on the operator dependency table, multiple target operators are added to each thread in different arrangements for execution;
[0013] Determine the running time corresponding to each of the arrangement modes according to the multiple first processing times corresponding to each of the operators;
[0014] The arrangement corresponding to the minimum running time is recorded in the preset operator scheduling table.
[0015] In some embodiments, adding the plurality of target operators to each thread in different arrangements based on the operator dependency table includes:
[0016] Adding a plurality of the target operators to each thread in different arrangements;
[0017] Based on the operator dependency table, determining whether the predecessor operator of each target operator on the thread has completed execution;
[0018] The threads are run according to the determination result.
[0019] In some embodiments, the first processing times are respectively the pre-processing time of the central processing unit, the post-processing time of the central processing unit and the processing time of the neural network processing unit.
[0020] In some of the embodiments, after generating the operator scheduling table corresponding to each of the operators, the method further includes:
[0021] Based on a pre-built runtime model, processing each thread in the operator scheduling table to obtain the runtime of each thread;
[0022] The load balancing degree corresponding to the operator scheduling table is evaluated according to the running time of each of the threads.
[0023] In some of the embodiments, before processing each thread in the operator scheduling table based on the pre-built runtime model, the process further includes:
[0024] Determine the wall time corresponding to the operator according to a plurality of second processing times corresponding to each operator; the second processing time includes the running time and waiting time of different processing units processing the operator;
[0025] Based on each of the wall times, a corresponding runtime model is constructed; the runtime model is used to determine the time consumption of executing all the operators on a single thread.
[0026] In some embodiments, the second processing time includes a first process scheduling time of a central processing unit and a second process scheduling time of a neural network processing unit; wherein
[0027] The first process scheduling time and the second process scheduling time are both positively correlated with thread scheduling times; the thread scheduling times include thread preemption times and thread active yield times.
[0028] In a second aspect, a multi-operator-based task scheduling device is provided in this embodiment, and the device includes: a measurement module, a determination module, and a scheduling module;
[0029] The measurement module is used to obtain a plurality of first processing times corresponding to each operator in the target program based on the preliminary running result of the target program; the first processing time is the time taken by different processing units to run each part of the task in the operator;
[0030] The determination module is used to determine the target processing unit for parallel execution of the operator in each of the processing units;
[0031] The scheduling module is used to generate an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of each operator, multiple first processing times corresponding to each operator, and a preset operator dependency table.
[0032] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the multi-operator-based task scheduling method described in the first aspect is implemented.
[0033] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the multi-operator-based task scheduling method described in the first aspect is implemented.
[0034] Compared with the related art, the multi-operator based task scheduling method, apparatus and computer equipment provided in this embodiment obtain multiple first processing times corresponding to each operator in the target program based on the preliminary running results of the target program, where the first processing time is the time for different processing units to run each part of the task in the operator; determine the target processing unit for running the operator in parallel in each processing unit; generate an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of operators, the multiple first processing times corresponding to each operator and the preset operator dependency table, which solves the problem of being unable to adapt to different application scenarios and resulting in poor versatility, and avoids using processor characteristics for task scheduling to adapt to different application scenarios and improve versatility.
[0035] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0037] Figure 1 It is a hardware structure block diagram of a terminal device of a multi-operator-based task scheduling method provided in an embodiment of the present application;
[0038] Figure 2 is a flowchart of a multi-operator based task scheduling method provided in one embodiment of the present application;
[0039] Figure 3 is a schematic diagram of different arrangements in operator scheduling analysis provided in an embodiment of the present application;
[0040] Figure 4 is a schematic diagram of an operator scheduling analysis process provided by an embodiment of the present application;
[0041] Figure 5 is a schematic diagram of an operator dependency table provided in an embodiment of the present application;
[0042] Figure 6 is a schematic diagram of the processing time distribution of different operators provided in an embodiment of the present application;
[0043] Figure 7 is a flowchart of a multi-operator based task scheduling method provided in a preferred embodiment of the present application;
[0044] Figure 8 It is a structural block diagram of a multi-operator based task scheduling device provided in one embodiment of the present application.
[0045] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, measurement module; 20, determination module; 30, scheduling module. DETAILED DESCRIPTION
[0046] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.
[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0048] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the multi-operator-based task scheduling method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0049] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the multi-operator-based task scheduling method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0050] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.
[0051] In this embodiment, a multi-operator based task scheduling method is provided. Figure 2 is a flowchart of the multi-operator based task scheduling method of this embodiment. Figure 2 As shown, the process includes the following steps:
[0052] Step S210, based on the preliminary running result of the target program, obtaining a plurality of first processing times corresponding to each operator in the target program; the first processing time is the time for different processing units to run each part of the task in the operator;
[0053] Step S220, determining a target processing unit for parallel execution of operators in each processing unit;
[0054] Step S230, generating an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of each operator, a plurality of first processing times corresponding to each operator, and a preset operator dependency table.
[0055] Specifically, the target program includes multiple operators, each operator usually includes multiple partial tasks, and each partial task is distributed in different processing units. By running the target program for a short time, the preliminary running results of the target program are obtained, and based on the preliminary running results of the target program, multiple first processing times corresponding to each operator in the target program are obtained. The first processing time is the time for different processing units to run each partial task in the operator. This embodiment does not limit the type and number of processing units, and the corresponding processing unit is selected according to different application scenarios. For example, in the field of image processing, the processing unit may include a central processing unit (CPU) and a neural network processing unit (NPU).
[0056] Furthermore, the processing unit used to run the operator in parallel is used as the target processing unit, and the number of threads corresponding to the target processing unit is determined. For example, if there are 3 central processing units and 1 neural network processing unit available for scheduling, the current target processing unit is the central processing unit, the number of threads is 3, and the multiple first processing times are respectively the pre-processing time of the central processing unit, the post-processing time of the central processing unit, and the processing time of the neural network processing unit.
[0057] Afterwards, each time according to the number of threads and the number of operators corresponding to the target processing unit, several target operators are selected from each operator, and based on the operator dependency table, multiple target operators are added to each thread in different arrangements to run, and the running time corresponding to each arrangement is obtained. The arrangement corresponding to the minimum running time is recorded in the preset operator scheduling table until all operators are scheduled, and the final operator scheduling table is output.
[0058] It should be noted that the operator scheduling analysis is performed based on the number of threads corresponding to the target processing unit because the target processing unit is a processing unit for running operators in parallel. During the program running process, multiple target processing units are available for scheduling, indicating that each operator can be scheduled to the target processing unit of different threads for execution. In addition, for other processing units that can only execute a single task at the same time, such as the neural network processing unit, after obtaining the optimal scheduling method for all operators, other processing units execute the corresponding tasks in each operator in turn according to the order of arrangement of the operators.
[0059] In existing technical solutions, a lock-free centralized task scheduling mechanism is usually designed based on the unique characteristics of the processor architecture to optimize the scheduling overhead. However, the above solutions are strongly related to the characteristics of hardware devices and cannot be adapted to different application scenarios, resulting in poor versatility.
[0060] Compared with the prior art, the present application obtains multiple first processing times corresponding to each operator in the target program based on the preliminary running results of the target program, where the first processing time is the time for different processing units to run various tasks in the operator; determines the target processing unit in each processing unit for running the operator in parallel; generates an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of operators, the multiple first processing times corresponding to each operator, and the preset operator dependency table. Based on this, by pre-measuring the multiple first processing times corresponding to each operator in the target program, and scheduling according to the current number of threads, the total number of operators, the multiple first processing times corresponding to each operator, and the operator dependency table, an operator scheduling table is obtained, which solves the problem of being unable to adapt to different application scenarios, resulting in poor versatility, and avoids using processor characteristics for task scheduling to adapt to different application scenarios and improve versatility.
[0061] In some embodiments, generating an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of operators, a plurality of first processing times corresponding to each operator, and a preset operator dependency table in step S230 includes the following steps:
[0062] Step S231, each time selecting a number of target operators from each operator according to the number of threads corresponding to the target processing unit and the number of each operator;
[0063] Step S232, based on the operator dependency table, adding multiple target operators to each thread in different arrangements;
[0064] Step S233, determining the running time corresponding to each arrangement mode according to the multiple first processing times corresponding to each operator;
[0065] Step S234, recording the arrangement corresponding to the minimum running time into a preset operator scheduling table.
[0066] Specifically, determine the number of threads and operators corresponding to the target processing unit. In each round of operator scheduling analysis, select several target operators according to the number of threads and operators, and arrange and combine multiple target operators with each thread so that multiple target operators are distributed to each thread in different arrangements. For example, when the number of operators to be scheduled is 20 and the number of threads is 3, Figure 3 (a) and (b) are two different arrangements. Figure 3 (a) in the example distributes operators 2, 5, and 9 to threads 1, 2, and 3 respectively, while Figure 3 (b) in FIG. 4 distributes operators 4, 10, and 11 to threads 1, 2, and 3 respectively.
[0067] Furthermore, different arrangements are traversed. Under each arrangement, it is determined whether the predecessor operator of each target operator on each thread has been executed. According to the determination result and the preset operator dependency table, the operators on each thread are run, and the running time under each arrangement is obtained according to the multiple first processing times corresponding to each operator. The arrangement corresponding to the minimum running time is used as the current optimal scheduling method, recorded in the preset operator scheduling table, and the operator arrangement order is recorded as the current operator priority order. For example, when the number of operators to be scheduled is 20 and the number of threads is 3, Figure 4 (a) in the figure is the optimal operator scheduling method obtained by the first analysis. Operator 1, operator 7, and operator 20 are distributed in thread 1, thread 2, and thread 3 respectively. Figure 3 Based on the operator scheduling method shown in (a), a second operator scheduling analysis is performed to obtain the optimal operator scheduling method selected for the second time, as shown in Figure 4 As shown in (b), operator 1, operator 7 and operator 20 are distributed in thread 1, thread 2 and thread 3 respectively, and operator 5, operator 2 and operator 10 are used as post-operators of operator 1, operator 7 and operator 20 respectively, so that the running time of each current thread is minimized.
[0068] After that, the above steps are repeated until all operators are scheduled, and the final operator scheduling table is output to achieve load balancing. When the number of operators remaining is less than the number of threads, the remaining operators are selected and added to each thread.
[0069] It should be noted that obtaining the running time corresponding to each arrangement means determining the thread with the longest current running time from all threads under the current arrangement, and using the running time of the thread as the running time corresponding to the current arrangement.
[0070] For example, when c CPUs and 1 NPU are available for scheduling, the target processing unit is the CPU, and the corresponding number of threads is c. The number of operators to be scheduled in the target program is obtained as n, and the calculation formula is arranged , each time c target operators are selected to be arranged and combined with each thread, so that the target operators are added to each thread in different arrangements, and the running time corresponding to each arrangement is obtained. The arrangement corresponding to the minimum running time is recorded in the preset operator scheduling table. Repeat the above steps until all operators are scheduled, obtain the final operator scheduling table, and record the total running time corresponding to the current operator scheduling table.
[0071] In addition, after obtaining the optimal scheduling method for the operator, the first processing time and other related parameters of each operator can be fixed unchanged to achieve static load balancing; or, during the running of the target program, the relevant parameters of each operator are continuously measured to adjust the task allocation method, thereby achieving dynamic load balancing.
[0072] Through this embodiment, several target operators are selected from each operator according to the number of threads and the number of operators corresponding to the target processing unit each time, and multiple target operators are added to each thread in different arrangements based on the operator dependency table to get the running time corresponding to each arrangement, and the arrangement corresponding to the minimum running time is recorded in the preset operator scheduling table, so as to achieve load balancing by selecting the local optimal solution each time based on the principle of the local greedy algorithm to obtain the overall optimal operator scheduling method.
[0073] In some embodiments, adding multiple target operators to each thread in different arrangements based on the operator dependency table in step S232 includes the following steps:
[0074] Step S2321, adding multiple target operators to each thread in different arrangements;
[0075] Step S2322: Based on the operator dependency table, determine whether the predecessor operator of each target operator on the thread has completed execution;
[0076] Step S2323, run each thread according to the judgment result.
[0077] Specifically, in each round of operator scheduling analysis, several target operators are selected and arranged and combined with each thread according to the number of threads and the number of operators, so that multiple target operators are added to each thread in different arrangements.
[0078] Furthermore, each arrangement is traversed, and in each arrangement, based on the operator dependency table, it is determined whether the predecessor operator of each target operator on the thread has completed execution, and each thread is run according to the determination result. The operator dependency table is associated with the processing flow of the target program. If it is detected that the predecessor operator of the target operator has completed execution, it indicates that the current target operator is executable; if it is detected that the predecessor operator of the target operator has not completed execution, it indicates that the current target operator is not executable.
[0079] For example, in Figure 5In the operator dependency table shown, operator 1, operator 2, and operator 3 are all the predecessor operators of operator 4, and operator 1 is the predecessor operator of operator 3. Based on this, if operator 4 is selected to be added to a thread, operator 4 will not be executed if operator 1, operator 2, and operator 3 have not completed execution; if operator 3 is selected to be added to a thread, operator 3 will not be executed if operator 1 has not completed execution.
[0080] Through this embodiment, multiple target operators are added to each thread in different arrangements. Based on the operator dependency table, it is determined whether the predecessor operator of each target operator on the thread has completed execution, and then each thread is run according to the judgment result to avoid executing operators with lower rankings when the predecessor operator has not been executed, thereby ensuring the accuracy of the execution order of the operator scheduling table.
[0081] In some of the embodiments, the first processing times are respectively a pre-processing time of a central processing unit, a post-processing time of a central processing unit, and a processing time of a neural network processing unit.
[0082] In this embodiment, the processing unit includes a central processing unit and a neural network processing unit, and the central processing unit and the neural network processing unit are used to execute different parts of the tasks in the operator. Based on this, the multiple first processing times are respectively the pre-processing time of the central processing unit, the post-processing time of the central processing unit, and the processing time of the neural network processing unit, and the first processing time of each operator is used to determine the thread running time under different arrangement modes.
[0083] For example, Figure 6 As shown, each part of the tasks in operator 1 is pre-processed by the CPU, NPU and CPU post-processed respectively; operator 2 is executed by the CPU.
[0084] Through this embodiment, when the processing unit includes a central processing unit and a neural network processing unit, the multiple first processing times are respectively the pre-processing time of the central processing unit, the post-processing time of the central processing unit and the processing time of the neural network processing unit, so as to accurately obtain the thread running time under different arrangements.
[0085] In some of the embodiments, after generating the operator scheduling table corresponding to each operator, the following steps are also included:
[0086] Based on the pre-built runtime model, each thread in the operator scheduling table is processed to obtain the runtime of each thread;
[0087] According to the running time of each thread, the load balancing degree corresponding to the operator scheduling table is evaluated.
[0088] Specifically, after the final operator scheduling table is output, the time consumption of each thread in the operator scheduling table is calculated based on the pre-built runtime model to obtain the runtime of each thread, and the load balancing degree of the current operator scheduling method is evaluated based on the runtime of each thread.
[0089] Exemplarily, the running time of each thread is averaged to obtain the average running time corresponding to each thread, and the load balancing degree of the current operator scheduling method is evaluated according to the average running time; or, the running time of each thread is variance or standard deviation calculated, and the load balancing degree of the current operator scheduling method is evaluated according to the size of the calculation result; or, the running time of each thread is converted into a corresponding histogram to intuitively view the current load balancing degree. It should be noted that this embodiment does not limit the specific method of load balancing evaluation.
[0090] Through this embodiment, based on the pre-built runtime model, each thread in the operator scheduling table is processed to obtain the runtime of each thread, and the load balancing degree corresponding to the operator scheduling table is evaluated according to the runtime of each thread, so as to quantify the load in the multi-operator coordination process and realize accurate analysis of load balancing.
[0091] In some of the embodiments, before processing each thread in the operator scheduling table based on the pre-built runtime model, the following steps are also included:
[0092] Determine the wall time corresponding to each operator according to multiple second processing times corresponding to each operator; the second processing time includes the running time and waiting time of processing operators by different processing units;
[0093] Based on each wall time, a corresponding runtime model is constructed; the runtime model is used to determine the time required to execute all operators on a single thread.
[0094] Specifically, during the execution of each operator, the consumed time includes a plurality of second processing times, and the second processing time includes the running time and waiting time of processing the operator by different processing units.
[0095] The following description is based on the CPU and NPU as the processing units. The second processing time of each operator includes the CPU calculation time T calculate_CPU , CPU-related data scheduling time T transmit_CPU and the first process scheduling time T scheduling_CPU , CPU suspension waiting time T hang_CPU , NPU computing time T calculate_NPU , NPU second thread scheduling time T scheduling_NPU And NPU suspension waiting time T hang_NPU , then the specific calculation formula for CPU time and NPU time is as follows:
[0096]
[0097] In formula (1), T CPU represents CPU time, and T CPU Including CPU pre-processing time and CPU post-processing time; in formula (2), T NPU Represents NPU time. Based on this, the specific expression of wall time (Wall Duration) is as follows:
[0098]
[0099] In formula (3), T WALL Indicates wall time.
[0100] The operator calculation time is affected by factors related to different application scenarios. For example, in the field of image processing, the operator calculation time T calculate_NPU With the number of single input image frames N frames , the number of samples N used in each iteration of the model in the operator batch Related, assuming that the single inference time of the model remains unchanged, then T calculate_NPU It can be calculated by the following formula:
[0101]
[0102] In formula (4), T calculate_NPU Indicates NPU computing time; Indicates the single inference time of the model. It should be noted that the single inference time of the model With N batch is positively correlated, that is Will follow N batch The increase of N batch When selecting a model, N is usually selected. batch The number of frames N currently fed into the image frames Similar models, when the model processes different numbers of image frames, the utilization of the model is maximized as a whole, improving processing efficiency.
[0103] Furthermore, according to equations (1), (2), (3) and (4), the wall time corresponding to each operator is calculated, and based on each wall time, the corresponding running time model is constructed. The specific time consumption model is as follows:
[0104]
[0105] In formula (5), T totalIndicates the time taken to execute all operators on a single thread; n is the number of operators, m is the total number of input frames; (i)(j) means that the i operator is fed into the j frame; and for Checkframe(i)(j), when the i operator is fed into the j frame for processing, its value is 1, and if it is not fed into the j frame, its value is 0.
[0106] Through this embodiment, the wall time corresponding to the operator is determined according to the multiple second processing times corresponding to each operator, and the corresponding runtime model is constructed based on each wall time. The runtime model can then be used to calculate the time consumption of executing all operators on a single thread, so as to analyze the load balancing degree corresponding to the operator scheduling table.
[0107] In some embodiments, the second processing time includes a first process scheduling time of a central processing unit and a second process scheduling time of a neural network processing unit; wherein,
[0108] The first process scheduling time and the second process scheduling time are both positively correlated with the number of thread scheduling times; the number of thread scheduling times includes the number of times the thread is preempted and the number of times the thread actively gives up.
[0109] It should be noted that the first process scheduling time of the CPU is T scheduling_CPU , NPU second process scheduling time T scheduling_NPU , are positively correlated with the number of thread scheduling times. The number of thread scheduling times includes the number of times the thread is preempted and the number of times the thread voluntarily gives up.
[0110] The above process scheduling time and thread scheduling times are modeled as follows:
[0111]
[0112] In formula (6), n is the number of thread scheduling; is the relationship between the first process scheduling time of the CPU and the number of thread scheduling times n; in formula (7), The relationship between the second process scheduling time of NPU and the number of thread scheduling times n is shown in Figure 1. The relationship can be obtained by regression fitting: and .
[0113] Through this embodiment, according to the correlation between the process scheduling time and the thread scheduling number, a model is built to obtain the specific first process scheduling time and the second process scheduling time, so as to accurately calculate the wall time corresponding to each operator.
[0114] The present embodiment is described and illustrated below through preferred embodiments.
[0115] Figure 7 is a flowchart of a multi-operator-based task scheduling method of this preferred embodiment, such as Figure 7 As shown, the multi-operator based task scheduling method includes the following steps:
[0116] Step S710, based on the preliminary running result of the target program, obtaining a plurality of first processing times corresponding to each operator in the target program; the first processing time is the time for different processing units to run each part of the task in the operator;
[0117] Step S720, determining a target processing unit for parallel execution of operators in each processing unit;
[0118] Step S730, each time selecting a number of target operators from each operator according to the number of threads corresponding to the target processing unit and the number of each operator;
[0119] Step S740, based on the operator dependency table, adding multiple target operators to each thread in different arrangements;
[0120] Step S750, determining the running time corresponding to each arrangement mode according to the multiple first processing times corresponding to each operator;
[0121] Step S760, recording the arrangement corresponding to the minimum running time into a preset operator scheduling table;
[0122] Step S770, based on the pre-built runtime model, processing each thread in the operator scheduling table to obtain the runtime of each thread;
[0123] Step S780: Evaluate the load balancing degree corresponding to the operator scheduling table according to the running time of each thread.
[0124] Through this embodiment, based on the preliminary running results of the target program, multiple first processing times corresponding to each operator in the target program are obtained, and the first processing time is the time for different processing units to run each part of the task in the operator, and the target processing unit for parallel operation of the operator in each processing unit is determined. Each time, according to the number of threads and the number of operators corresponding to the target processing unit, several target operators are selected from each operator, and based on the operator dependency table, multiple target operators are added to each thread in different arrangements to run, and then according to the multiple first processing times corresponding to each operator, the running time corresponding to each arrangement is determined, and the arrangement corresponding to the minimum running time is recorded in the preset operator scheduling table, so as to avoid using processor characteristics for task scheduling, adapt to different application scenarios, and improve versatility.
[0125] Furthermore, based on the pre-built runtime model, each thread in the operator scheduling table is processed to obtain the runtime of each thread. Based on the runtime of each thread, the load balancing degree corresponding to the operator scheduling table is evaluated, thereby quantifying the load in the multi-operator coordination process and achieving accurate analysis of load balancing.
[0126] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0127] In this embodiment, a multi-operator-based task scheduling device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0128] Figure 8 is a structural block diagram of a multi-operator-based task scheduling device of this embodiment, such as Figure 8 As shown, the device includes: a measuring module 10, a determining module 20 and a scheduling module 30;
[0129] The measuring module 10 is used to obtain a plurality of first processing times corresponding to each operator in the target program based on the preliminary running result of the target program; the first processing time is the time for different processing units to run each part of the task in the operator;
[0130] A determination module 20, used to determine a target processing unit for parallel execution of operators in each processing unit;
[0131] The scheduling module 30 is used to generate an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of operators, multiple first processing times corresponding to each operator, and a preset operator dependency table.
[0132] Through the device provided in this embodiment, based on the preliminary running results of the target program, multiple first processing times corresponding to each operator in the target program are obtained, and the first processing time is the time for different processing units to run each part of the task in the operator; the target processing unit for running the operator in parallel in each processing unit is determined; according to the number of threads corresponding to the target processing unit, the number of operators, the multiple first processing times corresponding to each operator and the preset operator dependency table, an operator scheduling table corresponding to each operator is generated, which solves the problem of being unable to adapt to different application scenarios and resulting in poor versatility, and avoids using processor characteristics for task scheduling to adapt to different application scenarios and improve versatility.
[0133] In some of the embodiments, the scheduling module 30 is also used to select several target operators from each operator each time according to the number of threads and the number of operators corresponding to the target processing unit; based on the operator dependency table, add multiple target operators to each thread in different arrangements; determine the running time corresponding to each arrangement according to multiple first processing times corresponding to each operator; and record the arrangement corresponding to the minimum running time into a preset operator scheduling table.
[0134] In some of the embodiments, the scheduling module 30 is also used to add multiple target operators to each thread in different arrangements; based on the operator dependency table, determine whether the predecessor operator of each target operator on the thread has completed execution; and run each thread according to the determination result.
[0135] In some of these embodiments, Figure 8 On the basis of, the device also includes an evaluation module, which is used to process each thread in the operator scheduling table based on a pre-built runtime model to obtain the runtime of each thread; and evaluate the load balancing degree corresponding to the operator scheduling table according to the runtime of each thread.
[0136] In some of these embodiments, Figure 8 On the basis of, the device also includes a construction module, which is used to determine the wall time corresponding to the operator according to multiple second processing times corresponding to each operator; the second processing time includes the running time and waiting time of different processing units processing the operator; based on each wall time, a corresponding running time model is constructed; the running time model is used to determine the time consumed to execute all operators on a single thread.
[0137] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0138] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0139] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0140] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0141] S1, based on the preliminary running result of the target program, obtaining a plurality of first processing times corresponding to each operator in the target program; the first processing time is the time for different processing units to run each part of the task in the operator;
[0142] S2, determining a target processing unit for parallel operation of operators in each processing unit;
[0143] S3, generating an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of each operator, a plurality of first processing times corresponding to each operator, and a preset operator dependency table.
[0144] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.
[0145] In addition, in combination with the multi-operator based task scheduling method provided in the above embodiment, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by the processor, any of the multi-operator based task scheduling methods in the above embodiment is implemented.
[0146] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.
[0147] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.
[0148] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.
[0149] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.
Claims
1. A multi-operator based task scheduling method, characterized in that: The method comprises: Based on the preliminary running result of the target program, a plurality of first processing times corresponding to each operator in the target program are obtained; the first processing time is the time taken by different processing units to run each part of the task in the operator; Determining a target processing unit among the processing units for executing the operators in parallel; According to the number of threads corresponding to the target processing unit, the number of each operator, the multiple first processing times corresponding to each operator, and the preset operator dependency table, an operator scheduling table corresponding to each operator is generated, including: determining the number of threads corresponding to the target processing unit and the number of each operator; in each round of operator scheduling analysis, selecting multiple target operators according to the number of threads and the number of each operator, and distributing each target operator to each thread according to different arrangements; traversing each arrangement, and in each arrangement, judging whether the predecessor operator of each target operator on each thread has been executed, and running the operator on each thread according to the judgment result and the operator dependency table; according to the multiple first processing times corresponding to each operator, obtaining the running time under each arrangement, and recording the arrangement corresponding to the minimum running time as the current optimal scheduling method in the preset operator scheduling table; executing multiple rounds of the operator scheduling analysis until all the operators are scheduled, and outputting the operator scheduling table corresponding to each operator.
2. The multi-operator based task scheduling method according to claim 1, characterized in that: The multiple first processing times are respectively the pre-processing time of the central processing unit, the post-processing time of the central processing unit and the processing time of the neural network processing unit.
3. The multi-operator based task scheduling method according to claim 1, characterized in that: After generating the operator scheduling table corresponding to each operator, the method further includes: Based on a pre-built runtime model, processing each thread in the operator scheduling table to obtain the runtime of each thread; The load balancing degree corresponding to the operator scheduling table is evaluated according to the running time of each of the threads.
4. The multi-operator based task scheduling method according to claim 3, characterized in that: Before processing each thread in the operator scheduling table based on the pre-built runtime model, the method further includes: Determine the wall time corresponding to the operator according to a plurality of second processing times corresponding to each operator; the second processing time includes the running time and waiting time of different processing units processing the operator; Based on each of the wall times, a corresponding runtime model is constructed; the runtime model is used to determine the time consumption of executing all the operators on a single thread.
5. The multi-operator based task scheduling method according to claim 4, characterized in that: The second processing time includes the first process scheduling time of the central processing unit and the second process scheduling time of the neural network processing unit; wherein, The first process scheduling time and the second process scheduling time are both positively correlated with thread scheduling times; the thread scheduling times include thread preemption times and thread active yield times.
6. A multi-operator based task scheduling device, characterized in that: The device comprises: a measuring module, a determining module and a scheduling module; The measurement module is used to obtain a plurality of first processing times corresponding to each operator in the target program based on the preliminary running result of the target program; the first processing time is the time taken by different processing units to run each part of the task in the operator; The determination module is used to determine the target processing unit for parallel execution of the operator in each of the processing units; The scheduling module is used to generate an operator scheduling table corresponding to each operator according to the number of threads corresponding to the target processing unit, the number of each operator, multiple first processing times corresponding to each operator, and a preset operator dependency table, including: determining the number of threads corresponding to the target processing unit and the number of each operator; in each round of operator scheduling analysis, selecting multiple target operators according to the number of threads and the number of each operator, and distributing each target operator to each thread according to different arrangements; traversing each arrangement, and in each arrangement, judging whether the predecessor operator of each target operator on each thread has been executed, and running the operator on each thread according to the judgment result and the operator dependency table; obtaining the running time under each arrangement according to the multiple first processing times corresponding to each operator, and recording the arrangement corresponding to the minimum running time as the current optimal scheduling method in the preset operator scheduling table; executing multiple rounds of the operator scheduling analysis until all the operators are scheduled, and outputting the operator scheduling table corresponding to each operator.
7. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the multi-operator based task scheduling method according to any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-operator based task scheduling method according to any one of claims 1 to 5 are implemented.
