Dynamic scheduling system and method for computing kernel

Abstract

The invention discloses a computing kernel dynamic scheduling system and method.The computing kernel dynamic scheduling system comprises a request management module, a scheduling management module, a monitoring management module and an operation management module, and the request management module is used for storing all to-be-operated computing kernels belonging to different target tasks into different queues for management; the scheduling management module is used for determining the allowance degree of each computing kernel for delay according to execution feature data, collected by the monitoring management module, of each computing kernel for each computing kernel in the queue so as to determine the post-coefficient of each computing kernel, and then scheduling the computing kernel according to the post-coefficient of each computing kernel. And each computing kernel is scheduled to the corresponding target computing kernel to run, so that the efficiency of running each target task through the specified chip can be improved.

Description

Technical Field

[0001] The present specification relates to the field of intelligent computing technology, and in particular to a computing kernel dynamic scheduling system and method. Background Art

[0002] With the rapid development of artificial intelligence technology and the emergence of particularly large models, the demand for intelligent computing power has also increased dramatically.

[0003] However, when intelligent computing chips such as Graphics Processing Unit (GPU) schedule the computing cores of different tasks to run in different computing units, the fixed computing power and black box scheduling mechanism set in the intelligent computing chips may lead to problems such as waste of computing power and interference with task real-time performance, thereby limiting the efficiency of task execution through intelligent computing chips.

[0004] For example, GPU chips are generally composed of multiple computing units with independent computing capabilities (Streaming Multiprocessor), which contain multiple computing cores (Streaming Processor), as well as components such as instruction cache, L1 cache, and shared memory. However, the computing power of each computing core in these computing units is fixed, and because the scheduling mechanism inside the GPU chip is a black box for the tasks being executed, and is limited by the limitations of hardware capabilities, the scheduling mechanism used when scheduling the computing cores of the tasks being executed to run in the GPU chip is a simple first-in-first-out scheduling mechanism, which makes the computing power of the computing cores and computing units in the GPU chip cannot be fully utilized, resulting in a waste of computing power.

[0005] Another example: different computing tasks that need to be executed by GPU chips often have different real-time requirements. That is, some tasks have high real-time requirements, while some tasks have no real-time requirements. When the above two types of tasks are executed at the same time, problems such as resource competition and scheduling strategy conflicts may arise, which will lead to a decrease in the real-time performance and throughput of the GPU chip in executing computing tasks.

[0006] Therefore, how to improve the efficiency of task execution through intelligent computing chips is an urgent problem to be solved. Summary of the invention

[0007] This specification provides a computing kernel dynamic scheduling system and method to partially solve the above-mentioned problems existing in the prior art.

[0008] This manual adopts the following technical solutions: This specification provides a computing kernel dynamic scheduling system, the computing kernel dynamic scheduling system includes: a request management module, a scheduling management module, a monitoring management module, and an operation management module; The request management module is used to obtain at least one computing kernel to be run, wherein the computing kernel to be run is a computing business logic code segment of a target task to be executed; The scheduling management module is used to determine the post-coefficient of each computing kernel to be run according to the execution characteristic data of the computing kernel to be run, and determine the target computing kernel from each computing kernel to be run according to the post-coefficient of each computing kernel to be run and the operation status data of the preset designated chip collected by the monitoring management module, and determine the target processing unit from each processing unit of the designated chip, and send the target computing kernel and the target processing unit to the operation management module, wherein the execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run, and the operation status data includes: status data used to reflect the status of each processing unit in the designated chip and video memory resource occupancy data; The operation management module is used to dispatch the target computing kernel to the target processing unit for execution after receiving the target computing kernel and the target processing unit.

[0009] Optionally, the request management module is used to obtain a task request, determine a target task to be executed according to the task request, and determine at least one to-be-run computing kernel that needs to be run by a designated chip when executing the target task, and for each to-be-run computing kernel, according to the task type of the target task to which the to-be-run computing kernel belongs, store the to-be-run computing kernel in a preset waiting queue that matches the to-be-run computing kernel, wherein the waiting queues include: a real-time task waiting queue, a non-real-time task waiting queue, and an interrupt task waiting queue; The scheduling management module is used to determine the post-coefficient of each computing kernel to be run in each waiting queue.

[0010] Optionally, the task type of the target task includes: real-time task and non-real-time task; The scheduling management module is used to determine the post-processing coefficient of each computing kernel to be run according to the service performance indicator parameters and estimated completion time corresponding to the computing kernel to be run when it is determined that the task type of the target task is a real-time task.

[0011] Optionally, the scheduling management module is used to determine that the computing kernel to be run is a target computing kernel according to the post-coefficient of the computing kernel to be run, and when it is determined according to the running status data that there is no processing unit in an idle state among the processing units, determine the target processing unit from the processing units according to the service performance indicator parameters of the computing kernel being run by each processing unit, and send the target computing kernel and the target processing unit to the running management module; The operation management module is used to, upon receiving the target computing kernel and the target processing unit and determining that there is a computing kernel running in the target processing unit, stop the computing kernel running in the target processing unit and dispatch the target computing kernel to the target processing unit for execution.

[0012] Optionally, the processing unit includes: a computing unit, a computing core in the computing unit; The scheduling management module is used for, when it is determined that the designated chip contains a computing core that interrupts the operation of the target processing unit and the target processing unit is a computing unit, reacquiring the operation status data of the designated chip, and when it is determined according to the operation status data that there is an idle computing core in each processing unit, using the idle computing core as an interrupt task processing unit, using the interrupted computing core as an interrupt computing core, and sending the interrupted computing core and the interrupted task processing unit to the operation management module; The operation management module is used for dispatching the interruption calculation kernel to the interruption task processing unit for execution after receiving the interruption calculation kernel and the interruption task processing unit.

[0013] Optionally, the task type of the target task includes: real-time task and non-real-time task; The scheduling management module is used to determine the post-processing coefficient of each computing kernel to be run according to the waiting time and estimated completion time corresponding to the computing kernel to be run when it is determined that the task type of the target task is a non-real-time task.

[0014] Optionally, the scheduling management module is used to determine that the computing kernel to be run is a target computing kernel based on the post-coefficient of the computing kernel to be run, and when it is determined based on the running status data that there is a processing unit in an idle state among the processing units, use the processing unit in the idle state as the target processing unit, and when it is determined that the number of floating-point operations required to run the computing kernel to be run is less than a preset threshold, determine at least some of the computing kernels to be run from the computing kernels to be run to form a target computing kernel set with the target computing kernel according to a preset target combination strategy, and send the target computing kernel set and the target processing unit to the running management module.

[0015] Optionally, the processing unit includes: at least one of a computing unit and a computing core in the computing unit; The operation management module is used to determine whether the target processing unit is a computing unit after receiving the target computing kernel set. If so, for each computing kernel included in the target computing kernel set, determine a selected computing core from the computing cores of the target processing unit, and dispatch the computing kernel to the selected computing core for execution.

[0016] Optionally, the operation management module is used to create a control thread in a designated computing core of the target processing unit, and through the control thread, determine a selected computing core from the computing cores of the target processing unit for each computing core included in the target computing kernel set, and dispatch the computing kernel to the selected computing core for execution.

[0017] Optionally, the operation management module is used to monitor the computing cores of the target processing unit through the control thread, so that when it is determined that there is a computing core that has completed the operation among the computing cores of the target processing unit, the computing core that has completed the operation is used as an idle computing core, and the status data of the idle computing core and the video memory resource occupancy data of the idle computing core after the idle computing core completes the operation are uploaded to the monitoring management module.

[0018] Optionally, the monitoring management module is used to call a preset query interface to send an operation status data query request for the processing unit to be queried in the specified chip to the operation management module, and receive the status data of the processing unit to be queried and the video memory resource occupancy data returned by the operation management module according to the operation status data query request; and Used to receive the status data of the idle computing core uploaded by the operation management module and the video memory resource occupancy data after the idle computing core completes the operation.

[0019] Optionally, the scheduling management module is used to determine a target combination strategy from preset combination strategies based on the service performance indicator parameters corresponding to each computing kernel to be run, when it is determined that the number of floating-point operations required to run the computing kernel to be run is less than a preset threshold value, and according to the target combination strategy, determine at least some of the computing kernels to be run from the computing kernels to be run to form a target computing kernel set with the target computing kernel, and send the target computing kernel set and the target processing unit to the operation management module, wherein the combination strategies include: at least one of a first combination strategy and a second combination strategy, wherein the first combination strategy is used to determine the target computing kernel set based on the video memory resource occupancy data of the specified chip and the video memory resource data required for the operation of each computing kernel to be run, and the second combination strategy is used to determine the target computing kernel set based on the number of floating-point operations required for the operation of each computing kernel to be run.

[0020] This specification provides a computing kernel dynamic scheduling method, the method is applied to a computing kernel dynamic scheduling system, the computing kernel dynamic scheduling system includes: a request management module, a scheduling management module, a monitoring management module, and an operation management module, the method includes: Receive a task request, and obtain at least one to-be-run computing kernel according to the task request through the request management module, wherein the to-be-run computing kernel is a computing business logic code segment that executes a target task corresponding to the task request; Receive, by the scheduling management module, for each computing kernel to be run, according to the execution characteristic data of the computing kernel to be run, determine the post-coefficient of the computing kernel to be run, and determine the target computing kernel from each computing kernel to be run, and determine the target processing unit from each processing unit of the designated chip according to the post-coefficient of each computing kernel to be run and the operation status data of the preset designated chip collected by the monitoring management module, wherein the execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run, and the operation status data includes: status data used to reflect the status of each processing unit in the designated chip and video memory resource occupancy data; The target computing kernel and the target processing unit are sent to the operation management module, so that after receiving the target computing kernel and the target processing unit, the operation management module dispatches the target computing kernel to the target processing unit for execution.

[0021] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects: In the computing kernel dynamic scheduling system provided in the present specification, the computing kernel dynamic scheduling system includes: a request management module, a scheduling management module, a monitoring management module, and an operation management module, wherein the request management module is used to obtain at least one computing kernel to be run, where the computing kernel to be run is a computing business logic code segment of a target task to be executed, and the scheduling management module is used to determine the post-coefficient of the computing kernel to be run according to the execution characteristic data of the computing kernel to be run for each computing kernel to be run, and according to the post-coefficient of each computing kernel to be run and the operation status data of the designated chip collected by the monitoring management module, determine the target computing kernel from each computing kernel to be run, and select the post-coefficient of the designated chip from the post-coefficient of the designated chip. In each processing unit, a target processing unit is determined, and the target computing kernel and the target processing unit are sent to the operation management module. The execution characteristic data here is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run. The operation status data includes: status data used to reflect the status of each processing unit in the specified chip and video memory resource occupancy data. The operation management module is used to dispatch the target computing kernel to the target processing unit for execution after receiving the target computing kernel and the target processing unit.

[0022] It can be seen from the above method that, through the request management module of the computing kernel dynamic scheduling system, the computing kernels to be run belonging to different target tasks can be stored in different queues for management, and the scheduling management module can determine the post-post coefficient of each computing kernel in the queue according to the allowed degree of delay of each computing kernel, and then each computing kernel can be scheduled to the corresponding target computing kernel for execution according to the post-post coefficient of each computing kernel, thereby improving the efficiency of running each target task through the specified chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A schematic diagram of a computing kernel dynamic scheduling system provided in this specification; Figure 2 A schematic diagram of the computing kernel scheduling process provided in this specification; Figure 3A schematic diagram of a computing kernel monitoring method provided in this specification; Figure 4 A schematic diagram of a computing kernel operation control method provided in this specification; Figure 5 A flowchart of a computing kernel dynamic scheduling method provided in this specification; Figure 6 A schematic diagram of a computing kernel dynamic scheduling device provided in this specification; Figure 7 A method corresponding to the Figure 5 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0025] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0026] At present, the computing power of each processing unit of intelligent computing chips such as graphics processing units (GPUs) and neural network processing units (NPUs) is often fixed. Taking the A100 chip as an example, each computing unit has a fixed 64 computing cores, and the number of floating-point operations that each computing core can perform per second is fixed. At this time, when the central processing unit (CPU) transfers a smaller computing core (such as a computing core that only needs 12 computing cores to run) to the A100 chip for execution, it may cause the computing power resources of some computing cores in the computing unit allocated to this computing core to be wasted, which in turn leads to the chip's task processing efficiency being often low.

[0027] In addition, when the intelligent computing power chip is running a large number of computing cores of non-real-time tasks, the computing cores of the newly allocated real-time tasks may not be able to obtain sufficient computing power to run, which may lead to the real-time requirements of the real-time tasks not being met. Or when the intelligent computing power chip is running a large number of computing cores of real-time tasks, the computing cores of non-real-time tasks may be delayed indefinitely, resulting in "starvation".

[0028] Based on this, this specification provides a computing kernel dynamic scheduling system, specifically as follows Figure 1 shown.

[0029] Figure 1 A schematic diagram of a computing kernel dynamic scheduling system provided in this specification.

[0030] Combination Figure 1 As shown, in this specification, the computing kernel dynamic scheduling system may include: a central processing unit and at least one designated chip, wherein a request management module, a scheduling management module, a monitoring management module, and an operation management module may be set in the central processing unit, so that the central processing unit can use these modules to schedule each computing kernel required to execute different tasks to each processing unit of the designated chip to run, so as to improve the efficiency of task execution through the designated chip, wherein the above-mentioned designated chip can be set according to actual needs, for example: a graphics processor, a neural network processor, etc., and the above-mentioned processing units can refer to computing units in the designated chip, or can refer to computing cores in the computing unit.

[0031] Specifically, when a user needs to execute a task, a task request can be sent to the CPU. After receiving the task request, the CPU can determine the task to be executed as the target task based on the received task request through the request management module, and can determine at least one computing core that needs to be run by the designated chip when executing the target task as the computing core to be run, and then schedule and manage each computing core to be run of the target task, so as to schedule each computing core to be run of the target task to run in each processing unit of the designated chip.

[0032] There are many ways for the user to send a task request to the CPU. For example, the user can send a task request to the CPU through an application installed in the device used by the user. Another example is that the user can input a control instruction through an input device and send the control instruction to the CPU, so that the CPU obtains the task request sent by the user according to the received control instruction.

[0033] In the above content, the computing kernel of the target task can refer to the code segment corresponding to the computing business logic of the target task, that is, the computing kernel can refer to the core code part of the target task that undertakes the computing business logic. It can be understood that the computing kernel is the expression operation required to execute the target task (for example: multilinear function operation, etc.).

[0034] It should be noted that when the complexity of the target task is different, the number of computing cores included in the target task is also different, that is, the target task may include only one computing core or multiple computing cores.

[0035] In actual application scenarios, the target tasks that need to be executed by the designated chip at the same time can be one or more. Among them, when the to-be-run computing kernels of multiple target tasks need to be scheduled to the designated chip for execution, in order to avoid the problems of resource competition and scheduling strategy conflicts between different target tasks due to different real-time requirements of different target tasks. In this specification, the request management module can determine the waiting queue that matches the to-be-run computing kernel from the preset waiting queues for each to-be-run computing kernel of different target tasks according to the task type of the target task to which the to-be-run computing kernel belongs, and store the to-be-run computing kernel in the waiting queue that matches the to-be-run computing kernel, so that the scheduling management module can schedule the to-be-run computing kernels stored in different waiting queues.

[0036] The task type of the target task mentioned above can be used to reflect the target task's demand for computing resource allocation, such as real-time tasks and non-real-time tasks. Real-time tasks here usually require priority allocation of resources, such as CPU time, memory, and computing resources of a specified chip, to ensure that real-time tasks can be executed in a timely manner. Non-real-time tasks here usually allow for delays in resource allocation and require the overall throughput of multiple non-real-time tasks.

[0037] The above-mentioned waiting queues may include: real-time task waiting queues, non-real-time task waiting queues, and interrupt task waiting queues, wherein the interrupt task waiting queues are used to store computing cores whose computing resources have been preempted by other computing cores during operation.

[0038] Furthermore, the scheduling management module can schedule the computing cores in each waiting queue according to the running status data of the specified chip returned by the monitoring management module, as shown in the following example: Figure 2 shown.

[0039] Figure 2 A schematic diagram of the computing kernel scheduling process provided in this specification.

[0040] Combination Figure 2 It can be seen that the scheduling management module can determine the post-processing coefficient of each to-be-run computing kernel of the target task according to the execution characteristic data of the to-be-run computing kernel.

[0041] Among them, the above-mentioned execution characteristic data can be used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run.

[0042] The lower the post-coefficient of the computing kernel mentioned above is, the more the computing kernel needs to be prioritized in scheduling computing resources to run.

[0043] The above-mentioned service level objective (SLO) can be an indicator parameter set by the user for the quality of real-time task execution, for example, indicator parameters such as response time and delay set for the real-time performance when executing real-time tasks.

[0044] For ease of understanding, the following takes the response time as an example to explain in detail the role of the above service performance indicator parameters. For example, when the response time of the target task set by the user is 10 milliseconds, the specified chip is required to return the execution result of the target task within 10 milliseconds.

[0045] The estimated completion time of the computing kernel to be executed may be predicted by a preset neural network model, or may be determined based on the completion time of each computing kernel that has been executed historically.

[0046] Among them, the method for determining the estimated completion time of the computing kernel to be run based on the completion time of each computing kernel that has been run historically can be, according to the size of different computing kernels, dividing the computing kernels that have been run historically into different intervals (for example: n intervals, where n can be set according to actual needs, such as: 10), so that for each interval, the average of the completion times of the computing kernels that have been run historically and belong to the interval can be determined as the reference completion time corresponding to the interval, and then for each computing kernel to be run, the interval to which the computing kernel to be run belongs can be determined based on the size of the computing kernel, and the estimated completion time required to run the computing kernel to be run can be determined based on the reference completion time of the interval to which the computing kernel to be run belongs.

[0047] In this specification, for each processing unit, the running status data of the processing unit includes: status data used to reflect the status of each processing unit in the specified chip, and video memory resource occupancy data, wherein for each processing unit, the status data of the processing unit is used to reflect whether the processing unit is in an idle state. The video memory resource occupancy data here can be the size of the video memory used in the specified chip.

[0048] In addition, it can be seen from the above content that the target tasks of different task types often have different requirements for computing resource allocation. Therefore, the scheduling management module can use different post-coefficient determination strategies to determine the post-coefficients of the computing kernels of target tasks belonging to different task types.

[0049] Specifically, for each to-be-run computing kernel in each waiting queue, if the scheduling management module determines that the task type of the target task to which the to-be-run computing kernel belongs is a real-time task, the post-processing coefficient of the to-be-run computing kernel can be determined according to the service performance indicator parameter and the estimated completion time corresponding to the to-be-run computing kernel. Specifically, the following formula can be referred to:

[0050] In the above formula, To adjust the coefficient, you can set it according to actual needs, for example: 0.5, The remaining time after subtracting the current waiting time of the computing kernel to be run from the running time of the computing kernel to be run to complete the running time of the computing kernel to be run according to the service performance indicator parameter (i.e., the running time of the computing kernel to be run if the service performance indicator parameter is met), The estimated completion time required to run the computation kernel to be run.

[0051] It can be seen from the above formula that by adjusting the above adjustment coefficient, under normal circumstances (such as: the waiting time of the to-be-running computing kernel of the non-real-time task is lower than the specified value), the post-coefficient of the to-be-running computing kernel of the determined real-time task is lower than the post-coefficient of the to-be-running computing kernel of the determined non-real-time task.

[0052] Further, for each to-be-run computing kernel in each waiting queue, if the scheduling management module determines that the task type of the target task to which the to-be-run computing kernel belongs is a non-real-time task, the post-processing coefficient of the to-be-run computing kernel can be determined according to the waiting time and estimated completion time corresponding to the to-be-run computing kernel. Specifically, the following formula can be referred to:

[0053] In the above formula, is the estimated completion time required to run the computation kernel to be run, The waiting time corresponding to the kernel to be run.

[0054] From the above content, it can be seen that when the waiting time of the computing kernel to be run of a non-real-time task is higher than the specified value, the post-coefficient determined for the computing kernel to be run of the non-real-time task begins to be lower than the post-coefficient of the computing kernel to be run of the real-time task with a shorter waiting time, thereby avoiding the situation where the computing kernel to be run of the non-real-time task cannot be run for a long time.

[0055] Furthermore, the scheduling management module can determine that the computing kernel to be run of the real-time task is the target computing kernel based on the post-post coefficient of each computing kernel to be run in each waiting queue, and when it is determined that there is an idle processing unit in each processing unit based on the running status data of the specified chip, the processing unit in the idle state can be used as the target processing unit, and then the target computing kernel and the target processing unit can be sent to the operation management module, so that the operation management module can dispatch the target computing kernel to the target processing unit for execution after receiving the target computing kernel and the target processing unit.

[0056] In addition, when the scheduling management module determines that the computing kernel of the real-time task is the computing kernel to be run according to the post-post coefficient of each computing kernel in each waiting queue, and determines that there is no idle processing unit in each processing unit according to the running status data of the specified chip, it can determine the target processing unit from each processing unit according to the service performance indicator parameters of the computing kernel running in each processing unit, and send the target computing kernel and the target processing unit to the operation management module, so that the operation management module receives the target computing kernel and the target processing unit, and when it is determined that there is a running computing kernel in the target processing unit, stops the running computing kernel of the target processing unit and dispatches the target computing kernel to the target processing unit for execution.

[0057] Specifically, the scheduling management module can determine the computing cores currently being run in the computing cores of each real-time task as the computing cores to be preempted based on the metadata of each computing core currently running saved in the preset running queue, and then, for each computing core to be preempted, determine the position of the metadata of the computing core to be preempted in the running queue based on the remaining running time of the computing core to be preempted while meeting the service performance indicator parameters.

[0058] The longer the remaining time of the computing kernel to be preempted is, the closer the position of the metadata of the computing kernel to be preempted is to the head of the queue in the running queue.

[0059] The metadata of each computing kernel mentioned above is updated by the operation management module. The metadata of each computing kernel includes: the task type of the target task to which the computing kernel belongs, the running time of the computing kernel, the service performance indicator parameters corresponding to the computing kernel, the size of the video memory resources required when the computing kernel is running, the processing unit occupied by the computing kernel, etc.

[0060] Furthermore, the scheduling management module can start from the head of the running queue and determine in turn for each computing kernel's metadata whether the size of video memory resources required for the computing kernel to be occupied when the computing kernel is running in the metadata of the computing kernel is lower than a preset occupancy threshold. If so, the processing unit where the computing kernel corresponding to the metadata of the computing kernel is located is determined as the target processing unit.

[0061] In addition, if the scheduling management module does not determine the target processing unit after traversing the running queue, it can determine the computing cores currently being run among the computing cores of each non-real-time task as the computing cores to be preempted, and use the processing unit of the computing core to be preempted with the highest post-coefficient among the computing cores to be preempted as the target processing unit.

[0062] Furthermore, when the scheduling management module determines that the computing kernel to be run of a non-real-time task is the target computing kernel based on the post-postion coefficient of each computing kernel to be run in each waiting queue, the method of scheduling the target computing kernel to run in each processing unit can include two cases, and these three cases are described in detail below.

[0063] The first case is that when the scheduling management module determines that there is an idle computing unit in each processing unit based on the running status data of the designated chip, the idle processing unit can be used as the target processing unit, and when it is determined that the number of floating-point operations required to run the computing kernel to be run is greater than a preset threshold, the target computing kernel and the target processing unit can be sent to the running management module.

[0064] The second situation is that when the scheduling management module determines that there is an idle computing unit in each processing unit based on the running status data of the designated chip, the idle processing unit can be used as the target processing unit, and when it is determined that the number of floating-point operations required to run the computing kernel to be run is less than a preset threshold, according to a preset target combination strategy, at least part of the computing kernels to be run and the target computing kernel are determined from each computing kernel to be run to form a target computing kernel set, and the target computing kernel set and the target processing unit are sent to the running management module.

[0065] It should be noted that in actual application scenarios, since the video memory resources of the designated chip are usually limited, if the video memory resources are insufficient, it may take extra time to acquire and migrate data, which will lead to a decrease in the efficiency of the designated chip running the computing kernel. Therefore, in order to improve the utilization rate of the video memory resources of the designated chip, the scheduling management module determines that the remaining time of each to-be-run computing kernel stored in the real-time task waiting queue is greater than the preset remaining time threshold when the service performance indicator parameters are met according to the service performance indicator parameters of each to-be-run computing kernel stored in the real-time task waiting queue. Then, the preset first combination strategy can be determined as the target combination strategy. Then, according to the target combination strategy, the target computing kernel set can be determined based on the video memory resource occupancy data of the designated chip and the video memory resource data required for each to-be-run computing kernel of the target task when running.

[0066] Of course, the scheduling management module can also determine the target computing kernel set according to the target combination strategy, based on the video memory resource occupancy data of the specified chip and the video memory resource data required for each running computing kernel in the non-real-time task waiting queue.

[0067] Specifically, the scheduling management module can determine the remaining video memory resource size of the specified chip based on the video memory resource occupancy data of the specified chip, and then determine at least some of the computing kernels to be run and the target computing kernel from each computing kernel to be run to form a target computing kernel set based on the video memory resource size required for the execution of each computing kernel to be run and the remaining video memory resource size of the specified chip.

[0068] In addition, when the scheduling management module determines, based on the service performance indicator parameters of each waiting computing kernel stored in the real-time task waiting queue, that there is a waiting computing kernel whose remaining time when meeting the service performance indicator parameters is less than a preset remaining time threshold among the waiting computing kernels stored in the real-time task waiting queue, then a preset second combination strategy can be determined as the target combination strategy, and then the target computing kernel set can be determined according to the target combination strategy based on the number of floating-point operations required for each waiting computing kernel to be run and the number of floating-point operations that the target processing unit can process per second.

[0069] From the above content, it can be seen that when the scheduling management module needs to combine the computing kernels of the non-real-time task and hand them over to the computing unit in the designated chip for execution, it can determine whether the remaining time of the computing kernels of the current real-time task is sufficient while satisfying the service performance indicator parameters based on the service performance indicator parameters of the computing kernels of the real-time task. If sufficient, the computing kernels to be run of the non-real-time task can be combined with the goal of maximizing the utilization of the video memory resources of the designated chip and handed over to the target processing unit for execution, thereby improving the throughput of the designated chip running the computing kernels. If insufficient, the computing kernels to be run of the non-real-time task can be combined with the goal of maximizing the utilization of the computing resources of the processing unit in an idle state, thereby improving the satisfaction rate of the service performance indicator parameters of the computing kernels of the real-time task.

[0070] Furthermore, after receiving the target computing kernel set and the target processing unit, the operation management module can determine whether the target processing unit is a computing unit. If so, it can determine the selected computing core from the computing cores of the target processing unit for each computing kernel included in the target computing kernel set, and dispatch the computing kernel to the selected computing core for execution.

[0071] Specifically, the operation management module can also create a control thread in the designated computing core of the target processing unit, and through the control thread, determine the selected computing core from the computing cores of the target processing unit for each computing core included in the target computing kernel set, and dispatch the computing core to the selected computing core for execution, and, further, can monitor the computing cores of the target processing unit through the control thread, so that when it is determined that there is a computing core that has completed execution among the computing cores of the target processing unit, the computing core that has completed execution is regarded as an idle computing core, and the status data of the idle computing core and the video memory resource occupancy data after the idle computing core completes execution are uploaded to the monitoring management module.

[0072] Furthermore, there are two methods for the monitoring management module to collect the operating status data of the specified chip, specifically: Figure 3 shown.

[0073] Figure 3 A schematic diagram of the computing kernel monitoring method provided in this specification.

[0074] Combination Figure 3It can be seen that the first method for the monitoring and management module to collect the operating status data of the specified chip is that the monitoring and management module can receive the status data of the idle computing core uploaded by the operation management module and the video memory resource occupancy data after the idle computing core completes the operation, and determine the operating status data of the specified chip based on the status data of the idle computing core uploaded by the operation management module and the video memory resource occupancy data after the idle computing core completes the operation.

[0075] The second method for the monitoring and management module to collect the operating status data of the specified chip is that the monitoring and management module can call a preset query interface to send an operating status data query request for the processing unit to be queried in the specified chip to the operation management module, and receive the status data of the processing unit to be queried and the video memory resource occupancy data returned by the operation management module according to the operating status data query request, and determine the operating status data of the specified chip based on the status data of the processing unit to be queried and the video memory resource occupancy data returned by the operation management module according to the operating status data query request.

[0076] Specifically, the above-mentioned monitoring and management module can treat each processing unit as a processing unit to be queried, and send an operation status data query request for the processing unit to be queried through a preset query interface according to a specified query cycle, so that the operation management module can respond to the received operation status data query request and return the status data of the processing unit to be queried and the video memory resource occupancy data.

[0077] The above-mentioned designated query period can be determined according to the operation status data of the designated chip, that is, if the number of processing units in the designated chip that are in idle state is greater according to the operation status data of the designated chip, the determined designated query period will be longer.

[0078] It should be noted that, since data needs to be transmitted between the CPU and the designated chip through a high-speed serial computer expansion bus (Peripheral Component Interconnect Express, PCIe), and the total amount of PCIe transmission bandwidth between the CPU and the designated chip (i.e., the amount of data that the PCIe bus can transmit per unit time) is limited, when the monitoring and management module actively initiates an operation status data query request through a preset query interface, competition for PCIe transmission bandwidth may occur because the PCIe transmission bandwidth between the CPU and the designated chip is being used by the computing tasks of each computing core running on the designated chip, thereby reducing the execution efficiency of the target task.

[0079] It can be seen from the above content that in order to avoid the influence of the query actively initiated by the monitoring management module on the execution efficiency of the target task, the monitoring management module can combine two different methods to obtain the operation status data of the specified chip, wherein the operation management module monitors the computing cores of the target processing unit through the control thread, so that when it is determined that there are computing cores that have completed the operation among the computing cores of the target processing unit, the computing cores that have completed the operation are regarded as idle computing cores, and the status data of the idle computing cores and the memory resource occupancy data of the idle computing cores after the idle computing cores complete the operation are actively uploaded to the monitoring management module. This can reduce the frequency of queries actively initiated by the monitoring management module, and thereby reduce the impact on the efficiency of the specified chip in executing the target task.

[0080] In addition, in order to avoid the PCIe resources of the designated chip being occupied, thereby reducing the efficiency of the designated chip in executing the target task, when the scheduling management module determines that there is a computing kernel of the target processing unit interrupting the operation in the designated chip, it can be scheduled to be stored in the interrupt task waiting queue, or the device to device (D2D) method can be used to directly migrate the computing kernel of the target processing unit interrupting the operation to other processing units inside the designated chip to run, so as to avoid the occupation of the PCIe resources of the designated chip during the process of scheduling it to be stored in the interrupt task waiting queue. Figure 4 shown.

[0081] Figure 4 A schematic diagram of a computing kernel operation control method provided in this specification.

[0082] Combination Figure 4 It can be seen that when the scheduling management module determines that there is a computing kernel that interrupts the operation of the target processing unit in the specified chip, and determines that the target processing unit is a computing unit, it can re-acquire the operating status data of the specified chip, and when it is determined based on the operating status data that there is a computing core in an idle state in each processing unit, the computing core in the idle state is used as an interrupt task processing unit, and the computing kernel that interrupts the operation is used as an interrupt computing kernel, and the interrupt computing kernel and the interrupt task processing unit are sent to the operation management module, so that the operation management module can schedule between the processing units of the execution chip after receiving the interrupt computing kernel and the interrupt task processing unit, so as to dispatch the interrupt computing kernel to the interrupt task processing unit for operation.

[0083] From the above content, it can be seen that through the request management module of the above-mentioned computing kernel dynamic scheduling system, the various computing kernels to be run belonging to different target tasks can be stored in different queues for management, and the scheduling management module can determine the post-post coefficient of each computing kernel in the queue according to the allowed degree of delay of each computing kernel, and then each computing kernel can be scheduled to the corresponding target computing kernel for execution according to the post-post coefficient of each computing kernel, thereby improving the efficiency of running each target task through the specified chip.

[0084] For ease of understanding, the process of dynamically scheduling the computing kernel through the above computing kernel dynamic scheduling system is described in detail below. Figure 5 shown.

[0085] Figure 5 A flowchart of a computing kernel dynamic scheduling method provided in this specification includes the following steps: S501: Receive a task request, and obtain at least one to-be-run computing kernel according to the task request through the request management module, wherein the to-be-run computing kernel is a computing business logic code segment that executes a target task corresponding to the task request.

[0086] S502: receiving, by the scheduling management module, for each computing kernel to be run, determining the post-coefficient of the computing kernel to be run according to the execution characteristic data of the computing kernel to be run, and determining the target computing kernel from each computing kernel to be run according to the post-coefficient of each computing kernel to be run and the running status data of the preset designated chip collected by the monitoring management module, and determining the target processing unit from each processing unit of the designated chip, wherein the execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run, and the running status data includes: status data used to reflect the status of each processing unit in the designated chip and video memory resource occupancy data.

[0087] S503: Send the target computing kernel and the target processing unit to the operation management module, so that after receiving the target computing kernel and the target processing unit, the operation management module dispatches the target computing kernel to the target processing unit for execution.

[0088] In this specification, the execution subject used to implement the computing kernel dynamic scheduling method can refer to a specified device such as a server, an embedded device, or a terminal device such as a desktop computer, a laptop computer, etc. For the sake of ease of description, the following only takes the terminal device as the execution subject as an example to illustrate the computing kernel dynamic scheduling method provided in this specification.

[0089] In this specification, a terminal device can receive a task request and, through a request management module of a computing kernel dynamic scheduling system deployed in a CPU of the terminal device and running therein, obtain, based on the received task request, at least one computing kernel that needs to be run when a target task corresponding to the received task request is executed, as the computing kernel to be run, wherein the computing kernel to be run is a computing business logic code segment for executing the target task corresponding to the task request.

[0090] Furthermore, the terminal device can determine the post-processing coefficient of each computing kernel to be run according to the execution characteristic data of the computing kernel to be run through the scheduling management module of the computing kernel dynamic scheduling system, and determine the target computing kernel from each computing kernel to be run and the target processing unit from each processing unit of the designated chip according to the post-processing coefficient of each computing kernel to be run and the running status data of the designated chip collected by the monitoring management module.

[0091] Among them, the above-mentioned execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run.

[0092] The above-mentioned operation status data includes: status data for reflecting the status of each processing unit in the specified chip and video memory resource occupancy data.

[0093] When the terminal device obtains the target computing kernel and the target processing unit determined by the scheduling management module, it can send the target computing kernel and the target processing unit to the operation management module, so that after receiving the target computing kernel and the target processing unit, the operation management module dispatches the target computing kernel to the target processing unit for execution.

[0094] From the above content, it can be seen that through the above-mentioned dynamic scheduling method of computing kernels, the computing kernels to be run belonging to different target tasks can be stored in different queues for management, and for each computing kernel in the queue, the post-post coefficient of each computing kernel is determined according to the allowed degree of delay of each computing kernel. Then, according to the post-post coefficient of each computing kernel, each computing kernel can be scheduled to run in the corresponding target computing kernel, thereby improving the efficiency of running each target task through the specified chip.

[0095] The above is a computing kernel dynamic scheduling system and method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding computing kernel dynamic scheduling device, such as Figure 6 shown.

[0096] Figure 6 A schematic diagram of a computing kernel dynamic scheduling device provided in this specification specifically includes: A first receiving module, configured to receive a task request, and obtain at least one to-be-run computing kernel according to the task request through the request management module, wherein the to-be-run computing kernel is a computing business logic code segment for executing a target task corresponding to the task request; A second receiving module is used to receive, by the scheduling management module, for each computing kernel to be run, the post-coefficient of the computing kernel to be run determined according to the execution characteristic data of the computing kernel to be run, and the target computing kernel determined from each computing kernel to be run according to the post-coefficient of each computing kernel to be run and the running status data of the preset designated chip collected by the monitoring management module, and the target processing unit determined from each processing unit of the designated chip, wherein the execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run, and the running status data includes: status data used to reflect the status of each processing unit in the designated chip and video memory resource occupancy data; The scheduling module is used to send the target computing kernel and the target processing unit to the operation management module, so that the operation management module dispatches the target computing kernel to the target processing unit for execution after receiving the target computing kernel and the target processing unit.

[0097] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 5 The dynamic scheduling method of computing kernels shown.

[0098] This manual also provides Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 3 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0099] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0100] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0101] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0102] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0103] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0108] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0109] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0111] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0113] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0114] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A computing kernel dynamic scheduling system, characterized in that: The computing kernel dynamic scheduling system includes: a request management module, a scheduling management module, a monitoring management module, and an operation management module; The request management module is used to obtain at least one computing kernel to be run, wherein the computing kernel to be run is a computing business logic code segment of a target task to be executed; The scheduling management module is used to determine the post-coefficient of each computing kernel to be run according to the execution characteristic data of the computing kernel to be run, and determine the target computing kernel from each computing kernel to be run according to the post-coefficient of each computing kernel to be run and the operation status data of the preset designated chip collected by the monitoring management module, and determine the target processing unit from each processing unit of the designated chip, and send the target computing kernel and the target processing unit to the operation management module, wherein the execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run, and the operation status data includes: status data used to reflect the status of each processing unit in the designated chip and video memory resource occupancy data; The operation management module is used to dispatch the target computing kernel to the target processing unit for execution after receiving the target computing kernel and the target processing unit.

2. The computing kernel dynamic scheduling system according to claim 1, characterized in that: The request management module is used to obtain a task request, determine a target task to be executed according to the task request, and determine at least one to-be-run computing kernel that needs to be run by a designated chip when executing the target task. For each to-be-run computing kernel, according to the task type of the target task to which the to-be-run computing kernel belongs, the to-be-run computing kernel is stored in a preset waiting queue that matches the to-be-run computing kernel, wherein the waiting queues include: a real-time task waiting queue, a non-real-time task waiting queue, and an interrupt task waiting queue; The scheduling management module is used to determine the post-coefficient of each computing kernel to be run in each waiting queue.

3. The computing kernel dynamic scheduling system according to claim 1, characterized in that: The task types of the target tasks include: real-time tasks and non-real-time tasks; The scheduling management module is used to determine the post-processing coefficient of each computing kernel to be run according to the service performance indicator parameters and estimated completion time corresponding to the computing kernel to be run when it is determined that the task type of the target task is a real-time task.

4. The computing kernel dynamic scheduling system according to claim 3, characterized in that: The scheduling management module is used to determine that the computing kernel to be run is a target computing kernel according to the post-position coefficient of the computing kernel to be run, and when it is determined according to the running status data that there is no processing unit in an idle state among the processing units, determine the target processing unit from the processing units according to the service performance indicator parameters of the computing kernel being run by each processing unit, and send the target computing kernel and the target processing unit to the running management module; The operation management module is used to, upon receiving the target computing kernel and the target processing unit and determining that there is a computing kernel running in the target processing unit, stop the computing kernel running in the target processing unit and dispatch the target computing kernel to the target processing unit for execution.

5. The computing kernel dynamic scheduling system according to claim 4, characterized in that: The processing unit includes: a computing unit and a computing core in the computing unit; The scheduling management module is used for, when it is determined that the designated chip contains a computing core that interrupts the operation of the target processing unit and the target processing unit is a computing unit, reacquiring the operation status data of the designated chip, and when it is determined according to the operation status data that there is an idle computing core in each processing unit, using the idle computing core as an interrupt task processing unit, using the interrupted computing core as an interrupt computing core, and sending the interrupted computing core and the interrupted task processing unit to the operation management module; The operation management module is used for dispatching the interruption calculation kernel to the interruption task processing unit for execution after receiving the interruption calculation kernel and the interruption task processing unit.

6. The computing kernel dynamic scheduling system according to claim 1, characterized in that: The task types of the target tasks include: real-time tasks and non-real-time tasks; The scheduling management module is used to determine the post-processing coefficient of each computing kernel to be run according to the waiting time and estimated completion time corresponding to the computing kernel to be run when it is determined that the task type of the target task is a non-real-time task.

7. The computing kernel dynamic scheduling system according to claim 6, characterized in that: The scheduling management module is used to determine that the computing kernel to be run is a target computing kernel according to the post coefficient of the computing kernel to be run, and when it is determined according to the running status data that there is a processing unit in an idle state among the processing units, use the processing unit in the idle state as the target processing unit, and when it is determined that the number of floating-point operations required to run the computing kernel to be run is less than a preset threshold, determine at least some of the computing kernels to be run from the computing kernels to be run to form a target computing kernel set with the target computing kernel according to a preset target combination strategy, and send the target computing kernel set and the target processing unit to the running management module.

8. The computing kernel dynamic scheduling system according to claim 7, characterized in that: The processing unit includes: at least one of a computing unit and a computing core in the computing unit; The operation management module is used to determine whether the target processing unit is a computing unit after receiving the target computing kernel set. If so, for each computing kernel included in the target computing kernel set, determine a selected computing core from the computing cores of the target processing unit, and dispatch the computing kernel to the selected computing core for execution.

9. The computing kernel dynamic scheduling system according to claim 8, characterized in that: The operation management module is used to create a control thread in the designated computing core of the target processing unit, and through the control thread, determine a selected computing core from the computing cores of the target processing unit for each computing core included in the target computing core set, and dispatch the computing core to the selected computing core for execution.

10. The computing kernel dynamic scheduling system according to claim 9, characterized in that: The operation management module is used to monitor the computing cores of the target processing unit through the control thread, so that when it is determined that there is a computing core that has completed operation among the computing cores of the target processing unit, the computing core that has completed operation is used as an idle computing core, and the status data of the idle computing core and the video memory resource occupancy data after the idle computing core completes the operation are uploaded to the monitoring management module.

11. The computing kernel dynamic scheduling system according to claim 10, characterized in that: The monitoring management module is used to call a preset query interface to send an operation status data query request for the processing unit to be queried in the specified chip to the operation management module, and receive the status data of the processing unit to be queried and the video memory resource occupancy data returned by the operation management module according to the operation status data query request; as well as, Used to receive the status data of the idle computing core uploaded by the operation management module and the video memory resource occupancy data after the idle computing core completes the operation.

12. The computing kernel dynamic scheduling system according to claim 7, characterized in that: The scheduling management module is used to determine a target combination strategy from preset combination strategies based on the service performance indicator parameters corresponding to each computing kernel to be run, when it is determined that the number of floating-point operations required to run the computing kernel to be run is less than a preset threshold value, and according to the target combination strategy, determine at least part of the computing kernels to be run from the computing kernels to be run to form a target computing kernel set with the target computing kernel, and send the target computing kernel set and the target processing unit to the operation management module, wherein the combination strategies include: at least one of a first combination strategy and a second combination strategy, wherein the first combination strategy is used to determine the target computing kernel set based on the video memory resource occupancy data of the specified chip and the video memory resource data required for the operation of each computing kernel to be run, and the second combination strategy is used to determine the target computing kernel set based on the number of floating-point operations required for the operation of each computing kernel to be run.

13. A computing kernel dynamic scheduling method, characterized in that: The method is applied to a computing kernel dynamic scheduling system, the computing kernel dynamic scheduling system includes: a request management module, a scheduling management module, a monitoring management module, and an operation management module, and the method includes: Receive a task request, and obtain at least one to-be-run computing kernel according to the task request through the request management module, wherein the to-be-run computing kernel is a computing business logic code segment that executes a target task corresponding to the task request; Receive, by the scheduling management module, for each computing kernel to be run, according to the execution characteristic data of the computing kernel to be run, determine the post-coefficient of the computing kernel to be run, and determine the target computing kernel from each computing kernel to be run, and determine the target processing unit from each processing unit of the designated chip according to the post-coefficient of each computing kernel to be run and the operation status data of the preset designated chip collected by the monitoring management module, wherein the execution characteristic data is used to reflect the delay size allowed by the computing kernel to be run, including: the task type of the target task to which the computing kernel to be run belongs, the waiting time of the computing kernel to be run, the estimated completion time of the computing kernel to be run, and at least one of the service performance indicator parameters corresponding to the computing kernel to be run, and the operation status data includes: status data used to reflect the status of each processing unit in the designated chip and video memory resource occupancy data; The target computing kernel and the target processing unit are sent to the operation management module, so that after receiving the target computing kernel and the target processing unit, the operation management module dispatches the target computing kernel to the target processing unit for execution.

Images