A flow scheduling method, computer device, medium and program product

By allocating unoccupied computing resources from multiple execution flows in the artificial intelligence model, the low parallelism problem caused by the exclusive resource of the execution flow is solved, and more efficient computing resource utilization is achieved and the computing parallelism of hardware resources is improved. .

CN118606034BActive Publication Date: 2024-11-08北京壁仞科技开发有限公司 +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411081658.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-08
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

In detection and inference business scenarios such as portraits and faces, the execution flow of the artificial intelligence model in the prior art exclusively occupies the computing resources of the device, resulting in low computational parallelism of hardware resources and the inability to effectively utilize the computing resources of the device.

Method used

Through a stream scheduling method, unoccupied computing resources are determined from multiple execution flows, and tasks are distributed to the resource for execution, avoiding any execution flow from exclusively occupying all computing resources, thereby improving the computing parallelism of hardware resources. The method includes obtaining a task, determining the execution flow, and scheduling it to an unoccupied computing resource for execution.

Benefits of technology

It improves the computing parallelism of hardware resources, improves the computing resource utilization rate of equipment, and solves the problem of low parallelism caused by the exclusive resource of execution flow.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118606034B_ABST
    Figure CN118606034B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a flow scheduling method, a computer device, a medium and a program product for improving the computational parallelism of hardware resources. The method includes: obtaining a first task from a first queue; determining a first execution flow from a plurality of execution flows, and distributing the first task to the first execution flow; determining a first computing resource that is not occupied by a second execution flow from a plurality of computing resources included in a hardware resource pool, and scheduling the first execution flow to execute on the first computing resource; wherein the second execution flow is an execution flow that is scheduled to the second computing resource before the first execution flow among the plurality of execution flows and is being executed. In this way, while the second execution flow is scheduled to execute on the second computing resource, the first execution flow can also be scheduled to execute on the first computing resource, so that any execution flow will not monopolize all computing resources in the hardware resource pool, thereby improving the computational parallelism of hardware resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a flow scheduling method, computer equipment, medium, and program product. Background Art

[0002] A heterogeneous system usually includes a host and a device. In practical applications, the operation of the artificial intelligence model can be realized through the device. The operation of the artificial intelligence model can be realized by the operators in the computational graph. The computational graph is a multi-graph structure used to represent the computing tasks and data flow processes of the artificial intelligence model. The operator refers to the various operations performed on the tensors of each layer in the artificial intelligence model.

[0003] At present, in the business scenarios of portrait and face detection and reasoning, when the device processes the tasks of operators in models such as Modnet and Yolo, as well as the tasks of model pre- and post-processing operators, multiple execution streams and computing resources in the device are used, where the execution stream is a sequence of tasks executed in sequence. These models or operators mutually exclusively share all execution streams and all computing resources in the device. Specifically, when a model uses an execution stream, for example, the execution stream is stream1, stream1 will monopolize all computing resources in the device. If other models or model pre- and post-processing operators need to be called at this time, and another execution stream, such as stream2, is used, since all computing resources in the device are monopolized by stream1, stream2 can only get a response after the task on stream1 is completed. After getting the response, stream2 monopolizes all computing resources in the device. This way of monopolizing all computing resources of the device by one execution stream will result in low computing parallelism of hardware resources. Summary of the invention

[0004] The embodiments of the present invention provide a flow scheduling method, a computer device, a medium and a program product, which are used to improve the computing parallelism of hardware resources.

[0005] In a first aspect, the present application provides a flow scheduling method, the method comprising: obtaining a first task from a first queue, the first queue being used to store tasks issued by an application; determining a first execution flow from multiple execution flows, and distributing the first task to the first execution flow; determining a first computing resource that is not occupied by a second execution flow from multiple computing resources included in a hardware resource pool, and scheduling the first execution flow to be executed on the first computing resource; wherein the second execution flow is an execution flow among the multiple execution flows that was scheduled to the second computing resource before the first execution flow and is being executed.

[0006] Through the above scheme, after obtaining the first task from the first queue, the first execution flow can be determined from multiple execution flows, and the first task can be distributed to the first execution flow; then, the first computing resource that is not occupied by the previously scheduled second execution flow is determined from the multiple computing resources included in the hardware resource pool, and then the first execution flow is scheduled to be executed on the first computing resource. In this way, while the second execution flow is scheduled to be executed on the second computing resource, the first execution flow can also be scheduled to be executed on the first computing resource. That is to say, any execution flow will not monopolize all computing resources in the hardware resource pool, thereby improving the computing parallelism of hardware resources.

[0007] In a possible implementation, after distributing the first task to the first execution flow, it also includes: creating a first event; inserting the first event into the first execution flow, and storing the first event in a second queue, the execution order of the first event on the first execution flow being after the execution order of the first task.

[0008] In a possible implementation, the method further includes: obtaining the first event from the second queue, querying the execution status of the first event, the execution status of the first event being used to indicate the execution status of the first task; if the execution status of the first event is found to be completed, releasing the first execution flow and the first computing resource.

[0009] In a possible implementation, the method further includes: if it is found that the execution status of the first event is a completed execution status, storing the completed execution status of the first event in a third queue.

[0010] In a possible implementation, before obtaining the first task from the first queue, it also includes: calling the first API to send the first task to the first queue; the method also includes: if the status of the first event being executed is queried from the third queue, determining that the first API call is completed.

[0011] In one possible implementation, the first API is a synchronous API. After calling the first API to send the first task to the first queue, it also includes: waiting for the first API call to be completed, and then calling the second API to send the second task to the first queue, where the second task is a task in the first queue that is arranged after the first task.

[0012] In one possible implementation, the first API is an asynchronous API. Before calling the first API to send the first task to the first queue, it also includes: calling a third API to create a logic flow, the logic flow includes multiple tasks, and the first task is any one of the multiple tasks; determining the first execution flow from multiple execution flows, including: if there is an execution flow that meets a first preset condition among the multiple execution flows, then determining the execution flow that meets the first preset condition as the first execution flow, the first preset condition includes being executed and distributed with a third task belonging to the logic flow, and the third task is a task arranged before the first task in the logic flow; or, if there is no execution flow that meets the first preset condition among the multiple execution flows, then determining the execution flow to which the task is not distributed from the multiple execution flows as the first execution flow.

[0013] In a possible implementation, the method further includes: calling a stream synchronization API to wait for all tasks on the logical stream to be completed.

[0014] In a second aspect, the present application provides a flow scheduling device, including:

[0015] An acquiring unit, configured to acquire a first task from a first queue, wherein the first queue is used to store tasks issued by an application;

[0016] a processing unit, configured to determine a first execution flow from a plurality of execution flows;

[0017] A scheduling unit, configured to distribute the first task to the first execution flow;

[0018] The processing unit is further used to determine a first computing resource that is not occupied by a second execution flow from a plurality of computing resources included in the hardware resource pool; wherein the second execution flow is an execution flow that is scheduled before the first execution flow and is being executed among the plurality of execution flows;

[0019] The scheduling unit is further used to schedule the first execution flow to be executed on the first computing resource.

[0020] In a third aspect, the present application provides a computer device, comprising a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, wherein the processor chip implements the method in the above-mentioned first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, comprising computer-executable instructions, which, when executed on a computer, enable the computer to execute the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0022] In a fifth aspect, the present application provides a computer program product, wherein the computer program product stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0023] The technical effects that can be achieved in any of the second to fifth aspects mentioned above can refer to the description of the beneficial effects in the first aspect mentioned above or any possible implementation method of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the architecture of a heterogeneous computing system applicable to embodiments of the present application;

[0025] Figure 2 A schematic diagram of the structure of key software components provided in the embodiments of the present application;

[0026] Figure 3 A flow chart of a flow scheduling method provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of a flow scheduling method in a synchronous API call scenario provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of a flow scheduling method in an asynchronous API call scenario provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of the structure of a flow scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0031] like Figure 1The following is a schematic diagram of the architecture of a heterogeneous computing system applicable to the present application. The heterogeneous computing system includes a host and a device. For example, the host may be a central processing unit (CPU), and the device may be a graphics processing unit (GPU) or a general-purpose graphics processing unit (GPGPU). In a heterogeneous computing system, the number of devices may be one or more, and the present application does not specifically limit the number of devices. It should be understood that the heterogeneous computing system may also include other modules or components, and the present application does not specifically limit this. For ease of description, the following embodiments are described by taking one device as an example.

[0032] The host is used to handle non-computing tasks in artificial intelligence applications. The host can include multiple processing cores, which can perform multiple tasks simultaneously, greatly improving the computing power and performance of servers including heterogeneous computing systems.

[0033] The device is used to handle a large number of computing tasks in artificial intelligence applications. The operation of the artificial intelligence model is realized through the device, and the operation of the artificial intelligence model can be realized by the operator in the computational graph, where the computational graph is a multi-graph structure used to represent the computing tasks and data flow process of the artificial intelligence model, and the operator refers to the various operations performed on the tensors of each layer in the artificial intelligence model.

[0034] The device includes a hardware resource pool and video memory, wherein the hardware resource pool includes multiple computing resources for executing the tasks of operators in the model or the tasks of pre- and post-processing operators of the model, and each computing resource may include one or more programmable multiprocessors, such as a stream processor cluster (SPC). It should be understood that the device may also include other modules or components, which are not limited in this application. This application also does not limit the number of computing resources included in the hardware resource pool.

[0035] It should be understood that the above-mentioned host and device may also include more or fewer components, which are not described in detail here.

[0036] At present, in the business scenarios of portrait and face detection and reasoning, when the device processes the tasks of operators in models such as Modnet and Yolo, as well as the tasks of pre- and post-processing operators of the model, multiple execution flows and computing resources in the device are used. The execution flow is a sequence of tasks executed in order. For an execution flow, the execution flow needs to be scheduled to the computing resources of the device, and the computing resources of the device execute the tasks on the scheduled execution flow. Generally speaking, all tasks on an execution flow are executed serially. If different tasks need to be executed concurrently, they need to be distributed to different execution flows. These models or operators mutually exclusively share all execution streams and all computing resources in the device. Take the example of all computing resources in the device including 12 SPCs. Specifically, when a model uses an execution stream, for example, the execution stream is stream1, stream1 will monopolize the 12 SPCs. If other models or model pre- and post-processing operators need to be called at this time, and another execution stream, such as stream2, is used, stream2 can only wait for the task on stream1 to be completed and the 12 SPCs to be released before it can get a response. Then stream2 monopolizes the 12 SPCs in the device. Similarly, when the model pre- and post-processing operators use an execution stream, they will also monopolize the 12 SPCs. If the operators in the model use another execution stream at this time, they can only wait for the 12 SPCs to be released before they can get a response. This way of one execution stream monopolizing all computing resources of the device will result in low computing parallelism of hardware resources.

[0037] In view of this, this application is based on Figure 1 The architecture diagram of the heterogeneous computing system shown in FIG. 1 provides a flow scheduling method for improving the computing parallelism of hardware resources. The flow scheduling method provided in the present application can include Figure 1 The server execution of the heterogeneous computing system is shown.

[0038] The server also includes Figure 2 The key software components shown include pipeline buffer library and image processing library (BPP Library), where the pipeline buffer library includes user thread, application program interface (Application ProgramInterface, API) wrapper, dispatch thread (DispatchThread), query thread (QueryThread), task manager (TaskManager), task queue (TaskQueue), event buffer (EventBuffer), status queue (StatusMap) and stream task table (StreamTaskTable). The user thread, dispatch thread and query thread run on the host in the server.

[0039] The API wrapper encapsulates one or more APIs, and any API can be a synchronous API or an asynchronous API.

[0040] The user thread is used to call the API to send tasks of various models or operators to the task queue. For example, when the user thread calls the synchronous API to send a task to the task queue, the user thread will wait for the execution result of the task to be returned before calling the next API to send the next task; for another example, when the user thread calls the asynchronous API to send a task to the task queue, it can continue to call the next API to send the next task to the task queue without waiting for the execution interface of the task. In these two cases, the next API can be a synchronous API or an asynchronous API.

[0041] The task manager is used to manage the dispatch thread and the query thread.

[0042] The distribution thread is used to obtain tasks from the task queue, distribute them to the execution flow in the driver layer, and schedule the execution flow to at least one computing resource in the hardware resource pool for execution. The execution flow is a sequence of tasks executed in sequence for the driver layer.

[0043] The distribution thread is also used to create an event after distributing the task to the execution flow, and insert the event into the execution flow, so as to judge the execution status of the task by the execution status of the event on the execution flow. The event is stored in the event buffer.

[0044] The query thread is used to obtain events from the event buffer and query the execution status of the event. The execution status of the event is used to indicate the execution status of the task before the event. For example, if the execution status of the event is the completed status, it can be determined that the task before the event has been completed. The query thread is also used to store the completed status of the event in the status queue.

[0045] If a user thread calls an asynchronous API to send a task to a task queue, before that, the user thread is also used to call an API to create a logical stream. A logical stream is a sequence of tasks that are executed in order for the application layer. A user thread can concurrently create multiple logical streams, each of which includes multiple tasks. The user thread can call an asynchronous API to send each task in each logical stream to the task queue.

[0046] It should be understood that the order of the tasks in the task queue can be sorted according to the time when the tasks are received. When the distribution thread takes tasks from the task queue, it takes tasks according to the order of the task queue and then distributes them to the available execution streams. The correspondence between the execution stream and the tasks distributed to the execution stream can be stored in the stream task table (StreamTaskTable). For example, the logical stream Logical Stream1 includes Task1, Task2 and Task3. Task1~Task3 are sent to the task queue in sequence. For example, after Task1 is distributed to execution stream 1, the distribution thread stores the correspondence between execution stream 1 and Task1 in the stream task table; when the distribution thread obtains Task2 from the task queue, it also distributes Task2 to execution stream 1, and the distribution thread stores the correspondence between execution stream 1 and Task2 in the stream task table; when the distribution thread obtains Task3 from the task queue, it distributes Task3 to execution stream 2, and the distribution thread stores the correspondence between execution stream 2 and Task3 in the stream task table.

[0047] Based on the above description, the flow scheduling method provided by this application is described in detail below. Figure 3 As shown, the flow scheduling method includes the following steps:

[0048] Step 301, obtaining a first task from a first queue.

[0049] In the embodiment of the present application, a distribution thread is running on the host in the server, and step 301 can be executed by the distribution thread.

[0050] In step 301, the first queue is used to store the tasks of each model or operator sent by the application layer. The host in the server also runs a user thread, and the application in the application layer sends the tasks of each model or operator to the first queue through the user thread. The first queue includes at least the first task. Here, the first queue is, for example, Figure 2 The task queue in .

[0051] In a possible implementation, before step 301, the user thread may call a first API to send the first task to the first queue. The first API may be a synchronous API or an asynchronous API.

[0052] In the case where the first API is a synchronous API, after the user thread calls the first API to send the first task to the first queue, it waits for the first API call to complete, and then calls the second API to send the second task to the first queue, where the second task is a task in the first queue that is arranged after the first task. Among them, waiting for the first API call to complete can be understood as waiting for the first task to complete.

[0053] When the first API is an asynchronous API, before the user thread calls the first API to send the first task to the first queue, the user thread calls the third API to create a logic flow, the logic flow includes multiple tasks, and the first task is any one of the multiple tasks.

[0054] For multiple tasks in the logic flow, the user thread calls different asynchronous APIs to send the multiple tasks in the logic flow to the first queue in sequence, and each task in the logic flow corresponds to an asynchronous API. After calling any asynchronous API to send a task to the first queue, the user thread can continue to call another asynchronous API to send the next task to the first queue without waiting for the asynchronous API call to complete.

[0055] Step 302: determine a first execution flow from multiple execution flows, and distribute the first task to the first execution flow.

[0056] This step 302 may be executed by the distribution thread. The server includes multiple execution flows, and the present application does not limit the specific number of execution flows included in the server.

[0057] There are many possible implementations for determining the first execution flow from multiple execution flows.

[0058] In implementation mode 1, the first API is a synchronous API, that is, the first task is sent to the first queue by the synchronous API. The distribution thread can determine any execution flow of undistributed tasks from multiple execution flows as the first execution flow, that is, distribute the first task to any execution flow of undistributed tasks.

[0059] In the second implementation mode, the first API is an asynchronous API, that is, the first task is sent to the first queue by the asynchronous API. This can be described in two cases.

[0060] In one case, if there is an execution flow that satisfies a first preset condition among multiple execution flows, the distribution thread will determine any execution flow that satisfies the first preset condition as the first execution flow, wherein the first preset condition includes a third task that is being executed and distributed and belongs to the logical flow, and the third task is a task that is arranged before the first task in the logical flow.

[0061] In another case, if there is no execution flow satisfying the first preset condition among the multiple execution flows, the distribution thread determines an execution flow to which the task has not been distributed from the multiple execution flows as the first execution flow.

[0062] In the second embodiment, when there are tasks belonging to a logical flow being executed and distributed among multiple execution flows, the distribution thread can preferentially distribute the first task to the execution flow that meets the first preset condition, that is, the execution flow to which the tasks in the logical flow are distributed and is being executed. Since multiple tasks on the same logical flow are likely to have dependencies, this method can make multiple tasks on the same logical flow be distributed to the same execution flow as much as possible, and all tasks distributed to the same execution flow are executed serially. Therefore, the execution order of multiple tasks on the same logical flow can be guaranteed, thereby ensuring the execution continuity of multiple tasks with dependencies.

[0063] In the embodiment of the present application, the distribution thread may also store the corresponding relationship between the tasks obtained from the first queue and the corresponding distributed execution flows in the fourth queue. The fourth queue here is, for example, Figure 2 The stream task table shown.

[0064] In the case where the above-mentioned first API is an asynchronous API, after the distribution thread obtains the first task from the first queue, it can first search from the fourth queue whether there is a task belonging to the same logical flow as the first task. For example, the third task mentioned above is found in the fourth queue, that is, the third task and the first task belong to the same logical flow, and the arrangement order of the third task in the logical flow is before the first task. If the execution flow corresponding to the third task in the fourth queue is being executed, the execution flow corresponding to the third task in the fourth queue can be used as the first execution flow, and the distribution thread can also distribute the first task to the execution flow corresponding to the third task.

[0065] Step 303, determine a first computing resource that is not occupied by the second execution flow from the multiple computing resources included in the hardware resource pool, and schedule the first execution flow to be executed on the first computing resource; wherein the second execution flow is an execution flow among the multiple execution flows that was scheduled to the second computing resource before the first execution flow and is being executed.

[0066] This step 303 may be performed by a distribution thread.

[0067] In the embodiment of the present application, the device in the server includes a hardware resource pool, which may include multiple computing resources. For example, all computing resources included in the hardware resource pool are N SPCs, where N is a positive integer, and each computing resource may include one or more SPCs. For example, N is 12, which means that the hardware resource pool includes 12 SPCs.

[0068] Before scheduling the first execution flow, the distribution thread has scheduled the second execution flow to the second computing resource, and the tasks distributed on the second execution flow are also obtained by the distribution thread from the first queue. Among them, the specific implementation of the process of the distribution thread obtaining tasks, distributing the obtained tasks to the second execution flow, and scheduling the second execution flow to the second computing resource pool can refer to the relevant description of steps 301 to 303, which will not be repeated here.

[0069] When the first execution flow needs to be scheduled, since the second execution flow is being executed on the second computing resource, the computing resources allocated to the first execution flow can be determined from the unoccupied computing resources in the hardware resource pool, that is, the first computing resources. For example, all computing resources included in the hardware resource pool are 12 SPCs, and the second computing resources include 3 SPCs. That is to say, when the first execution flow needs to be scheduled, there are 9 SPCs left in the hardware resource pool that are available and not occupied by other execution flows. The distribution thread can determine one or more SPCs from the available 9 SPCs as the first computing resource. The specific number of SPCs determined as the first computing resource can be determined based on the amount of computing required to be performed on the first execution flow, and there is no limitation here. For example, if 5 SPCs are needed to execute the first task, then 5 SPCs can be determined from the 9 unoccupied SPCs, and then the first execution flow can be scheduled to execute on the determined 5 SPCs.

[0070] After any task involved in the embodiments of the present application is executed on the computing resource, the execution result obtained can be stored in the video memory of the device. After the user thread obtains the status of the task being executed through the API previously used to call the task, the user thread obtains the execution result of the task from the video memory.

[0071] In the above embodiments, when scheduling the first execution flow, an execution flow is scheduled before the first execution flow and is being executed as an example for illustration. In some other embodiments, multiple execution flows may be scheduled to computing resources in the hardware resource pool before the first execution flow and are being executed. In this case, the aforementioned second execution flow refers to multiple execution flows being executed. At this time, if there are computing resources in the hardware resource pool that are not occupied by the second execution flow, the first computing resources can be determined from the computing resources that are not occupied by the second execution flow. If all computing resources in the hardware resource pool are completely occupied by the second execution flow, that is, when the first execution flow needs to be scheduled, there are no available computing resources in the hardware resource pool. You can wait until there are available computing resources in the hardware resource pool before determining the first computing resources. For example, after the tasks on other execution flows are executed, the previously occupied computing resources will be released, and the released computing resources are available, that is, not occupied.

[0072] In an embodiment of the present application, after obtaining the first task from the first queue, the first execution flow can be determined from multiple execution flows, and the first task can be distributed to the first execution flow; then, the first computing resource that is not occupied by the previously scheduled second execution flow is determined from the multiple computing resources included in the hardware resource pool, and then the first execution flow is scheduled to be executed on the first computing resource. In this way, while the second execution flow is scheduled to be executed on the second computing resource, the first execution flow can also be scheduled to be executed on the first computing resource. That is to say, all computing resources in the hardware resource pool can be allocated to different execution flows for use, and any execution flow will not monopolize all computing resources in the hardware resource pool. Compared with the related art, when computing resources are needed to execute an execution flow, all computing resources in the hardware resource pool are directly used by the same execution flow, and the computing resources in the hardware resource pool are not split and used, which causes one execution flow to monopolize all computing resources in the hardware resource pool. The flow scheduling scheme of the present application can improve the computing parallelism of hardware resources.

[0073] Furthermore, when computing resources need to be allocated for a certain execution flow, the distribution thread can dynamically allocate available computing resources to the execution flow that needs to be allocated computing resources based on the available computing resources in the hardware resource pool. Since the number of available computing resources in the hardware resource pool changes dynamically, the number of computing resources required for tasks on each execution flow is not fixed, so the number of computing resources ultimately allocated to each execution flow is also not fixed, that is, the number of computing resources occupied by different execution flows may be the same or different.

[0074] It should be noted that the second execution flow in the above step 303 is scheduled to be executed on the second computing resource before the first execution flow. The specific implementation details of the entire process from a task being distributed to the second execution flow to the second execution flow being scheduled to be executed on the second computing resource can be referred to the description of the specific implementation details of the entire process from the first task being distributed to the first execution flow to the first execution flow being scheduled to be executed on the first computing resource in this application, and this application will not repeat them.

[0075] Based on the above embodiment, after distributing the first task to the first execution flow in the above step 302, the distribution thread may also create a first event and insert the first event into the first execution flow, and the execution status of the first event is used to indicate the execution status of the first task. Since the execution time point of the first event on the first execution flow is after the execution time point of the first task, if the execution status of the first event is a completed state, then it can be determined that the first task has been completed.

[0076] In a possible implementation, the first event may be stored in the second queue. Here, the second queue is, for example, Figure 2The event buffer is shown.

[0077] A query thread is also running on the host in the server. The query thread can obtain the first event from the second queue and query the execution status of the first event. If the query thread finds that the execution status of the first event is completed, the first execution flow and the first computing resource are released. After the first execution flow is released, the first execution flow is available, and the distribution thread can distribute other tasks to the available first execution flow; after the second computing resource is released, the second computing resource is also available and can be allocated to the execution flow with task execution requirements, that is, the distribution thread can schedule the execution flow with task execution requirements to the second computing resource for execution.

[0078] If the query finds that the execution status of the first event is completed, the query thread can store the completed execution status of the first event in the third queue. The first API called by the user thread can query whether the first event is completed from the third queue. If the query finds that the first event is completed from the third queue, the user thread determines that the first API call is completed. The third queue here is, for example, Figure 2 The status queue is shown.

[0079] In the embodiment of the present application, when the first API is an asynchronous API, the user thread can also call the stream synchronization API to wait for all tasks on the logical flow to be completed.

[0080] Based on any of the above embodiments, taking multiple execution flows including execution flow 1 to execution flow 4, and all computing resources included in the hardware resource pool as SPC1 to SPC12 as an example, the flow scheduling method implemented in the synchronous API call scenario and the asynchronous API call scenario is introduced in detail in combination with specific examples.

[0081] Figure 4 Schematic diagram of a flow scheduling method in a synchronous API call scenario provided by an embodiment of the present application. Figure 4 As shown, the flow scheduling method mainly includes the following steps:

[0082] Step 401: The user thread calls a synchronous API to send Task 1 to a task queue, and waits for Task 1 to be completed.

[0083] Step 402: The distribution thread obtains task 1 from the task queue.

[0084] Step 403 : The distribution thread obtains an available execution flow 1 from multiple execution flows, and distributes task 1 to execution flow 1 .

[0085] Here, when task 1 is distributed, if there are multiple available execution flows among multiple execution flows, for example, execution flows 1 to 3 are all available execution flows, then in step 403, task 1 may also be distributed to execution flow 2 or execution flow 3, which is not limited here. Any execution flow of the distributed task is no longer an available execution flow. For example, the distribution thread distributes task 1 to execution flow 1. After task 1 is distributed to execution flow 1, execution flow 1 is no longer an available execution flow until task 1 on execution flow 1 is executed and execution flow 1 is released. After being released, execution flow 1 becomes an available execution flow again, and other tasks can be distributed to execution flow 1.

[0086] Step 404: the distribution thread obtains available SPC1-SPC3 from the hardware resource pool, and schedules execution flow 1 to SPC1-SPC3.

[0087] Among them, the hardware resource pool includes 12 SPCs, among which the SPCs that are not occupied by any execution flow are available SPCs, for example, the available SPCs include SPC1~SPC9, and the distribution thread can determine how many SPCs are required for all tasks on execution flow 1 according to the task amount of task 1. The above step 404 is explained by taking the example that task 1 requires 3 SPCs. The distribution thread can select 3 SPCs from the available SPCs in the hardware resource pool, for example, select the available SPC1~SPC3, of course, it can also select other three available SPCs, and this application does not limit this.

[0088] Step 405 , the distribution thread calls the asynchronous API to schedule the target operator 1 in the image processing library to SPC1 - SPC3 , so as to use the target operator 1 to execute task 1 .

[0089] Step 406, distribute thread creation event 1.

[0090] Step 407 : The distribution thread inserts event 1 into execution flow 1 .

[0091] Step 408: The distribution thread stores event 1 in the event buffer.

[0092] Step 409: The query thread obtains event 1 from the event buffer.

[0093] Step 410, query the execution status of event 1.

[0094] Step 411 , when the query thread finds that the execution status of event 1 is completed, execution flow 1 is released, and SPC1 - SPC3 are released.

[0095] Step 412: The query thread stores the status of event 1 being completed in the status queue.

[0096] Step 413, while the synchronous API called by the user thread is waiting for the completion of Task 1, the status queue can be queried, and when the status of Event 1 being completed is queried, it is determined that Task 1 is completed.

[0097] Among them, the synchronous API queries the status queue to find that event 1 has been executed, indicating that task 1 has been executed and the synchronous API call is completed; when the synchronous API queries the status queue, if the status queue does not contain the status that event 1 has been executed, indicating that task 1 has not been executed, then wait for a while and continue to query the status queue until the status that event 1 has been executed is found.

[0098] In actual applications, for example, if a user thread needs to send Task 1, Task 2, and Task 3 to the task queue, first, the user thread calls synchronization API 1 to send Task 1 to the task queue, and then waits for Task 1 to be completed; after Task 1 is completed, the user thread calls another synchronization API 2 to send Task 2 to the task queue, and then waits for Task 2 to be completed; after Task 2 is completed, the user thread calls another synchronization API 3 to send Task 3 to the task queue, and then waits for Task 3 to be completed. Similarly, the process of calling subsequent synchronization APIs (such as synchronization API 2 and synchronization API 3) refers to the process of calling synchronization API 1, and will not be repeated one by one.

[0099] It should be noted that the distribution thread creation event in the above step 406 can be to create an event for each task. For example, after calling the synchronous API 1 to send task 1 to the task queue, event 1 is created; then after calling the synchronous API 2 to send task 2 to the task queue, event 2 is created; after calling the synchronous API 2 to send task 3 to the task queue, event 3 is created.

[0100] Figure 5 Schematic diagram of the flow scheduling method in the asynchronous API call scenario provided by the embodiment of the present application. Figure 5 As shown, the flow scheduling method mainly includes the following steps:

[0101] Step 501: The user thread creates a logic flow 1, and the logic flow 1 includes multiple tasks.

[0102] Step 502: The user thread calls asynchronous API 1 to send a target task to a task queue. The target task may be any one of the multiple tasks included in logic flow 1.

[0103] Step 503: The distribution thread obtains the target task from the task queue.

[0104] Step 504 , the distribution thread searches the flow task table to see whether it has found the target execution flow. If so, step 505 is executed; if not, step 508 is executed.

[0105] The target execution flow is the execution flow corresponding to other tasks in the flow task table that belong to the same logical flow as the target task.

[0106] Step 505 , the distribution thread determines whether the target execution flow is being executed, and if so, executes step 506 ; if not, executes step 508 .

[0107] Step 506: The distribution thread distributes the target task to the target execution flow.

[0108] Step 507 , the distribution thread calls asynchronous API 2 to schedule the target operator in the image processing library to computing resource 1 so as to use the target operator to execute the target task. Thereafter, step 511 is executed.

[0109] Step 508: The distribution thread obtains available execution flows from the multiple execution flows, and distributes the target task to the available execution flows.

[0110] Step 509: the distribution thread obtains available computing resource 2 from the hardware resource pool, and schedules the available execution flow to computing resource 2.

[0111] Step 510: the distribution thread calls asynchronous API 3 to schedule the target operator in the image processing library to computing resource 2, so as to use the target operator to execute the target task. Then, step 511 is executed.

[0112] Step 511: the distribution thread creates a target event.

[0113] Step 512: The distribution thread inserts the target event into the execution flow where the target task is located, and stores the target event in the event buffer.

[0114] Step 513: The query thread obtains the target event from the event buffer.

[0115] Step 514, query the execution status of the target event.

[0116] Step 515: if the query thread finds that the execution status of the target event is completed, the execution flow where the target task is located is released, and the computing resources occupied by the execution flow where the target task is located are released.

[0117] Step 516: The query thread stores the target event's completed execution status in the status queue.

[0118] Step 517: The user thread calls the stream synchronization API to wait for all tasks belonging to the logical stream 1 to be completed.

[0119] All tasks included in the logic flow one in the above step 501 can be sent to the task queue by the user thread calling different asynchronous APIs. The distribution process of other tasks in the logic flow one except the above target task and the process of scheduling the execution flow after being distributed to the execution flow can be referred to steps 503 to 516, which will not be repeated here.

[0120] The above step 517 can be executed after all tasks included in the logic flow one are sent to the task queue. If the stream synchronization API call in step 517 is completed, it means that all tasks belonging to the logic flow one are executed.

[0121] The following takes an example in which multiple execution flows include execution flow 1 to execution flow 4, all computing resources included in the hardware resource pool are SPC1 to SPC12, logic flow one includes task 1, task 2 and task 3, and logic flow two includes task 4 and task 5, to illustrate the specific details of distributing each task on logic flow one and logic flow two in sequence and scheduling execution flows.

[0122] S1, the user thread can call different asynchronous APIs 1 to send task 1, task 2, task 3, task 4, and task 5 to the task queue, and arrange them in the task queue according to the time they arrived at the task queue. Task 1, task 2, task 4, task 3, and task 5 are arranged in the task queue in sequence. When the distribution thread obtains tasks from the task queue for distribution, it can take tasks from the task queue in sequence according to the first-come-first-out principle for distribution.

[0123] S2, the distribution thread obtains task 1 from the task queue, determines the available execution flow 1 from execution flow 1 to execution flow 4, and distributes task 1 to execution flow 1.

[0124] S3, the distribution thread determines the unoccupied SPC1~SPC3 from the hardware resource pool according to the computing amount of task 1, and schedules execution flow 1 to SPC1~SPC3. After task 1 is executed, execution flow 1 and the corresponding computing resources SPC1~SPC3 will be released.

[0125] Among them, the computing amount of task 1 needs to be executed using three SPCs. In S3, the example of determining the unoccupied SPC1~SPC3 to execute task 1 on execution flow 1 is used for explanation. If there are other unoccupied SPCs in the hardware resource pool, execution flow 1 can also be scheduled to execute on the other three unoccupied SPCs.

[0126] S4, the distribution thread obtains task 2 from the task queue, and searches the flow task table for tasks that belong to the same logical flow as task 2. As a result, task 1 that belongs to the same logical flow as task 2 is found, and then the execution flow corresponding to task 1 is determined from the flow task table to be execution flow 1. This can be explained in two cases:

[0127] Case 1: If execution flow 1 is being executed, the distribution thread also distributes task 2 to execution flow 1. In this case, execution flow 1 still occupies SPC1~SPC3, and there is no need to schedule new computing resources for execution flow 1. Of course, if the computational amount of task 2 is relatively large, more computing resources can be allocated to execution flow 1, such as allocating SPC4 and SPC5 to execution flow 1, so that SPC1~SPC5 can execute task 2 together. It should be understood that if the computational amount of task 2 is very small, some of the SPCs in SPC1~SPC3 occupied by execution flow 1 can also be released to achieve flexible allocation of computing resources for tasks on the execution flow.

[0128] In case 2, if execution flow 1 has been completed, it means that execution flow 1 has been released, and the distribution thread re-determines an available execution flow from execution flow 1 to execution flow 4, for example, determines an available execution flow 2, and then distributes task 2 to execution flow 2. In this case, SPC6 to SPC10 can be determined from the unoccupied computing resources in the hardware resource pool according to the computing amount of task 2, and execution flow 2 can be scheduled to SPC6 to SPC10.

[0129] S5, the distribution thread obtains task 4 from the task queue and searches the flow task table for tasks that belong to the same logical flow as task 4. The result is: no task that belongs to the same logical flow 2 as task 4 is found. Then the distribution thread determines the unoccupied execution flow 3 from execution flows 1 to 4 and distributes task 1 to execution flow 3.

[0130] S6, the distribution thread determines SPC4~SPC5 and SPC11~SPC12 from the unoccupied computing resources in the hardware resource pool according to the computing amount of task 4, and schedules the execution flow 3 to SPC1~SPC3.

[0131] S7, the distribution thread obtains task 3 from the task queue, and searches the flow task table for tasks that belong to the same logical flow as task 3. As a result, task 1 and task 2 that belong to the same logical flow as task 3 are found, and then it is determined from the flow task table that the execution flow corresponding to tasks 1 and 2 is execution flow 1. The following description can also be divided into two cases. For details, please refer to the description of case 1 and case 2 in the aforementioned S4, which will not be repeated here.

[0132] S8, the distribution thread obtains task 5 from the task queue, and searches the flow task table for tasks that belong to the same logical flow as task 5. As a result, task 4 that belongs to the same logical flow 2 as task 5 is found, and then it is determined from the flow task table that the execution flow corresponding to task 4 is execution flow 3. The following description can also be divided into two cases. For details, please refer to the description of case 1 and case 2 in the aforementioned S4, which will not be repeated here.

[0133] Based on the same technical concept, the embodiment of the present application provides a flow scheduling device, such as Figure 6 The structural diagram of the flow scheduling device shown in FIG. 600 includes:

[0134] An acquisition unit 601 is used to acquire a first task from a first queue, where the first queue is used to store tasks issued by an application program;

[0135] A processing unit 602, configured to determine a first execution flow from a plurality of execution flows;

[0136] The scheduling unit 603 is used to distribute the first task to the first execution flow;

[0137] The processing unit 602 is further configured to determine, from the plurality of computing resources included in the hardware resource pool, a first computing resource that is not occupied by the second execution flow; wherein the second execution flow is an execution flow that is scheduled before the first execution flow and is being executed among the plurality of execution flows;

[0138] The scheduling unit 603 is further configured to schedule the first execution flow to be executed on the first computing resource.

[0139] In a possible implementation, the scheduling unit 603 is further used to: create a first event; insert the first event into the first execution flow, and store the first event in the second queue, and the execution order of the first event on the first execution flow is after the execution order of the first task.

[0140] In one possible implementation, the processing unit 602 is further used to: obtain the first event from the second queue and query the execution status of the first event, where the execution status of the first event is used to indicate the execution status of the first task; if the execution status of the first event is found to be completed, then release the first execution flow and the first computing resource.

[0141] In a possible implementation, the processing unit 602 is further configured to: if it is found that the execution status of the first event is a completed execution status, store the completed execution status of the first event in the third queue.

[0142] In one possible implementation, the stream scheduling device 600 also includes an interface calling unit 604, which is used to: call the first API to send the first task to the first queue; the processing unit 602 is also used to: if the status of the first event being executed is queried from the third queue, then determine that the first API call is completed.

[0143] In a possible implementation, the first API is a synchronous API, and the interface calling unit 604 is also used to: wait for the first API call to be completed, and then call the second API to send a second task to the first queue, where the second task is a task in the first queue that is arranged after the first task.

[0144] In one possible implementation, the first API is an asynchronous API, and the interface calling unit 604 is further used to: call the third API to create a logic flow, the logic flow includes multiple tasks, and the first task is any one of the multiple tasks; the processing unit 602 is specifically used to: if there is an execution flow that meets the first preset condition among the multiple execution flows, then determine the execution flow that meets the first preset condition as the first execution flow, the first preset condition includes being executed and distributed with a third task belonging to the logic flow, and the third task is a task arranged before the first task in the logic flow; or, if there is no execution flow that meets the first preset condition among the multiple execution flows, then determine the execution flow that has not been distributed with a task from the multiple execution flows and determine it as the first execution flow.

[0145] In a possible implementation, the interface calling unit 604 is further configured to call a stream synchronization API to wait for all tasks belonging to the logic flow to be completed.

[0146] Based on the same inventive concept, the present application provides a computer device, which includes a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip. When the processor chip executes the computer program, the method steps in any of the above method embodiments are implemented.

[0147] Based on the same technical concept, an embodiment of the present application provides a computer-readable storage medium, including computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer executes the method steps in any of the above method embodiments.

[0148] Based on the same technical concept, an embodiment of the present application provides a computer program product, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the method steps in any of the above method embodiments.

[0149] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, devices (equipment), systems, chips, computer-readable storage media, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware, which are collectively referred to as "modules" or "systems" herein.

[0150] The present application is described with reference to at least one of the following diagrams of the method, apparatus (device) or system of the present application: a flowchart, a block diagram. It should be understood that at least one of the following can be implemented by computer program instructions: each process in the flowchart, each box in the block diagram, and a combination of a process in the flowchart and a box in the block diagram.

[0151] These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce instructions for implementing at least one of the following: Figure 1 Devices or blocks that specify functions in a process or multiple processes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, the instruction device implementing at least one of the following: Figure 1 A function or a block specified in a process or multiple processes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing at least one of the following: steps for specifying a function in one or more processes in a flowchart, steps for specifying a function in one or more boxes in a block diagram.

[0154] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the claims of the present invention and their equivalents.

Claims

1. A flow scheduling method, characterized in that: Applied to a server, the server includes a host and a device, the device includes a hardware resource pool, and the method includes: Obtaining a first task from a first queue, where the first queue is used to store tasks issued by an application; Determine a first execution flow from a plurality of execution flows, and distribute the first task to the first execution flow; wherein, determining the first execution flow from a plurality of execution flows comprises: if there is an execution flow satisfying a first preset condition among the plurality of execution flows, then determining the execution flow satisfying the first preset condition as the first execution flow, the first preset condition comprising being executed and being distributed with a third task belonging to the same logical flow as the first task, the third task being a task arranged before the first task in the logical flow; or, if there is no execution flow satisfying the first preset condition among the plurality of execution flows, then determining an execution flow to which a task has not been distributed from the plurality of execution flows, and determining it as the first execution flow; the logical flow is a task sequence executed sequentially for the application layer; A first computing resource that is not occupied by a second execution flow is determined from the multiple computing resources included in the hardware resource pool, and the first execution flow is scheduled to be executed on the first computing resource; wherein the second execution flow is an execution flow among the multiple execution flows that was scheduled to the second computing resource in the hardware resource pool before the first execution flow and is being executed.

2. The method according to claim 1, characterized in that After distributing the first task to the first execution flow, the method further includes: Create the first event; The first event is inserted into the first execution flow, and the first event is stored in a second queue, wherein the execution order of the first event on the first execution flow is after the execution order of the first task.

3. The method according to claim 2, characterized in that The method further comprises: Obtaining the first event from the second queue, and querying an execution status of the first event, where the execution status of the first event is used to indicate an execution status of the first task; If it is found that the execution status of the first event is completed, the first execution flow and the first computing resource are released.

4. The method according to claim 3, characterized in that The method further comprises: If it is found that the execution status of the first event is a completed execution status, the completed execution status of the first event is stored in the third queue.

5. The method according to claim 4, characterized in that Before obtaining the first task from the first queue, the method further includes: Calling a first API to send the first task to the first queue; The method further comprises: If the status that the first event has been executed is queried from the third queue, it is determined that the first API call is completed.

6. The method according to claim 5, characterized in that The first API is a synchronous API, and after calling the first API to send the first task to the first queue, the method further includes: Wait for the first API call to be completed, and then call the second API to send a second task to the first queue, where the second task is a task in the first queue that is arranged after the first task.

7. The method according to claim 5, characterized in that The first API is an asynchronous API, and before calling the first API to send the first task to the first queue, the method further includes: A third API is called to create the logic flow, where the logic flow includes multiple tasks, and the first task is any one of the multiple tasks.

8. The method according to claim 7, characterized in that The method further comprises: The stream synchronization API is called to wait for all tasks belonging to the logic flow to complete execution.

9. A computer device comprising a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, characterized in that: When the processor chip executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The method comprises computer executable instructions, which, when executed on a computer, cause the computer to execute the steps of the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The computer program product stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute the steps of the method according to any one of claims 1 to 8.