Equipment queue execution method and device, computer equipment, medium and product

Abstract

The invention relates to an equipment queue execution method and device, computer equipment, a medium and a product. The method comprises the following steps: extracting a device queue interface of a parent kernel from a device queue cache, and after determining an execution sequence of the parent kernel and each sub-kernel, writing the device queue interface of each sub-kernel into the device queue cache to obtain an updated device queue cache; generating a starting instruction of the parent kernel according to the device queue interface of the parent kernel, and instructing an execution unit in the heterogeneous device to execute a kernel program of the parent kernel through the starting instruction of the parent kernel; reading a device queue interface of the sub-kernel through a kernel program of the parent kernel, and generating a starting instruction of the sub-kernel; and instructing an execution unit in the heterogeneous equipment to execute the kernel program of the sub-kernel through the starting instruction of the sub-kernel. Therefore, the kernel calling efficiency can be remarkably improved on the basis of not increasing hardware, and the method is suitable for scenes of frequently calling kernels and is easy to expand and use in different heterogeneous devices.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and apparatus for executing a device queue, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] The open computing language platform OpenCL (Open Computing Language) can write kernel functions, similar to C language, and also provides an application programming interface (Application Programming Interface, API) for controlling the platform. OpenCL is a cross-system heterogeneous platform that can support various operating systems such as Windows, Linux, and Android. The OpenCL compiler compiles the source code to generate the host program and kernel call program, thereby realizing the call of multiple kernels in the queue.

[0003] In traditional technology, when the host program starts the device program, the host program enters the queue and then executes the kernels in the queue in order. After each kernel is executed, it returns to the host program, and then the host program calls the next kernel in the queue.

[0004] However, the above traditional method is not suitable for the scenario of frequent kernel calls, and the kernel call efficiency is low. Summary of the invention

[0005] Based on this, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium and computer program product for executing a device queue that can be applicable to scenarios where the kernel is frequently called and can significantly improve the efficiency of kernel calls in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for executing a device queue, comprising:

[0007] Extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel;

[0008] According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain an updated device queue cache;

[0009] Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel;

[0010] Reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel;

[0011] The startup instruction of the sub-kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel.

[0012] In one embodiment, before extracting the device queue interface of the parent kernel from the device queue cache, the method further includes:

[0013] The driver program calls the OpenCL compiler to compile the source code and generate an executable program for the device queue;

[0014] Parsing the executable program of the device queue to determine whether the executable program of the device queue has a device queue feature, wherein the device queue feature refers to: allowing the kernel program of the parent kernel to add a child kernel in the device queue, and allowing the kernel program of the parent kernel to call the kernel program of the child kernel;

[0015] If the device queue feature exists, a device queue cache is constructed in the video memory, wherein the device queue cache is used to store the device queue interface.

[0016] In one embodiment, the device queue interface of the parent kernel contains information for starting the kernel, and the information for starting the kernel includes: address information of the child kernel, event information, kernel function, and a timestamp for executing the kernel program.

[0017] In one embodiment, the step of compiling source code by calling an OpenCL compiler through a driver to generate an executable program for a device queue includes:

[0018] According to the kernel function of the device queue, a device queue interface table is established, wherein the information stored in the device queue interface table includes: the size of the device queue interface in the device queue cache, the startup index, the shared memory size, the event offset, the global work group, the work group size, the global work offset, the work group offset, the parameter information, and any one or a combination of two or more of the shared memory offset;

[0019] According to the device queue interface table, collecting dependency information between different kernels;

[0020] Analyze the dependency information according to the event information, and update the dependency information according to the analysis result to obtain updated dependency information;

[0021] According to the updated dependency information, a dependency relationship between different kernels is established, wherein the dependency relationship includes: a relationship between a parent kernel and a child kernel, and a relationship between child kernels under the same parent kernel;

[0022] Generate an executable program for the device queue according to the dependency relationship between different kernels and the information stored in the device queue interface table.

[0023] In one embodiment, determining the execution order of the parent kernel and each child kernel under the parent kernel includes:

[0024] According to the dependency relationship between different kernels, the identifiers corresponding to the sub-kernels are set, wherein the identifiers are used to indicate the execution mode of the parent kernel and the sub-kernel, and the execution mode includes: the sub-kernel is executed in parallel with the parent kernel, the sub-kernel waits for the parent kernel to be executed before executing, and the sub-kernel work group under the parent kernel is executed before executing;

[0025] The execution order of the parent kernel and each sub-kernel under the parent kernel is determined according to the identifier.

[0026] In one embodiment, before reading the device queue interface of the child kernel from the updated device queue cache by the kernel program of the parent kernel, the method further includes:

[0027] During the execution of the kernel program of the parent kernel, the child kernel to be called is determined according to the event information.

[0028] In one of the embodiments, in the process of instructing the execution unit in the heterogeneous device to execute the kernel program of the parent kernel through the startup instruction of the parent kernel, the method further includes:

[0029] The device queue interface of the sub-kernel is updated, wherein the information in the device queue interface of the sub-kernel includes: the instruction position of the sub-kernel and the parameters of the sub-kernel.

[0030] In a second aspect, the present application further provides an execution device of a device queue, the device comprising:

[0031] A determination module, used for extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel;

[0032] A writing module, used for writing the device queue interface of each sub-kernel into the device queue cache according to the execution order of the parent kernel and each sub-kernel under the parent kernel, so as to obtain an updated device queue cache;

[0033] An execution module, configured to generate a startup instruction of the parent kernel according to a device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel;

[0034] A reading module, used for reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel;

[0035] The execution module is also used to instruct the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel through the startup instruction of the sub-kernel.

[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] Extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel;

[0038] According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain an updated device queue cache;

[0039] Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel;

[0040] Reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel;

[0041] The startup instruction of the sub-kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0043] Extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel;

[0044] According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain an updated device queue cache;

[0045] Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel;

[0046] Reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel;

[0047] The startup instruction of the sub-kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel.

[0048] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0049] Extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel;

[0050] According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain an updated device queue cache;

[0051] Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel;

[0052] Reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel;

[0053] The startup instruction of the sub-kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel.

[0054] The execution method, apparatus, computer equipment, computer-readable storage medium and computer program product of the device queue described above extract the device queue interface of the parent kernel from the device queue cache, and determine the execution order of the parent kernel and each sub-kernel under the parent kernel; thus, the cache can be used to store the device queue interface information of the kernel, and the execution order between the parent kernel and the sub-kernel can be flexibly changed, which is suitable for the scenario of frequent kernel calls. According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain the updated device queue cache; thus, the device queue interface information of the sub-kernel can be recorded by the cache, so that the kernel program of the parent kernel can freely schedule the sub-kernel in the future without returning to the main program, thereby improving the efficiency of kernel scheduling. According to the device queue interface of the parent kernel, the startup instruction of the parent kernel is generated, and the execution unit in the heterogeneous device is instructed to execute the kernel program of the parent kernel through the startup instruction of the parent kernel; thus, the heterogeneous device can quickly start the kernel program of the parent kernel. Through the kernel program of the parent kernel, the device queue interface of the child kernel is read from the updated device queue cache, and the startup instruction of the child kernel is generated according to the device queue interface of the child kernel; thus, the kernel program of the child kernel under it can be flexibly called by the parent kernel. The startup instruction of the child kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the child kernel. Thus, without adding hardware, the device queue feature of OpenCL2.0 is well adapted, which can be applied to the scenario of frequent kernel calls, and can significantly improve the kernel call efficiency, and is easy to be expanded and used in different heterogeneous devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 An application environment diagram of an execution method of a device queue in one embodiment;

[0057] Figure 2 A schematic diagram of a flow chart of a method for executing a device queue in one embodiment;

[0058] Figure 3 A schematic diagram of a kernel call situation when no sub-kernel is included in an embodiment;

[0059] Figure 4A schematic diagram of a kernel calling situation when a parent kernel and a child kernel are included in one embodiment;

[0060] Figure 5 A schematic diagram of the interaction between a driver, a device queue cache, and an execution unit of a heterogeneous device in one embodiment;

[0061] Figure 6 is a flow chart of an execution method of a device queue in another embodiment;

[0062] Figure 7 A schematic diagram of a flow chart of a compiler establishing a device queue interface table according to a kernel function of a device queue in one embodiment;

[0063] Figure 8 A schematic diagram of a process for establishing a dependency relationship for a device queue kernel in one embodiment;

[0064] Fig. 9 is a flowchart of a method for executing a device queue in yet another embodiment;

[0065] Fig.10 is a structural block diagram of an execution device of a device queue in one embodiment;

[0066] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] For example, Figure 1 FIG. 1 is an application environment diagram of an execution method of a device queue in an embodiment, such as Figure 1 As shown, the device queue is mainly used in heterogeneous devices. The heterogeneous program written by OpenCL adds corresponding functions and modules in the heterogeneous program compiler and heterogeneous device driver of the host device, so as to realize the function of the device queue. Exemplarily, the driver of the host device sends the relevant compilation configuration and starts the device-side program compilation. After the compiler compiles, the heterogeneous device instructions and heterogeneous device register configuration information are generated. Then, the heterogeneous device driver configures the heterogeneous device registers and sends execution instructions to the heterogeneous device. Finally, the heterogeneous device returns to the host program of the host device after the execution is completed.

[0069] In an exemplary embodiment, Figure 2 As shown, a method for executing a device queue is provided, and the method is applied to Figure 1The host device and the heterogeneous device in the example are used to illustrate, including the following steps 201 to 205. Among them:

[0070] Step 201 : extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel.

[0071] In this embodiment, the device queue cache is a cache module opened by the driver in the video memory, which is used to store the device queue interface (enqueue-entry). The device queue interface includes all the information of the boot kernel, which mainly includes three surfaces: the enqueuer surface, which mainly records the address information of the child kernel; the offset surface, which mainly records the information of events and kernel functions; and the profiling surface, which mainly records the timestamp of the execution kernel, which is used to count the execution time of the kernel.

[0072] The parent kernel is a concept similar to a parent node, that is, the kernel program of the parent kernel can call the kernel program of at least one child kernel. The device queue interface of the parent kernel contains information for starting the kernel, and the information for starting the kernel includes: the address information of the child kernel, event information, kernel function, and the timestamp of executing the kernel program.

[0073] For example, a new feature of device enqueue is introduced in OpenCL2.0, which allows kernel programs to start new kernel programs in the device queue, that is, the first kernel in the device queue can directly call the second kernel without handing over control to the host according to demand. At this time, the first kernel is called the parent kernel, the second kernel is called the child kernel, and the first kernel can continue to call the third kernel. Of course, the second kernel can also call the third kernel, and the third kernel is the child kernel of the second kernel. The parent kernel program can call a new kernel directly without waiting for the end of the parent kernel, or it can control the call of the kernel through events, and at the same time, it can call a record (marker) to show whether the end status of the event is successful. The characteristics of the device queue reduce the time for the first kernel to return from the device end to the host end, and the time for the host end to call the second kernel, which can improve the performance of the program for scenarios where kernels are frequently called.

[0074] The execution order of the parent kernel and each sub-kernel under the parent kernel refers to the order in which the execution units in the heterogeneous devices execute the kernel programs of the parent kernel and each sub-kernel in the time dimension.

[0075] For example, Figure 3 The kernel call without sub-kernels is shown in Figure 3 As shown, after the host program starts the device program, assuming that kernel P0 needs to be scheduled, after kernel P0 is executed (that is, the device program is finished), it is necessary to return to the host program, and then the host program starts the device program and calls other kernels (such as P1, P2, P3).

[0076] For example, Figure 4 The kernel call situation when including parent kernel and child kernel is shown in Figure 4 As shown, after the host program starts the device program, it is assumed that kernel P0 and sub-kernels C0, C1, and C2 under P0 need to be scheduled; during the execution of kernel P0 (or after the execution of kernel P0), the kernel program of kernel P0 can directly start C0 and / or C1 without returning to the host program. C2 is a sub-kernel of C1, that is, there are sub-kernels under the sub-kernel. In this case, sub-kernel C1 and sub-kernel C2 are regarded as a working group of a sub-kernel. In another way of description, C1 can also be regarded as the parent kernel of C2, that is, the sub-kernel under the parent kernel may also be the parent kernel of other sub-kernels. When kernel C2 is to be called, the kernel program of kernel C1 can start the kernel program of kernel C2.

[0077] In this embodiment, the function of the device queue of OpenCL2.0 is applied, that is, the kernel program is allowed to add a new kernel program to the device queue for execution, that is, the kernel program P0 on the device side is allowed to call the kernel programs C0 and C1 for execution according to the program needs, without having to return the control to the host side first, and then the host controls the execution of the kernel program. This method can reduce the time for the kernel program P0 to return to the host and the time for the host to call the kernel programs C0 and C1, which can improve the performance of the program for application scenarios where the kernel program is frequently called; and reduce the amount of code configured on the host side of the heterogeneous program, making the code more compact and more readable.

[0078] Step 202: write the device queue interface of each sub-kernel into the device queue cache according to the execution order of the parent kernel and each sub-kernel under the parent kernel, and obtain an updated device queue cache.

[0079] In this embodiment, when the kernel program of the parent kernel is started, the device queue interface of each child kernel under the parent kernel can be obtained, and the kernel program of the parent kernel can be called to add the device queue interface of each child kernel to the device queue cache.

[0080] It should be noted that this embodiment does not limit the update timing of the device queue cache, that is, the device queue cache can be flexibly and dynamically updated.

[0081] Step 203: Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct the execution unit in the heterogeneous device to execute the kernel program of the parent kernel through the startup instruction of the parent kernel.

[0082] In this embodiment, the device queue interface of the parent kernel contains the startup information of all kernels. After the driver configures the information according to the device queue interface of the parent kernel, it sends a startup instruction for starting the parent kernel to the heterogeneous device. The execution unit of the heterogeneous device executes the kernel program of the parent kernel according to the received startup instruction.

[0083] Step 204 , reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel.

[0084] In this embodiment, the child kernel can be called by the enqueue_kernel function (high-level language) of the parent kernel to read the device queue interface of the child kernel from the updated device queue cache. After configuring information according to the device queue interface of the child kernel, the driver sends a startup instruction for starting the child kernel to the heterogeneous device.

[0085] Step 205: Instruct the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel through the startup instruction of the sub-kernel.

[0086] In this embodiment, the execution unit of the heterogeneous device executes the kernel program of the sub-kernel according to the received startup instruction.

[0087] For example, Figure 5 The interactive relationship between the driver, device queue cache, and execution unit of heterogeneous devices is shown in the figure. Step 1: The driver configures the device queue cache; Step 2: The driver sends a startup instruction to the heterogeneous device to start the kernel; Step 3: The execution unit (UE) in the heterogeneous device reads the startup instruction and executes the corresponding kernel program; Step 4: The execution unit updates the address information (entry-surface) of the sub-kernel; Step 5: The driver reads the kerenl information (the called sub-kernel information) on the device queue cache; Step 6: The driver sends a startup instruction to the heterogeneous device to start the kernel (here is the called sub-kernel) until all sub-kernel programs are executed.

[0088] In the above-mentioned device queue execution method, the device queue interface of the parent kernel is extracted from the device queue cache, and the execution order of the parent kernel and each sub-kernel under the parent kernel is determined; thus, the cache can be used to store the kernel's device queue interface information, and the execution order between the parent kernel and the sub-kernel can be flexibly changed, which is suitable for the scenario of frequent kernel calls. According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain an updated device queue cache; thus, the device queue interface information of the sub-kernel can be recorded by the cache, so that the kernel program of the parent kernel can freely schedule the sub-kernel in the future without returning to the main program, thereby improving the efficiency of kernel scheduling. According to the device queue interface of the parent kernel, the startup instruction of the parent kernel is generated, and the execution unit in the heterogeneous device is instructed to execute the kernel program of the parent kernel through the startup instruction of the parent kernel; thus, the heterogeneous device can quickly start the kernel program of the parent kernel. Through the kernel program of the parent kernel, the device queue interface of the child kernel is read from the updated device queue cache, and the startup instruction of the child kernel is generated according to the device queue interface of the child kernel; thus, the kernel program of the child kernel under it can be flexibly called by the parent kernel. The startup instruction of the child kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the child kernel. Thus, without adding hardware, the device queue feature of OpenCL2.0 is well adapted, which can be applied to the scenario of frequent kernel calls, and can significantly improve the kernel call efficiency, and is easy to be expanded and used in different heterogeneous devices.

[0089] In another exemplary embodiment, Figure 6 As shown, another execution method of a device queue is provided, which is applied to Figure 1 The host device and the heterogeneous device in the example are used to illustrate, including the following steps 601 to 608. Among them:

[0090] Step 601: The driver program calls the OpenCL compiler to compile source code to generate an executable program for the device queue.

[0091] For example, Figure 7 The process of the compiler establishing a device queue interface table according to the kernel function of the device queue is shown in the figure. Specifically, the device queue interface table is established according to the kernel function of the device queue, wherein the information stored in the device queue interface table includes: the size of the device queue interface in the device queue cache, the startup index, the shared memory size, the event offset, the global work group, the work group size, the global work offset, the work group offset, the parameter information, and the shared memory offset, any one or a combination of two or more thereof.

[0092] In this embodiment, the cache size identifies the memory size occupied by the current device queue interface in the device queue cache. Each kernel function has a device queue interface, but since the parameter information and shared memory offset in the device queue interface are of unfixed size, the cache size is required to distinguish the ranges of different device queue interfaces.

[0093] For example, Figure 8 The process of establishing dependencies between device queue kernels is shown in Figure 8 As shown, first, according to the device queue interface table, the dependency information between different kernels is collected; according to the event information, the dependency information is analyzed, and according to the analysis result, the dependency information is updated to obtain updated dependency information; according to the updated dependency information, the dependency relationship between different kernels is established.

[0094] The dependency relationship includes: the relationship between the parent kernel and the child kernel, and the relationship between the child kernels under the same parent kernel; and generating the executable program of the device queue according to the dependency relationship between different kernels and the information stored in the device queue interface table.

[0095] Step 602: parse the executable program of the device queue to determine whether the executable program of the device queue has a device queue characteristic.

[0096] The device queue feature refers to: allowing the kernel program of the parent kernel to add a child kernel in the device queue, and allowing the kernel program of the parent kernel to call the kernel program of the child kernel.

[0097] In this embodiment, the source code program string is called as a parameter of the clBuildProgram API, so that the driver calls the OpenCL compiler to compile the source code and generate a binary executable program of the device queue. The driver parses the generated binary executable program and detects whether the binary executable program has the characteristics of the device queue.

[0098] Step 603: If the device queue feature exists, then a device queue cache is constructed in the video memory.

[0099] The device queue buffer is used to store the device queue interface.

[0100] In this embodiment, in addition to opening up a device queue cache in the video memory, a device queue cache module may be added to specifically store the device queue interface.

[0101] Step 604: extract the device queue interface of the parent kernel from the device queue cache, and determine the execution order of the parent kernel and each child kernel under the parent kernel.

[0102] In this embodiment, the identifier corresponding to each sub-kernel can be set according to the dependency relationship between different kernels, wherein the identifier is used to indicate the execution mode of the parent kernel and the sub-kernel, and the execution mode includes: the sub-kernel is executed in parallel with the parent kernel, the sub-kernel waits for the parent kernel to be executed before executing, and the sub-kernel work group under the parent kernel is executed before executing; the execution order of the parent kernel and each sub-kernel under the parent kernel is determined according to the identifier.

[0103] Exemplarily, a kernel program starts a new kernel program and uses a flag to determine when to execute a child kernel. The flags include the following three types: the first type, CLK_ENQUEUE_FLAGS_NO_WAIT: the child kernel does not need to wait for the parent kernel to complete execution before executing, that is, the child kernel and the parent kernel execute at the same time; the second type, CLK_ENQUEUE_FLAGS_WAIT_KERNEL: the child kernel needs to wait for the parent kernel to complete execution before executing; the third type, CLK_ENQUEUE_FLAGS_WAIT_WORK_GROUP: the child kernel will wait for the work group of the child kernel queued in the parent kernel to complete execution before executing.

[0104] It should be noted that after the child kernel is queued to the parent kernel, if the host detects that the parent kernel is executed successfully, it means that both the parent kernel and the child kernel are successfully executed. If the child kernel fails to execute, the host obtains the error code of the execution status failure of the parent kernel.

[0105] Step 605 , writing the device queue interface of each sub-kernel into the device queue cache according to the execution order of the parent kernel and each sub-kernel under the parent kernel, to obtain an updated device queue cache.

[0106] Step 606: Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct the execution unit in the heterogeneous device to execute the kernel program of the parent kernel through the startup instruction of the parent kernel.

[0107] Step 607 , reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel.

[0108] Step 608: Instruct the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel through the startup instruction of the sub-kernel.

[0109] In this embodiment, the specific implementation process and technical effects of steps 605 to 608 are shown in Figure 2 The descriptions of steps 202 to 205 in the illustrated method embodiment are not repeated here.

[0110] In this embodiment, there is no need to add hardware logic in heterogeneous computing devices (GPU, NPU, TPU), and only software is used to implement device queue performance, which reduces the hardware design cost. Since it is a pure software implementation, it can be extended to different heterogeneous devices, which facilitates continuous iterative optimization as needed.

[0111] In another exemplary embodiment, Fig. 9 As shown, another method for executing a device queue is provided, and the method is applied to Figure 1 The host device and the heterogeneous device in the example are used to illustrate, including the following steps 901 to 907. Among them:

[0112] Step 901: extract the device queue interface of the parent kernel from the device queue cache, and determine the execution order of the parent kernel and each child kernel under the parent kernel.

[0113] Step 902: write the device queue interface of each sub-kernel into the device queue cache according to the execution order of the parent kernel and each sub-kernel under the parent kernel, and obtain an updated device queue cache.

[0114] Step 903: Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct the execution unit in the heterogeneous device to execute the kernel program of the parent kernel through the startup instruction of the parent kernel.

[0115] For the specific implementation process and technical effects of steps 901 to 903 in this embodiment, please refer to Figure 2 The descriptions of steps 201 to 203 in the illustrated method embodiment are not repeated here.

[0116] Step 904: during the execution of the kernel program of the parent kernel, determine the child kernel to be called according to the event information.

[0117] In this embodiment, the parent kernel program can directly start calling the new kernel without waiting for the end of the parent kernel, or can control the calling of the kernel through events, and can call a record to display whether the end status of the event is successful.

[0118] Step 905 , reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel.

[0119] Step 906: Instruct the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel through the startup instruction of the sub-kernel.

[0120] For the specific implementation process and technical effects of steps 905 to 906 in this embodiment, please refer to Figure 2The descriptions of steps 204 to 205 in the illustrated method embodiment are not repeated here.

[0121] Step 907: Update the device queue interface of the sub-kernel.

[0122] The information in the device queue interface of the sub-kernel includes: the instruction position of the sub-kernel and the parameters of the sub-kernel.

[0123] In this embodiment, during the execution of the kernel program of the parent kernel, the information of the child kernel (enqueue-entry, including the instruction location of the child kernel, the parameters of the child kernel, etc.) can be updated, and the information of the child kernel can be written into the device queue cache; after the driver receives the parent kernel execution completion mark, it reads the information of the child kernel in the device queue cache, and starts the kernel program of the child kernel according to the information in the device queue interface of the child kernel until the execution of the kernel programs of all the child kernels is completed.

[0124] This embodiment can process the parent kernel and the child kernel in parallel, and dynamically update the information of the child kernel during the processing of the parent kernel, so that the child kernel called by the kernel program of the parent kernel can be continuously updated, which greatly simplifies the calling process. It is particularly suitable for complex multi-core heterogeneous devices and has strong scalability.

[0125] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0126] Based on the same inventive concept, the embodiment of the present application also provides a device queue execution device for implementing the device queue execution method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the execution device of one or more device queues provided below can refer to the limitations of the device queue execution method above, and will not be repeated here.

[0127] In an exemplary embodiment, Fig.10As shown, a device queue execution apparatus is provided, comprising: a determination module 1001, a writing module 1002, an execution module 1003 and a reading module 1004, wherein:

[0128] The determination module 1001 is used to extract the device queue interface of the parent kernel from the device queue cache, and determine the execution order of the parent kernel and each child kernel under the parent kernel;

[0129] A writing module 1002 is used to write the device queue interface of each sub-kernel into the device queue cache according to the execution order of the parent kernel and each sub-kernel under the parent kernel, so as to obtain an updated device queue cache;

[0130] An execution module 1003 is used to generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel;

[0131] The reading module 1004 is used to read the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generate a startup instruction of the child kernel according to the device queue interface of the child kernel;

[0132] The execution module 1003 is further used to instruct the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel through the startup instruction of the sub-kernel.

[0133] Exemplarily, the above device may further include:

[0134] An execution program generation module 1005 is used to generate an executable program of the device queue by calling an OpenCL compiler to compile source code through a driver before extracting the device queue interface of the parent kernel from the device queue cache;

[0135] The parsing module 1006 is used to parse the executable program of the device queue to determine whether the executable program of the device queue has a device queue feature, wherein the device queue feature refers to: allowing the kernel program of the parent kernel to add a child kernel in the device queue, and allowing the kernel program of the parent kernel to call the kernel program of the child kernel;

[0136] The cache construction module 1007 is used to construct a device queue cache in the video memory when the device queue feature exists, wherein the device queue cache is used to store the device queue interface.

[0137] Exemplarily, the device queue interface of the parent kernel contains information for starting the kernel, and the information for starting the kernel includes: address information of the child kernel, event information, kernel function, and a timestamp of executing the kernel program.

[0138] Exemplarily, the execution program generation module 1005 is specifically used to: establish a device queue interface table according to the kernel function of the device queue, wherein the information stored in the device queue interface table includes: the size of the device queue interface in the device queue cache, the startup index, the shared memory size, the event offset, the global work group, the work group size, the global work offset, the work group offset, the parameter information, and any one or a combination of two or more of them; collect dependency information between different kernels according to the device queue interface table; analyze the dependency information according to the event information, and update the dependency information according to the analysis result to obtain updated dependency information; establish dependency relationships between different kernels according to the updated dependency information, wherein the dependency relationships include: the relationship between the parent kernel and the child kernel, and the relationship between the child kernels under the same parent kernel; generate an executable program for the device queue according to the dependency relationships between different kernels and the information stored in the device queue interface table.

[0139] Exemplarily, the determination module 1001 is specifically used to: set an identifier corresponding to each sub-kernel according to the dependency relationship between different kernels, wherein the identifier is used to indicate the execution mode of the parent kernel and the sub-kernel, and the execution mode includes: the sub-kernel is executed in parallel with the parent kernel, the sub-kernel waits for the parent kernel to be executed before executing, and the sub-kernel waits for the sub-kernel work group under the parent kernel to be executed before executing; determine the execution order of the parent kernel and each sub-kernel under the parent kernel according to the identifier.

[0140] Exemplarily, the determination module 1001 is also used to: determine the child kernel that needs to be called according to event information during the execution of the kernel program of the parent kernel before reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel.

[0141] Exemplarily, the above-mentioned device also includes: an update module 1008, which is used to update the device queue interface of the child kernel in the process of instructing the execution unit in the heterogeneous device to execute the kernel program of the parent kernel through the startup instruction of the parent kernel, wherein the information in the device queue interface of the child kernel includes: the instruction position of the child kernel and the parameters of the child kernel.

[0142] Each module in the execution device of the above-mentioned device queue can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0143] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the device queue. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for executing a device queue is implemented.

[0144] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0147] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0149] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0150] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0151] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for executing a device queue, characterized in that: The method comprises: Extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel; According to the execution order of the parent kernel and each sub-kernel under the parent kernel, the device queue interface of each sub-kernel is written into the device queue cache to obtain an updated device queue cache; Generate a startup instruction of the parent kernel according to the device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel; Reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel; The startup instruction of the sub-kernel instructs the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel.

2. The method according to claim 1, characterized in that Before extracting the device queue interface of the parent kernel from the device queue cache, the method further includes: The driver program calls the OpenCL compiler to compile the source code and generate an executable program for the device queue; Parsing the executable program of the device queue to determine whether the executable program of the device queue has a device queue feature, wherein the device queue feature refers to: allowing the kernel program of the parent kernel to add a child kernel in the device queue, and allowing the kernel program of the parent kernel to call the kernel program of the child kernel; If the device queue feature exists, a device queue cache is constructed in the video memory, wherein the device queue cache is used to store the device queue interface.

3. The method according to claim 1, characterized in that The device queue interface of the parent kernel contains information for starting the kernel, and the information for starting the kernel includes: address information of the child kernel, event information, kernel function, and a timestamp for executing the kernel program.

4. The method according to claim 2, characterized in that: The method of calling the OpenCL compiler to compile the source code through the driver to generate an executable program of the device queue includes: According to the kernel function of the device queue, a device queue interface table is established, wherein the information stored in the device queue interface table includes: the size of the device queue interface in the device queue cache, the startup index, the shared memory size, the event offset, the global work group, the work group size, the global work offset, the work group offset, the parameter information, and any one or a combination of two or more of the shared memory offset; According to the device queue interface table, collecting dependency information between different kernels; Analyze the dependency information according to the event information, and update the dependency information according to the analysis result to obtain updated dependency information; According to the updated dependency information, a dependency relationship between different kernels is established, wherein the dependency relationship includes: a relationship between a parent kernel and a child kernel, and a relationship between child kernels under the same parent kernel; Generate an executable program for the device queue according to the dependency relationship between different kernels and the information stored in the device queue interface table.

5. The method according to claim 4, characterized in that Determining the execution order of the parent kernel and each child kernel under the parent kernel includes: According to the dependency relationship between different kernels, the identifiers corresponding to the sub-kernels are set, wherein the identifiers are used to indicate the execution mode of the parent kernel and the sub-kernel, and the execution mode includes: the sub-kernel is executed in parallel with the parent kernel, the sub-kernel waits for the parent kernel to be executed before executing, and the sub-kernel work group under the parent kernel is executed before executing; The execution order of the parent kernel and each sub-kernel under the parent kernel is determined according to the identifier.

6. The method according to any one of claims 1 to 5, characterized in that: Before reading the device queue interface of the child kernel from the updated device queue cache by the kernel program of the parent kernel, the method further includes: During the execution of the kernel program of the parent kernel, the child kernel to be called is determined according to the event information.

7. The method according to any one of claims 1 to 5, characterized in that: In the process of instructing the execution unit in the heterogeneous device to execute the kernel program of the parent kernel through the startup instruction of the parent kernel, the method further includes: The device queue interface of the sub-kernel is updated, wherein the information in the device queue interface of the sub-kernel includes: the instruction position of the sub-kernel and the parameters of the sub-kernel.

8. An execution device for a device queue, characterized in that: The device comprises: A determination module, used for extracting the device queue interface of the parent kernel from the device queue cache, and determining the execution order of the parent kernel and each child kernel under the parent kernel; A writing module, used for writing the device queue interface of each sub-kernel into the device queue cache according to the execution order of the parent kernel and each sub-kernel under the parent kernel, so as to obtain an updated device queue cache; An execution module, configured to generate a startup instruction of the parent kernel according to a device queue interface of the parent kernel, and instruct an execution unit in a heterogeneous device to execute a kernel program of the parent kernel through the startup instruction of the parent kernel; A reading module, used for reading the device queue interface of the child kernel from the updated device queue cache through the kernel program of the parent kernel, and generating a startup instruction of the child kernel according to the device queue interface of the child kernel; The execution module is also used to instruct the execution unit in the heterogeneous device to execute the kernel program of the sub-kernel through the startup instruction of the sub-kernel.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images