A task processing method and device

Abstract

The present application provides a task processing method and device. The method includes: when the kernel-mode scheduler receives a first task message, it sends the first task message to the kernel-mode driver, and the kernel-mode driver sends the first task message to the first type of hardware unit; the kernel-mode scheduler receives a second task message returned by the kernel-mode driver; if the task type is a kernel-mode bound call, the kernel-mode scheduler obtains the working order of multiple hardware units from the second task message, and determines whether the first type of hardware unit is the last hardware unit based on the working order; if so, the kernel-mode scheduler stores the processed data in a specified storage medium; if not, the kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first type of hardware unit. Through the technical solution of the present application, resource overhead can be saved.

Description

Technical Field

[0001] This application relates to the field of edge computing technology, and in particular, to a task processing method and device. Background Art

[0002] The operating system is the core for managing computer hardware resources and software resources, responsible for allocating and scheduling computer resources, and providing various services to support the operation of application programs. In the operating system, there are user mode and kernel mode, and the user mode and kernel mode define the interaction mode between application programs and the operating system.

[0003] In the operating system, the kernel mode is the state for running operating system programs and operating hardware, with the highest privilege. The kernel mode is also called the kernel mode. In this mode, all instructions can be executed, all memory addresses can be accessed, all interrupt requests can be responded to, and hardware events and system calls can be processed. The user mode is the state for running user programs, and its privilege is restricted. The user mode is also called the user mode. In this mode, the application program has limited access rights to system resources, can only run within a specific space delimited by the operating system, cannot directly access hardware devices, and all accesses to hardware devices need to be carried out through the operating system.

[0004] The switching between the kernel mode and the user mode is mainly achieved through interrupts and system calls. When a process in the user mode needs to execute code in the kernel mode, it will enter the kernel mode through a system call. This process involves saving the context of the user mode (such as stack information) to the kernel mode, and then loading the new context of the kernel mode to start execution. After the code in the kernel mode is executed, the control right will be returned to the user mode, and the original context will be restored to complete the switching from the user mode to the kernel mode. Similarly, when an interrupt occurs, it will automatically switch to the kernel mode, execute the corresponding interrupt handler, and then return to the user mode after processing.

[0005] During the task processing, it is necessary to repeatedly switch between the user mode and the kernel mode, and the repeated switching between the user mode and the kernel mode will generate a large amount of CPU overhead and consume a large amount of resources. Summary of the Invention

[0006] This application provides a task processing method, which is applied to an electronic device including a processor and multiple hardware units. The multiple hardware units include a first type of hardware unit. The first type of hardware unit corresponds to a kernel mode scheduler and a kernel mode driver in the kernel mode of the processor. The method includes:

[0007] When the kernel mode scheduler receives a first task message, it sends the first task message to the kernel mode driver, and the kernel mode driver sends the first task message to the first type of hardware unit;

[0008] The kernel-mode scheduler receives a second task message returned by the kernel-mode driver. The second task message includes processed data and a task type. The processed data is obtained by the first type of hardware unit after processing the data to be processed carried in the first task message.

[0009] If the task type is a kernel-mode bound call, the kernel-mode scheduler obtains the working order of multiple hardware units from the second task message, and determines whether the first type of hardware unit is the last hardware unit based on the working order. The kernel-mode bound call means calling the kernel-mode drivers of each hardware unit in sequence based on the working order to call the hardware units to work.

[0010] If so, the kernel-mode scheduler stores the processed data in a specified storage medium.

[0011] If not, the kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first type of hardware unit.

[0012] This application provides a task processing method applied to an electronic device. The electronic device includes a processor and multiple hardware units. For each hardware unit, the hardware unit corresponds to a kernel-mode driver in the kernel mode of the processor. For each kernel-mode driver, the method includes:

[0013] When the kernel-mode driver receives a first task message, it sends the first task message to the hardware unit corresponding to the kernel-mode driver, and receives a second task message returned by the hardware unit. The second task message includes processed data. The processed data is obtained by the hardware unit after processing the data to be processed carried in the first task message.

[0014] The kernel-mode driver obtains the working order of multiple hardware units from the second task message, and determines whether the hardware unit is the last hardware unit based on the working order.

[0015] If so, the kernel-mode driver stores the processed data in a specified storage medium.

[0016] If not, the kernel-mode driver determines the next hardware unit of the hardware unit based on the working order, and sends the second task message to the kernel-mode driver corresponding to the next hardware unit.

[0017] The present application provides a task processing method, which is applied to an electronic device including a processor and multiple hardware units. For each hardware unit, the hardware unit corresponds to a kernel-mode driver in the kernel mode of the processor and a user-mode API in the user mode of the processor. For each kernel-mode driver, the method includes:

[0018] When the kernel-mode driver receives a first task message, it sends the first task message to the hardware unit corresponding to the kernel-mode driver, and receives a second task message returned by the hardware unit. The second task message includes processed data; wherein, the processed data is obtained after the hardware unit processes the to-be-processed data carried by the first task message;

[0019] The kernel-mode driver obtains a first identity identifier corresponding to the requested hardware unit from the second task message; wherein, when a hardware unit expects another hardware unit to jointly complete the same task, then this one hardware unit is the requested hardware unit, and this other hardware unit is the assisting hardware unit;

[0020] If the second identity identifier corresponding to the kernel-mode driver is different from the first identity identifier, then the kernel-mode driver sends the second task message to the kernel-mode driver corresponding to the requested hardware unit;

[0021] If the second identity identifier is the same as the first identity identifier, then the kernel-mode driver determines whether the assisting hardware unit has completed task processing; if so, it sends the second task message to the user-mode API; if not, it sends the second task message to the kernel-mode driver corresponding to the assisting hardware unit.

[0022] The present application provides an electronic device. The electronic device includes a processor and multiple hardware units. The multiple hardware units include a first type of hardware unit. The electronic device further includes a kernel-mode scheduler and a kernel-mode driver corresponding to the first type of hardware unit in the kernel mode of the processor; wherein:

[0023] The kernel-mode scheduler is configured to, when receiving a first task message, send the first task message to the kernel-mode driver. The first task message includes to-be-processed data;

[0024] The kernel-mode driver is configured to send the first task message to the first type of hardware unit;

[0025] The first type of hardware unit is configured to process the to-be-processed data to obtain processed data, and send a second task message to the kernel-mode driver. The second task message includes processed data and a task type;

[0026] The kernel state driver is further used to send the second task message to the kernel state scheduler;

[0027] The kernel-mode scheduler is further configured to receive the second task message; if the task type is a kernel-mode binding call, obtain the working order of multiple hardware units from the second task message, and determine whether the first-type hardware unit is the last hardware unit based on the working order; if so, store the processed data in a designated storage medium; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit;

[0028] The kernel-mode binding call indicates calling the kernel-mode driver of each hardware unit in sequence based on the working order to call the hardware unit to work.

[0029] The present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned task processing method is implemented.

[0030] The present application provides an electronic device, which includes: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; wherein the processor is used to execute the machine-executable instructions to implement the above-mentioned task processing method.

[0031] The present application provides a machine-readable storage medium, which stores machine-executable instructions that can be executed by a processor; wherein the processor is used to execute the machine-executable instructions, and implement the above-mentioned task processing method when the machine-executable instructions are executed.

[0032] It can be seen from the above technical solutions that in the embodiments of the present application, by deploying a kernel state scheduler and a kernel state driver in the kernel state of the processor, during the task processing process, the task processing is completed in the kernel state without the need to repeatedly switch between the user state and the kernel state, thereby saving CPU overhead, saving resource overhead, and being able to save switching overhead between the user state and the kernel state (such as time overhead and resource overhead). By completing task processing in the kernel state, the requirements of high real-time performance of the task can be met, and the requirements of mutual calls between hardware units for collaborative work can be met. By deploying a kernel state scheduler, the hardware unit simultaneously supports user state successive calls, kernel state binding calls, and kernel state successive calls, so that the hardware unit has both flexibility and efficiency, can efficiently respond to real-time tasks, and respond to occasional tasks with flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A is a flowchart of a task processing method in one embodiment of the present application;

[0034] Figure 1B is a schematic flowchart of a task processing method in an embodiment of the present application;

[0035] Figure 1C is a schematic flowchart of a task processing method in an embodiment of the present application;

[0036] Figure 2 is a schematic diagram of an application scenario of a CPU and multiple hardware units in an embodiment of the present application;

[0037] Figure 3 is a schematic diagram of the collaborative work among multiple hardware units in an embodiment of the present application;

[0038] Figure 4 is a schematic diagram of the collaborative work among multiple hardware units in an embodiment of the present application;

[0039] Figure 5 is a schematic diagram of the collaborative work among multiple hardware units in an embodiment of the present application;

[0040] Figure 6 is a schematic diagram of the successive calls for the kernel mode in an embodiment of the present application;

[0041] Figure 7 is a schematic diagram of the collaborative work among multiple hardware units in an embodiment of the present application;

[0042] Figure 8 is a hardware structure diagram of an electronic device in an embodiment of the present application. Detailed implementation manners

[0043] In an embodiment of the present application, a task processing method is proposed. This method can be applied to an electronic device (i.e., a device for implementing task processing), and the electronic device may include a processor and multiple hardware units. The multiple hardware units include a first type of hardware unit, and the first type of hardware unit corresponds to a kernel mode scheduler and a kernel mode driver in the kernel mode of the processor. Refer to Figure 1A As shown, it is a schematic flowchart of this method, and this method includes:

[0044] Step 101: When the kernel mode scheduler receives a first task message, it sends the first task message to the kernel mode driver, and the kernel mode driver sends the first task message to the first type of hardware unit.

[0045] Step 102: The kernel mode scheduler receives a second task message returned by the kernel mode driver. The second task message may include processed data and a task type; wherein, the processed data may be obtained after the first type of hardware unit processes the to-be-processed data carried in the first task message.

[0046] Step 103: If the task type is a kernel-mode bound call, the kernel-mode scheduler obtains the working order of multiple hardware units from the second task message, and determines whether the first type of hardware unit is the last hardware unit based on this working order. Exemplarily, a kernel-mode bound call means calling the kernel-mode drivers of each hardware unit in sequence based on the working order to call the hardware units to work.

[0047] If so, step 104 can be executed; if not, step 105 can be executed.

[0048] Step 104: The kernel-mode scheduler stores the processed data in a specified storage medium.

[0049] Step 105: The kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first type of hardware unit.

[0050] Exemplarily, the first type of hardware unit corresponds to a user-mode API in the user mode of the processor. After the kernel-mode scheduler receives the second task message returned by the kernel-mode driver, if the task type is a user-mode sequential call, the kernel-mode scheduler sends the second task message to the user-mode API, so that the application can obtain the second task message from the user-mode API and obtain the processed data from the second task message; where a user-mode sequential call means calling the hardware unit to work by calling the user-mode API of the hardware unit.

[0051] Exemplarily, after the kernel-mode scheduler receives the second task message returned by the kernel-mode driver, if the task type is a kernel-mode sequential call, the kernel-mode scheduler obtains the first identity identifier corresponding to the requested hardware unit from the second task message; where a kernel-mode sequential call means calling the hardware unit to work by calling the kernel-mode driver of the hardware unit. If the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, the kernel-mode scheduler determines that the first type of hardware unit is an auxiliary hardware unit, and sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the requested hardware unit.

[0052] Wherein, when a hardware unit expects another hardware unit to jointly complete the same task, the one hardware unit is the requested hardware unit, and the other hardware unit is the auxiliary hardware unit.

[0053] Exemplarily, after the kernel-mode scheduler obtains the first identity identifier corresponding to the requested hardware unit from the second task message, if the second identity identifier is the same as the first identity identifier, the kernel-mode scheduler determines that the first type of hardware unit is the requested hardware unit and determines whether the auxiliary hardware unit has completed task processing; wherein, if the auxiliary hardware unit has returned a task message, the auxiliary hardware unit has completed task processing; if the auxiliary hardware unit has not returned a task message, the auxiliary hardware unit has not completed task processing.

[0054] If so, the kernel-mode scheduler can send the second task message to the user-mode API so that the application can obtain the second task message from the user-mode API; if not, the kernel-mode scheduler can send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the auxiliary hardware unit.

[0055] Exemplarily, when the kernel-mode scheduler receives multiple first task messages, for each first task message, the kernel-mode scheduler can obtain the task priority of the first task message from the first task message; based on the task priorities of each first task message, the kernel-mode scheduler selects the first task message with the highest task priority and sends the first task message with the highest task priority to the kernel-mode driver.

[0056] Exemplarily, the multiple hardware units may further include a second type of hardware unit. The second type of hardware unit corresponds to a kernel-mode driver in the kernel mode of the processor and corresponds to a user-mode API in the user mode of the processor. Among them, when sending the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first type of hardware unit, if the next hardware unit is the first type of hardware unit, the kernel-mode scheduler sends the second task message to the kernel-mode scheduler corresponding to the next hardware unit; if the next hardware unit is the second type of hardware unit, the kernel-mode scheduler sends the second task message to the kernel-mode driver corresponding to the next hardware unit. Among them, the first type of hardware unit may include, but is not limited to, NPU (Neural-network Processing Unit) units and / or GPU (Graphics Processing Unit) units, and the second type of hardware unit may include, but is not limited to, at least one of ASIC (Application Specific Intergrated Circuits) units, FPGA (Field Programmable Gate Array) units, and CPLD (Complex Programmable logic device) units.

[0057] As can be seen from the above technical solutions, in the embodiments of the present application, by deploying a kernel-mode scheduler and a kernel-mode driver in the kernel mode of the processor, during the task processing, the task is processed in the kernel mode without the need to repeatedly switch between the user mode and the kernel mode, thereby saving CPU overhead, saving resource overhead, and being able to save the switching overhead (such as time overhead and resource overhead) between the user mode and the kernel mode. By processing the task in the kernel mode, the requirements of high real-time tasks can be met, and the need for mutual calls between each hardware unit to work together can be satisfied. By deploying the kernel-mode scheduler, the hardware unit supports sequential calls in the user mode, bound calls in the kernel mode, and sequential calls in the kernel mode at the same time, making the hardware unit both flexible and efficient, being able to efficiently respond to real-time tasks and flexibly respond to sporadic tasks.

[0058] In the embodiments of the present application, a task processing method is proposed. This method can be applied to an electronic device (i.e., a device for implementing task processing), and the electronic device may include a processor and multiple hardware units. For each hardware unit, there is a corresponding kernel-mode driver in the kernel mode of the processor. Refer to Figure 1B As shown, it is a flowchart of the task processing method, and the task processing method may include:

[0059] Step 111: For each kernel-mode driver, when the kernel-mode driver receives a first task message, it sends the first task message to the hardware unit corresponding to this kernel-mode driver, and receives a second task message returned by the hardware unit. The second task message may include processed data; where the processed data is obtained after the hardware unit processes the to-be-processed data carried in the first task message.

[0060] Step 112: The kernel-mode driver obtains the working order of multiple hardware units from the second task message, and determines whether the hardware unit is the last hardware unit based on this working order.

[0061] If so, step 113 can be executed; if not, step 114 can be executed.

[0062] Step 113: The kernel-mode driver stores the processed data in a specified storage medium.

[0063] Step 114: The kernel-mode driver determines the next hardware unit of the hardware unit based on this working order, and sends the second task message to the kernel-mode driver corresponding to the next hardware unit.

[0064] As can be seen from the above technical solutions, in the embodiments of the present application, by deploying a kernel-mode scheduler and a kernel-mode driver in the kernel mode of the processor, during the task processing, the task processing is completed in the kernel mode without the need to repeatedly switch between the user mode and the kernel mode, thereby saving CPU overhead, saving resource overhead, and being able to save the switching overhead (such as time overhead and resource overhead) between the user mode and the kernel mode. By completing the task processing in the kernel mode, the requirements of high real-time tasks can be met, the needs of mutual calls between each hardware unit to cooperate with each other can be met, and real-time tasks can be efficiently responded to.

[0065] In the embodiments of the present application, a task processing method is proposed, which can be applied to an electronic device (i.e., a device for implementing task processing). The electronic device may include a processor and multiple hardware units. For each hardware unit, the hardware unit corresponds to a kernel-mode driver in the kernel mode of the processor, and the hardware unit corresponds to a user-mode API (Application Programming Interface) in the user mode of the processor. Refer to Figure 1C As shown, for the flowchart of the task processing method, the method may include:

[0066] Step 121: For each kernel-mode driver, when the kernel-mode driver receives a first task message, it sends the first task message to the hardware unit corresponding to the kernel-mode driver, and receives a second task message returned by the hardware unit. The second task message may include processed data; wherein, the processed data is obtained after the hardware unit processes the to-be-processed data carried by the first task message.

[0067] Step 122: The kernel-mode driver obtains a first identity identifier corresponding to the requested hardware unit from the second task message; wherein, when a hardware unit expects another hardware unit to jointly complete the same task, the one hardware unit is the requested hardware unit, and the other hardware unit is the auxiliary hardware unit.

[0068] Step 123: Determine whether the second identity identifier corresponding to the kernel-mode driver is the same as the first identity identifier.

[0069] If not, that is, the second identity identifier is different from the first identity identifier, step 124 may be executed.

[0070] If so, that is, the second identity identifier is the same as the first identity identifier, step 125 may be executed.

[0071] Step 124: The kernel-mode driver sends the second task message to the kernel-mode driver corresponding to the requested hardware unit.

[0072] Step 125: The kernel-mode driver determines whether the auxiliary hardware unit has completed task processing. If so, that is, if the task processing has been completed, the second task message is sent to the user-mode API. If not, that is, if the task processing has not been completed, the second task message is sent to the kernel-mode driver corresponding to the auxiliary hardware unit.

[0073] As can be seen from the above technical solutions, in the embodiments of the present application, by deploying a kernel-mode scheduler and a kernel-mode driver in the kernel mode of the processor, during task processing, the task processing is completed in the kernel mode, without the need to repeatedly switch between the user mode and the kernel mode, saving CPU overhead, saving resource overhead, and saving the switching overhead between the user mode and the kernel mode (such as time overhead and resource overhead). By completing task processing in the kernel mode, the requirements of high real-time tasks can be met, the need for mutual calls between hardware units to work collaboratively can be satisfied, making the hardware units flexible and efficient, and flexibly responding to occasional tasks while taking into account flexibility.

[0074] The above technical solutions of the embodiments of the present application will be described below in combination with specific application scenarios.

[0075] The task processing device in this embodiment may be an electronic device, such as an edge device. An edge device is various computing devices deployed on edge nodes, such as smartphones, tablets, smart watches, smart speakers, smart homes, routers, cameras, etc. The type of this electronic device is not limited.

[0076] Exemplarily, the electronic device may include a processor (such as a CPU (Central Processing Unit, central processor), etc.) and multiple hardware units. The multiple hardware units may be homogeneous hardware units, that is, all hardware units are of the same type, such as all hardware units are NPU units, or all hardware units are GPU units, or all hardware units are ASIC units, or all hardware units are FPGA units, or all hardware units are CPLD units. Or, the multiple hardware units may also be heterogeneous hardware units, that is, all hardware units may include at least two types among NPU units, GPU units, ASIC units, FPGA units, and CPLD units. For example, the multiple hardware units include at least one NPU unit and at least one ASIC unit, or, the multiple hardware units include at least one NPU unit and at least one FPGA unit, or, the multiple hardware units include at least one NPU unit, at least one ASIC unit, and at least one FPGA unit. The type of the multiple hardware units is not limited in this embodiment.

[0077] For example, heterogeneous hardware units refer to the fact that in an electronic device, in addition to the CPU, there are also heterogeneous units such as NPU units, ASIC units, and FPGA units. These heterogeneous units are controlled by the CPU, and the heterogeneous units can cooperate around the CPU. For example, an ASIC unit, an NPU unit, and a CPU form a heterogeneous system. For the NPU unit in the heterogeneous hardware units, the NPU unit is a hardware module specifically used to accelerate neural network computing, aiming to improve the processing efficiency and performance of machine learning and deep learning tasks.

[0078] In the scenario of a CPU (i.e., a processor) and multiple hardware units, the multiple hardware units are controlled by the CPU. The CPU and the operating system (such as the Linux operating system) distinguish between the kernel mode and the user mode. For each hardware unit, the hardware unit corresponds to a driver in the kernel mode of the CPU. For the convenience of distinction, it can be denoted as the kernel-mode driver hereinafter. The hardware unit corresponds to an API in the user mode of the CPU. For the convenience of distinction, it can be denoted as the user-mode API hereinafter. Refer to Figure 2 As shown, it is a schematic diagram of the application scenario of a CPU and multiple hardware units. Taking hardware unit 1 and hardware unit 2 as examples, hardware unit 1 corresponds to kernel-mode driver 1 in the kernel mode of the CPU, and hardware unit 1 corresponds to user-mode API 1 in the user mode of the CPU. Hardware unit 2 corresponds to kernel-mode driver 2 in the kernel mode of the CPU, and hardware unit 2 corresponds to user-mode API 2 in the user mode of the CPU.

[0079] For example, the kernel-mode driver is a program running in the kernel mode and is used to control the operation of the hardware unit. It can include configuring registers, signals for commanding the hardware unit to start, and receiving interrupt signals from the hardware unit. The user-mode API is an interface provided to the user. The user-mode API internally calls the kernel-mode driver of the hardware unit, that is, the user controls the hardware unit by calling the user-mode API. For the convenience of understanding, in this embodiment, the user-mode API is used to represent the user-mode program and the user-mode program interface of the hardware unit as a whole.

[0080] When the application needs to call the hardware unit 1 to process data, it can call the user-mode API1 to send a task message 1 to the kernel-mode driver 1 (the kernel-mode driver 1 is a driver program). The user-mode API1 is used to convert the task message 1 into a message format that the kernel-mode driver 1 can understand. The kernel-mode driver 1 sends the task message 1 to the hardware unit 1. The kernel-mode driver 1 is used to convert the task message 1 into a message format that the hardware unit 1 can understand. After receiving the task message 1, the hardware unit 1 can process the data to be processed in the task message 1 and return a task message 2 to the kernel-mode driver 1. The task message 2 includes the processed data (i.e., the processing result of the data to be processed). The kernel-mode driver 1 returns the task message 2 to the user-mode API1. The kernel-mode driver 1 is used to convert the task message 2 into a message format that the application can understand. The application can obtain the task message 2 from the user-mode API1 and then get the processed data.

[0081] When the application needs to call the hardware unit 2 to process data (such as the processed data returned by the hardware unit 1), it can call the user-mode API2 to send a task message 3 to the kernel-mode driver 2. The kernel-mode driver 2 sends the task message 3 to the hardware unit 2. After receiving the task message 3, the hardware unit 2 can process the data to be processed in the task message 3 and return a task message 4 to the kernel-mode driver 2. The task message 4 includes the processed data. The kernel-mode driver 2 returns the task message 4 to the user-mode API2. The application can obtain the task message 4 from the user-mode API2 and then get the processed data.

[0082] In the above scenario, the kernel-mode driver 1 is used to start the hardware unit 1, send the task message of the user-mode API1 to the hardware unit 1, and send the task message of the hardware unit 1 to the user-mode API1. The user-mode API1 is an interface provided for the application to call. The user-mode API1 calls the kernel-mode driver 1 to control the hardware unit. The kernel-mode driver 2 is used to start the hardware unit 2, send the task message of the user-mode API2 to the hardware unit 2, and send the task message of the hardware unit 2 to the user-mode API2. The user-mode API2 is an interface provided for the application to call. The user-mode API2 calls the kernel-mode driver 2 to control the hardware unit.

[0083] In a possible implementation, the same task can be achieved through multiple hardware units, that is, multiple hardware units need to work together. For example, a task can be split into three subtasks. Subtask 1 is used for preprocessing the image (ISP processing), such as white balance processing, noise reduction, color correction processing, automatic exposure control, etc. Subtask 2 is used for artificial intelligence processing of the image, such as detecting the image based on a network model, classifying the image based on a network model, segmenting the image based on a network model, etc. Subtask 3 is used for fusion processing of the image, such as fusing the artificial intelligence processing result with the original image. Of course, the above is only an example of multiple hardware units achieving the same task.

[0084] On this basis, three hardware units can be used to process the three subtasks respectively. Hardware unit 1 is used to process Subtask 1, hardware unit 2 is used to process Subtask 2, and hardware unit 3 is used to process Subtask 3. On this basis, see Figure 3 As shown, it is a schematic diagram of the collaborative work between multiple hardware units. Taking hardware unit 1, hardware unit 2, and hardware unit 3 as examples, if hardware unit 1 is ASIC unit 1, hardware unit 2 is ASIC unit 2, and hardware unit 3 is NPU unit.

[0085] See Figure 3 As shown, hardware unit 1 corresponds to kernel driver 1 in the kernel mode of the CPU, and hardware unit 1 corresponds to user state API1 in the user mode of the CPU. Hardware unit 2 corresponds to kernel driver 2 in the kernel mode of the CPU, and hardware unit 2 corresponds to user state API2 in the user mode of the CPU. Hardware unit 3 corresponds to kernel driver 3 in the kernel mode of the CPU, and hardware unit 3 corresponds to user state API3 in the user mode of the CPU.

[0086] The application program calls user state API1 to send task message 1 to kernel driver 1. Task message 1 includes the image to be processed a1 (i.e., the original image). Kernel driver 1 sends task message 1 to hardware unit 1. Hardware unit 1 preprocesses (ISP processes) the image to be processed a1 in task message 1 to obtain the processed image a2 (i.e., the image after preprocessing), and returns task message 2 to kernel driver 1. Task message 2 includes the processed image a2. Kernel driver 1 returns task message 2 to user state API1. The application program obtains task message 2 from user state API1 and then obtains the processed image a2.

[0087] The application calls the user-mode API2 to send a task message 3 to the kernel-mode driver 2. The task message 3 includes the processed image a2. The kernel-mode driver 2 sends the task message 3 to the hardware unit 2. The hardware unit 2 performs artificial intelligence processing on the processed image a2 in the task message 3, such as detecting the processed image a2 based on a network model to obtain an image detection result a3 (such as a rectangular box of a license plate area, etc.). The hardware unit 2 returns a task message 4 to the kernel-mode driver 2. The task message 4 includes the processed image a2 and the image detection result a3. The kernel-mode driver 2 returns the task message 4 to the user-mode API2. The application obtains the task message 4 from the user-mode API2 and then gets the processed image a2 and the image detection result a3.

[0088] The application calls the user-mode API3 to send a task message 5 to the kernel-mode driver 3. The task message 5 includes the processed image a2 and the image detection result a3. The kernel-mode driver 3 sends the task message 5 to the hardware unit 3. The hardware unit 3 performs fusion processing on the processed image a2 and the image detection result a3 in the task message 5 to obtain an image fusion result a4 (such as superimposing the image detection result a3 on the processed image a2). The hardware unit 3 returns a task message 6 to the kernel-mode driver 3. The task message 6 includes the image fusion result a4. The kernel-mode driver 3 returns the task message 6 to the user-mode API3. The application obtains the task message 6 from the user-mode API3 and then gets the image fusion result a4. Thus, the processing process of the same task is completed.

[0089] Obviously, in the above collaborative working mode, each time the application calls the user-mode API, a switch between the user mode and the kernel mode will occur, and this inter-state switch will generate a large amount of CPU overhead, thus consuming a large amount of resources. If the user-mode API is called frequently, the CPU overhead caused by the inter-state switch will increase significantly. For example, the inter-state switch refers to the state switch between the user mode and the kernel mode. When calling the hardware unit through the user-mode API, the process of calling the hardware unit needs to go through the kernel-mode driver.

[0090] In view of the above findings, in an embodiment of the present application, a task processing method is proposed. By creating a task of the kernel-mode binding call type, the kernel-mode binding call of multiple hardware units can be realized. The kernel-mode binding call means that the kernel-mode drivers of each hardware unit are called in sequence based on the working order to drive the hardware unit to work, and the working order is determined by the application program according to actual requirements. For the kernel-mode binding call, the working order between each hardware unit has been created in the initialization stage, and then the kernel-mode driver controls each hardware unit to work according to this working order, without entering the user mode during the work. By realizing the kernel-mode binding call of multiple hardware units, the user mode is not entered during task processing, which can avoid the repeated switching between the user mode and the kernel mode, save the switching overhead between modes, thereby reducing the CPU overhead and saving resources.

[0091] For example, a task can be split into three subtasks. Subtask 1 is used for pre-processing the image, subtask 2 is used for artificial intelligence processing of the image, and subtask 3 is used for fusion processing of the image. Three hardware units are used to process the three subtasks respectively. Hardware unit 1 is used to process subtask 1, hardware unit 2 is used to process subtask 2, and hardware unit 3 is used to process subtask 3. On this basis, see Figure 4 As shown, it is a schematic diagram of the collaborative work between multiple hardware units. Taking hardware unit 1, hardware unit 2, and hardware unit 3 as examples, for example, hardware unit 1 is ASIC unit 1, hardware unit 2 is ASIC unit 2, and hardware unit 3 is NPU unit.

[0092] See Figure 4 As shown, hardware unit 1 corresponds to kernel-mode driver 1 in the kernel mode of the CPU, and hardware unit 1 corresponds to user-mode API1 in the user mode of the CPU. Hardware unit 2 corresponds to kernel-mode driver 2 in the kernel mode of the CPU, and hardware unit 2 corresponds to user-mode API2 in the user mode of the CPU. Hardware unit 3 corresponds to kernel-mode driver 3 in the kernel mode of the CPU, and hardware unit 3 corresponds to user-mode API3 in the user mode of the CPU.

[0093] To implement the kernel-mode binding call, the application program needs to determine the working order of multiple hardware units. For example, the working order can be expressed as hardware unit 1 - hardware unit 2 - hardware unit 3. In this way, the kernel-mode binding call means that based on the working order, first, kernel-mode driver 1 of hardware unit 1 is used to call hardware unit 1 to work, then, kernel-mode driver 2 of hardware unit 2 is used to call hardware unit 2 to work, and then, kernel-mode driver 3 of hardware unit 3 is used to call hardware unit 3 to work.

[0094] In the above application scenario, for this task processing method, the following steps may be included:

[0095] Step S11: The application starts a task of the kernel-mode binding call type and sends task message 1 to kernel-mode driver 1. For example, the application can call user-mode API1 to send task message 1 to kernel-mode driver 1, or directly send task message 1 to kernel-mode driver 1, and there is no restriction on this.

[0096] Task message 1 can include data to be processed and the working order. The data to be processed can be the image a1 to be processed (i.e., the original image), and the working order can represent Hardware Unit 1 - Hardware Unit 2 - Hardware Unit 3.

[0097] Step S12: After kernel-mode driver 1 receives task message 1 (i.e., the first task message), kernel-mode driver 1 sends task message 1 to Hardware Unit 1 corresponding to this kernel-mode driver 1.

[0098] Step S13: Hardware Unit 1 pre-processes the image a1 to be processed in task message 1 to obtain the processed image a2 (i.e., the processed data), and returns task message 2 to kernel-mode driver 1. Task message 2 can include the processed image a2 and the working order (obtained from task message 1).

[0099] Step S14: Kernel-mode driver 1 receives task message 2 (i.e., the second task message) returned by Hardware Unit 1, obtains the working order of multiple hardware units from task message 2, and determines whether Hardware Unit 1 is the last hardware unit based on this working order. Since the working order represents Hardware Unit 1 - Hardware Unit 2 - Hardware Unit 3 and Hardware Unit 1 is not the last hardware unit, step S15 is executed.

[0100] Step S15: Kernel-mode driver 1 determines the next hardware unit of Hardware Unit 1 (i.e., Hardware Unit 2) based on this working order, and sends task message 2 to kernel-mode driver 2 corresponding to Hardware Unit 2.

[0101] Obviously, kernel-mode driver 1 directly sends task message 2 to kernel-mode driver 2, rather than first sending task message 2 to user-mode API1, and then the application obtains task message 2 from user-mode API1 and then sends task message 2 to kernel-mode driver 2. In this way, it can avoid sending task message 2 to the user mode, avoid the repeated switching between the user mode and the kernel mode, and save the overhead of inter-state switching.

[0102] Step S16: After kernel-mode driver 2 receives task message 2 (i.e., the first task message), kernel-mode driver 2 sends task message 2 to Hardware Unit 2 corresponding to this kernel-mode driver 2.

[0103] Step S17: The hardware unit 2 performs artificial intelligence processing on the processed image a2 in the task message 2 (the processed image a2 is data to be processed for the hardware unit 2), obtains an image detection result a3 (such as a rectangular frame of a license plate area, etc.), and returns a task message 3 to the kernel-mode driver 2. The task message 3 may include the processed image a2, the image detection result a3, and the working order. For example, the processed image a2 and the image detection result a3 together are used as processed data, and the image detection result a3 can also be used as processed data alone.

[0104] Step S18: The kernel-mode driver 2 receives the task message 3 (i.e., the second task message) returned by the hardware unit 2, obtains the working orders of multiple hardware units from the task message 3, and determines whether the hardware unit 2 is the last hardware unit based on this working order. Since the working order represents Hardware Unit 1 - Hardware Unit 2 - Hardware Unit 3, and the hardware unit 2 is not the last hardware unit, step S19 is executed.

[0105] Step S19: The kernel-mode driver 2 determines the next hardware unit of the hardware unit 2 (i.e., the hardware unit 3) based on this working order, and sends the task message 3 to the kernel-mode driver 3 corresponding to the hardware unit 3.

[0106] Step S20: After the kernel-mode driver 3 receives the task message 3 (i.e., the first task message), the kernel-mode driver 3 sends the task message 3 to the hardware unit 3 corresponding to this kernel-mode driver 3.

[0107] Step S21: The hardware unit 3 performs fusion processing on the processed image a2 and the image detection result a3 in the task message 3, obtains an image fusion result a4. The processed image a2 and the image detection result a3 are data to be processed, and the image fusion result a4 is processed data. The hardware unit 3 returns a task message 4 to the kernel-mode driver 3. The task message 4 may include the image fusion result a4 and the working order.

[0108] Step S22: The kernel-mode driver 3 receives the task message 4 (i.e., the second task message) returned by the hardware unit 3, obtains the working orders of multiple hardware units from the task message 4, and determines whether the hardware unit 3 is the last hardware unit based on this working order. Since the working order represents Hardware Unit 1 - Hardware Unit 2 - Hardware Unit 3, and the hardware unit 3 is the last hardware unit, step S23 is executed.

[0109] Step S23: The kernel-mode driver 3 stores the image fusion result a4 (i.e., the processed data) in the task message 4 into a specified storage medium (such as storage media like memory, hard disk, etc.).

[0110] Obviously, when the hardware unit 3 is the last hardware unit, the kernel-mode driver 3 ends the processing of the task message, and can store the image fusion result a4 in the specified storage medium, instead of returning the task message to the user-mode API. In addition, the kernel-mode driver 3 can send a task end message to the application, indicating that the task has been processed. When the application needs to obtain the image fusion result a4, the application can directly read the image fusion result a4 from the specified storage medium without going through the user-mode API.

[0111] Obviously, in the above collaborative working mode, the application is only responsible for starting the task. Then, the kernel-mode drivers 1, 2, and 3 will sequentially complete the task in the kernel mode, without returning to the user-mode API during the task processing, and only need to store the image fusion result a4 in the specified storage medium at the end of the task. In this way, there is no need to switch repeatedly between the user mode and the kernel mode, which can save CPU overhead. Moreover, when sequentially completing the task in the kernel mode, it can also meet the high real-time requirements of the task.

[0112] In a possible implementation manner, for tasks of the kernel-mode bound call type, multiple hardware units process the tasks sequentially, so as to flexibly call the multiple hardware units sequentially, and the working order is specified by the application, that is, the user perceives the working order of the multiple hardware units. The kernel-mode bound call is applicable to tasks with a relatively fixed working order, many hardware units, frequent hardware unit calls, and high real-time requirements.

[0113] For example, when a certain task is used to encode the images collected by a camera, since it involves the image acquisition process, the image pre-processing process, the image artificial intelligence processing process, and the image encoding process, multiple hardware units can be coordinated to complete this task. Since the camera collects images in real time, that is, a large number of images need to be encoded, and the working order of these images is the same, the hardware units need to be called frequently. In summary, it can be seen that the working order of such tasks is fixed, there are many hardware units, the hardware units are called frequently, and the real-time requirements are high. Therefore, such tasks can be implemented through the kernel-mode bound call.

[0114] In addition to tasks of the kernel-mode bound call type, the embodiments of the present application propose a type of task of kernel-mode sequential call, which can implement the kernel-mode sequential call of multiple hardware units. The kernel-mode sequential call means that the hardware units are sequentially called to work by calling the kernel-mode drivers of the hardware units. For the kernel-mode sequential call, it is necessary to meet the requirement that multiple hardware units call each other to work together.

[0115] Different from the bound call in kernel mode, multiple hardware units include a requesting hardware unit and at least one assisting hardware unit. For example, when Hardware Unit 1 expects Hardware Unit 2 to jointly complete the same task, then Hardware Unit 1 is the requesting hardware unit and Hardware Unit 2 is the assisting hardware unit. Another example is that when Hardware Unit 1 expects Hardware Unit 2 and Hardware Unit 3 to jointly complete the same task, then Hardware Unit 1 is the requesting hardware unit, and Hardware Unit 2 and Hardware Unit 3 are the assisting hardware units.

[0116] For the sequential call in kernel mode, there is no need for the application program to specify the working order. Instead, the working order of the assisting hardware unit is pre-configured in the kernel-mode driver corresponding to the requesting hardware unit. In this way, the application program / user is not aware of the working order of the assisting hardware unit. In addition, for tasks of the sequential call type in kernel mode, multiple hardware units process the task sequentially. However, the task message of the assisting hardware unit can only be returned to the requesting hardware unit, and cannot be sent to another assisting hardware unit.

[0117] The sequential call in kernel mode is applicable to tasks with occasional hardware unit calls and high real-time requirements. For example, for occasional human detection, vehicle detection, etc. tasks, that is, only perform human detection and vehicle detection on a single frame or several frames of images, such tasks can be implemented through the sequential call in kernel mode. For example, during the JEPG encoding process, the JEPG encoding hardware unit expects to call the NPU hardware unit to assist in the encoding work and expects to hide this intermediate call process from the user. Then this type of task is implemented through the sequential call in kernel mode, that is, the JEPG encoding hardware unit initiates a sequential call to call the NPU hardware unit through the NPU kernel-mode driver.

[0118] For the sequential call in kernel mode, the working order of the assisting hardware unit is pre-configured in the kernel-mode driver corresponding to the requesting hardware unit, and then the kernel-mode driver controls the work of each hardware unit according to this working order, without entering the user mode during the work. By implementing the sequential call in kernel mode of multiple hardware units, it is possible to avoid entering the user mode during task processing, avoid the repeated switching between the user mode and the kernel mode, and can automatically complete the mutual call between each hardware unit, avoiding the user (application program) from being aware, which is convenient for the user to call. It can make the interface compatible with the user when the function is upgraded. For example, the kernel-mode implementation method of the first version of JPEG encoding only calls the JPEG hardware encoding unit, and the kernel-mode implementation method of the second version of JPEG encoding calls the JPEG hardware encoding unit and the NPU hardware encoding unit, and the user-mode APIs of these two versions are exactly the same. That is, when the JPEG encoding function is upgraded, it will not increase the workload of modifying the code of the application program product, and only the kernel-mode driver needs to support the sequential call in kernel mode.

[0119] For example, hardware unit 1 is a requesting hardware unit, and hardware unit 1 expects hardware unit 2 and hardware unit 3 to complete the same task together, that is, hardware unit 2 and hardware unit 3 are auxiliary hardware units. Figure 5 As shown, it is a schematic diagram of the collaborative work between multiple hardware units, taking hardware unit 1, hardware unit 2 and hardware unit 3 as examples. Hardware unit 1 corresponds to kernel state driver 1 in the kernel state of the CPU, and hardware unit 1 corresponds to user state API 1 in the user state of the CPU. Hardware unit 2 corresponds to kernel state driver 2 in the kernel state of the CPU, and hardware unit 2 corresponds to user state API 2 in the user state of the CPU. Hardware unit 3 corresponds to kernel state driver 3 in the kernel state of the CPU, and hardware unit 3 corresponds to user state API 3 in the user state of the CPU.

[0120] In order to realize kernel-mode sequential calls, the working order of multiple auxiliary hardware units (such as hardware unit 2 and hardware unit 3) can be pre-configured in the kernel-mode driver 1 of the requesting hardware unit (hardware unit 1). There is no restriction on this configuration process. For example, the working order can represent hardware unit 2-hardware unit 3.

[0121] In the above application scenario, with respect to the task processing method, the method may include the following steps:

[0122] Step S31 : The application program calls the user state API 1 to send a task message 1 to the kernel state driver 1 .

[0123] Step S32 : After receiving the task message 1 (ie, the first task message), the kernel state driver 1 sends the task message 1 to the hardware unit 1 corresponding to the kernel state driver 1 .

[0124] Step S33: Hardware unit 1 processes the data to be processed in task message 1 to obtain processed data, and returns task message 2 to kernel state driver 1, where task message 2 includes the processed data.

[0125] Step S34 , the kernel-mode driver 1 receives the task message 2 (ie, the second task message) returned by the hardware unit 1 , and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 2 .

[0126] For example, when kernel-mode driver 1 receives task message 1 from user-mode API 1, since task message 1 originates from user-mode API 1, kernel-mode driver 1 determines that hardware unit 1 corresponding to kernel-mode driver 1 is the requesting hardware unit, and adds the first identity corresponding to the requesting hardware unit in task message 1. The first identity may be a unique identifier of kernel-mode driver 1 (such as a driver identifier). Hardware unit 1 may obtain the first identity from task message 1 and add the first identity to task message 2. Based on this, kernel-mode driver 1 may obtain the first identity from task message 2.

[0127] Step S35: The kernel-mode driver 1 determines whether the second identity identifier corresponding to this kernel-mode driver 1 (such as the unique identifier of the kernel-mode driver 1) is the same as the first identity identifier. If so, the kernel-mode driver 1 determines whether the auxiliary hardware unit has completed task processing. If not, the kernel-mode driver 1 sends the task message 2 to the kernel-mode driver corresponding to the auxiliary hardware unit. Among them, if the auxiliary hardware unit has returned the task message, the task processing has been completed; if the auxiliary hardware unit has not returned the task message, the task processing has not been completed.

[0128] For example, the kernel-mode driver 1 pre-configures the working order of multiple auxiliary hardware units. For example, the working order can represent Hardware Unit 2 - Hardware Unit 3. Since neither Hardware Unit 2 nor Hardware Unit 3 has returned the task message, neither Hardware Unit 2 nor Hardware Unit 3 has completed task processing. The kernel-mode driver 1 sends the task message 2 to the kernel-mode driver 2 corresponding to Hardware Unit 2 (i.e., the auxiliary hardware unit).

[0129] In a possible implementation manner, when the kernel-mode driver 1 pre-configures the working order of multiple auxiliary hardware units, it can configure the corresponding relationship between the task identifier and the working order. For example, the task identifier of the task message 1 can be obtained in advance, and then the corresponding relationship between the task identifier and the working order can be configured. When the application sends the task message 1 to the kernel-mode driver 1, the task message 1 includes the task identifier, and the task message 2 also includes this task identifier. In this way, the kernel-mode driver 1 can parse the task identifier from the task message 2, and then query the working order corresponding to the task identifier. For example, the working order represents Hardware Unit 2 - Hardware Unit 3.

[0130] Step S36: After the kernel-mode driver 2 receives the task message 2 (i.e., the first task message), the kernel-mode driver 2 sends the task message 2 to the Hardware Unit 2 corresponding to this kernel-mode driver 2.

[0131] Step S37: The Hardware Unit 2 processes the data to be processed in the task message 2 to obtain the processed data, and returns the task message 3 to the kernel-mode driver 2. The task message 3 includes the processed data.

[0132] Step S38: The kernel-mode driver 2 receives the task message 3 (i.e., the second task message) returned by the Hardware Unit 2, and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 3.

[0133] For example, the Hardware Unit 2 can obtain the first identity identifier from the task message 2, add the first identity identifier to the task message 3, and the kernel-mode driver 2 can obtain the first identity identifier from the task message 3.

[0134] Step S39: The kernel-mode driver 2 determines whether the second identity identifier corresponding to this kernel-mode driver 2 (such as the unique identifier of the kernel-mode driver 2) is the same as the first identity identifier. If not, the kernel-mode driver 2 sends the task message 3 to the kernel-mode driver 1 corresponding to the requesting hardware unit (i.e., the hardware unit 1).

[0135] For example, if the kernel-mode driver 2 determines that the hardware unit 1 corresponding to the second identity identifier is the requesting hardware unit, the kernel-mode driver 2 sends the task message 3 to the kernel-mode driver 1 corresponding to the second identity identifier.

[0136] Step S40: The kernel-mode driver 1 receives the task message 3 (i.e., the second task message) returned by the kernel-mode driver 2, and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 3.

[0137] Step S41: The kernel-mode driver 1 determines whether the second identity identifier corresponding to this kernel-mode driver 1 is the same as the first identity identifier. If so, the kernel-mode driver 1 determines whether the auxiliary hardware unit has completed the task processing. If the auxiliary hardware unit has not completed the task processing, the kernel-mode driver 1 sends the task message 3 to the kernel-mode driver corresponding to the auxiliary hardware unit. For example, since the hardware unit 2 has returned the task message while the hardware unit 3 has not, the hardware unit 3 has not completed the task processing, and the kernel-mode driver 1 can send the task message 3 to the kernel-mode driver 3 corresponding to the hardware unit 3.

[0138] Step S42: After receiving the task message 3 (i.e., the first task message), the kernel-mode driver 3 sends the task message 3 to the hardware unit 3 corresponding to this kernel-mode driver 3.

[0139] Step S43: The hardware unit 3 processes the data to be processed in the task message 3 to obtain the processed data, and returns the task message 4 to the kernel-mode driver 3. The task message 4 includes the processed data.

[0140] Step S44: The kernel-mode driver 3 receives the task message 4 (i.e., the second task message) returned by the hardware unit 3, and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 4.

[0141] Step S45: The kernel-mode driver 3 determines whether the second identity identifier corresponding to this kernel-mode driver 3 (such as the unique identifier of the kernel-mode driver 3) is the same as the first identity identifier. If not, the kernel-mode driver 3 sends the task message 4 to the kernel-mode driver 1 corresponding to the requesting hardware unit (i.e., the hardware unit 1).

[0142] Step S46: The kernel-mode driver 1 receives the task message 4 (i.e., the second task message) returned by the kernel-mode driver 3, and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 4.

[0143] Step S47, the kernel-mode driver 1 determines whether the second identity identifier corresponding to this kernel-mode driver 1 is the same as the first identity identifier. If so, the kernel-mode driver 1 determines whether the auxiliary hardware unit has completed task processing. If so, the kernel-mode driver 1 sends task message 4 to the user-mode API1. For example, hardware unit 2 and hardware unit 3 have returned task messages, and all auxiliary hardware units have completed task processing.

[0144] After sending task message 4 to the user-mode API1, the application can obtain task message 4 from the user-mode API1, and then obtain the processed data. Thus, the processing process of the same task is completed.

[0145] In a possible implementation, as shown in Figure 6 the figure, it is a schematic diagram of successive calls for the kernel mode. The application can call the ASIC hardware unit through the ASIC user-mode API of the ASIC hardware unit to complete a job. In order to improve the working effect, the ASIC hardware unit hopes to use the NPU hardware unit to assist in the work, and does not need to explicitly expose the process of the ASIC hardware unit calling the NPU hardware unit to the user of the ASIC user-mode API. In this way, successive calls in the kernel mode can be adopted.

[0146] In the above process, tasks of the kernel-mode bound call type (corresponding to Figure 4 ), tasks of the user-mode successive call type (corresponding to Figure 2 ), and tasks of the kernel-mode successive call type (corresponding to Figure 5 ) are involved. In order to support at least two of these three types of tasks simultaneously, this embodiment proposes a task processing method. The hardware unit supports kernel-mode bound calls, user-mode successive calls, and kernel-mode successive calls at the same time, forming a heterogeneous system with flexibility, real-time performance, and high efficiency. Kernel-mode bound call means calling the hardware unit to work by successively calling the kernel-mode drivers of each hardware unit based on the working order. User-mode successive call means calling the hardware unit to work successively by calling the user-mode API of the hardware unit. For example, when the user calls the user-mode API of the NPU once, a neural network inference can be completed. Kernel-mode successive call means calling the hardware unit to work by calling the kernel-mode driver of the hardware unit.

[0147] Exemplarily, multiple hardware units are divided into a first type of hardware units and a second type of hardware units. The number of the second type of hardware units may be 0 or at least one. The first type of hardware units correspond to a kernel scheduler and kernel drivers in the kernel mode of the processor, and the first type of hardware units correspond to user-state APIs in the user mode of the processor. The second type of hardware units correspond to kernel drivers in the kernel mode of the processor, and the second type of hardware units correspond to user-state APIs in the user mode of the processor. Obviously, compared with the second type of hardware units, the first type of hardware units need to correspond to a kernel scheduler in the kernel mode, that is, deploy an additional kernel scheduler, and perform task scheduling with the kernel scheduler as the core. In summary, based on the kernel drivers and user-state APIs, a kernel scheduler can be developed, and the kernel scheduler undertakes tasks of the kernel-mode bound call type, user-mode sequential call type, and kernel-mode sequential call type at the same time.

[0148] Exemplarily, the first type of hardware units may include, but are not limited to, NPU units and / or GPU units, and the second type of hardware units may include at least one of, but are not limited to, ASIC units, FPGA units, and CPLD units. For example, if there is an NPU unit, the NPU unit can be used as the first type of hardware units, and if there is a GPU unit, the GPU unit can be used as the first type of hardware units. If there is an ASIC unit, the ASIC unit can be used as the second type of units, if there is an FPGA unit, the FPGA unit can be used as the second type of units, and if there is a CPLD unit, the CPLD unit can be used as the second type of units.

[0149] See Figure 7 As shown, it is a schematic diagram of the collaborative work among multiple hardware units. Taking hardware unit 1, hardware unit 2, hardware unit 3, and hardware unit 4 as examples, if hardware unit 1 is ASIC unit 1, hardware unit 2 is ASIC unit 2, hardware unit 3 is an NPU unit, and hardware unit 4 is ASIC unit 3.

[0150] In Figure 7 taking hardware unit 3 as the first type of hardware units as an example, hardware unit 1, hardware unit 2, and hardware unit 4 are the second type of hardware units. Of course, Figure 7 this is just an example. For example, hardware unit 1 and hardware unit 3 are the first type of hardware units, or hardware unit 1 and hardware unit 2 are the first type of hardware units, or all hardware units are the first type of hardware units, and there is no restriction on this.

[0151] In Figure 7Among them, the hardware unit 1 corresponds to the kernel-mode driver 1 in the kernel mode of the CPU and the user-mode API1 in the user mode of the CPU. The hardware unit 2 corresponds to the kernel-mode driver 2 in the kernel mode of the CPU and the user-mode API2 in the user mode of the CPU. The hardware unit 3 corresponds to the kernel-mode driver 3 and the kernel-mode scheduler 3 in the kernel mode of the CPU, and the hardware unit 3 corresponds to the user-mode API3 in the user mode of the CPU. The hardware unit 4 corresponds to the kernel-mode driver 4 in the kernel mode of the CPU and the user-mode API4 in the user mode of the CPU.

[0152] In Figure 7 Among them, the tasks of the kernel-mode binding call type can be completed by the hardware unit 1, the hardware unit 2, and the hardware unit 3. The application program needs to determine the working order of multiple hardware units. For example, the working order can be expressed as hardware unit 1 - hardware unit 2 - hardware unit 3. In addition, the tasks of the user-mode sequential call type can be completed by the hardware unit 3. In addition, the tasks of the kernel-mode sequential call type can be completed by the hardware unit 3 and the hardware unit 4. The hardware unit 3 is used as the requesting hardware unit, and the hardware unit 4 is used as the auxiliary hardware unit. Or, the hardware unit 4 is used as the requesting hardware unit, and the hardware unit 3 is used as the auxiliary hardware unit. In the subsequent process, take the hardware unit 4 as the requesting hardware unit and the hardware unit 3 as the auxiliary hardware unit as an example.

[0153] Case 1. For the task processing method of the kernel-mode binding call type, it can include the following steps:

[0154] Step S51: The application program starts the task of the kernel-mode binding call type and sends task message 1 to the kernel-mode driver 1. After receiving task message 1, the kernel-mode driver 1 sends task message 1 to the hardware unit 1 corresponding to this kernel-mode driver 1. The hardware unit 1 processes the data to be processed in task message 1 to obtain the processed data, and returns task message 2 to the kernel-mode driver 1. Task message 2 can include the processed data. In task message 1 and task message 2, the working order can also be included.

[0155] Step S52: The kernel-mode driver 1 receives task message 2 returned by the hardware unit 1, obtains the working order of multiple hardware units from task message 2, and determines whether the hardware unit 1 is the last hardware unit based on this working order. Since the hardware unit 1 is not the last hardware unit, the kernel-mode driver 1 determines the next hardware unit of the hardware unit 1 (i.e., the hardware unit 2) based on this working order, and the kernel-mode driver 1 sends task message 2 to the kernel-mode driver 2 corresponding to the hardware unit 2.

[0156] Step S53: The kernel-mode driver 2 sends task message 2 to the corresponding hardware unit 2 of this kernel-mode driver 2. The hardware unit 2 processes the data to be processed in the task message 2 to obtain the processed data, and returns task message 3 to the kernel-mode driver 2. The task message 3 may include the processed data and the working order.

[0157] Step S54: The kernel-mode driver 2 obtains the working orders of multiple hardware units from the task message 3, and determines whether the hardware unit 2 is the last hardware unit based on the working order. Since the hardware unit 2 is not the last hardware unit, the kernel-mode driver 2 determines the next hardware unit (hardware unit 3) of the hardware unit 2 based on the working order, and sends the task message 3 to the kernel-mode scheduler 3 corresponding to the hardware unit 3.

[0158] Exemplarily, since the hardware unit 3 corresponds to the kernel-mode scheduler 3 in the kernel mode, the kernel-mode driver 2 sends the task message 3 to the kernel-mode scheduler 3 instead of the kernel-mode driver 3.

[0159] Step S55: After receiving the task message 3 (i.e., the first task message), the kernel-mode scheduler 3 sends the task message 3 to the kernel-mode driver 3, and the kernel-mode driver 3 sends the task message 3 to the corresponding hardware unit 3 of this kernel-mode driver 3. The hardware unit 3 processes the data to be processed in the task message 3 to obtain the processed data, returns task message 4 to the kernel-mode driver 3, and the kernel-mode driver 3 returns task message 4 to the kernel-mode scheduler 3. The task message 4 may include the processed data and the working order.

[0160] Step S56: The kernel-mode scheduler 3 receives the task message 4 (the second task message) returned by the kernel-mode driver 3. In addition to the processed data and the working order, the task message 4 may include the task type.

[0161] Exemplarily, when the application sends the task message 1 to the kernel-mode driver 1, the task message 1 may include the task type. In this way, the task messages 2, 3, and 4 all include the task type. For other entities (such as kernel-mode drivers) other than the kernel-mode scheduler 3, they do not pay attention to the task type, while the kernel-mode scheduler 3 pays attention to the task type. For example, after receiving the task message 4, the kernel-mode scheduler 3 can parse the task type from the task message 4. Since Case 1 is a task processing method for the kernel-mode bound call type, the task type is the kernel-mode bound call.

[0162] Step S57: If the task type is a kernel-mode bound call, the kernel-mode scheduler 3 obtains the working order of multiple hardware units from the task message 4, and determines whether the hardware unit 3 (the first type of hardware unit) is the last hardware unit based on this working order. If so, the kernel-mode scheduler 3 stores the processed data in a specified storage medium. If not, the kernel-mode scheduler 3 sends the task message 4 to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit. If the next hardware unit is the first type of hardware unit, the task message 4 is sent to the kernel-mode scheduler corresponding to the next hardware unit. If the next hardware unit is the second type of hardware unit, the task message 4 is sent to the kernel-mode driver corresponding to the next hardware unit.

[0163] In this example, since the working order represents Hardware Unit 1 - Hardware Unit 2 - Hardware Unit 3, that is, Hardware Unit 3 is the last hardware unit, the kernel-mode scheduler 3 can end the processing of the task message and store the processed data in the task message 4 in a specified storage medium. In addition, the kernel-mode scheduler 3 can send a task end message to the application program indicating that the task has been processed. When the application program needs to obtain the processed data, it can directly read the processed data from the specified storage medium.

[0164] Case 2: For the task processing method of the user-mode sequential call type, the following steps may be included:

[0165] Step S61: The application program calls the user-mode API1 to send the task message 1 to the kernel-mode scheduler 3.

[0166] Step S62: After receiving the task message 1 (i.e., the first task message), the kernel-mode scheduler 3 sends the task message 1 to the kernel-mode driver 3, and the kernel-mode driver 3 sends the task message 1 to the hardware unit 3 corresponding to this kernel-mode driver 3. The hardware unit 3 processes the data to be processed in the task message 1 to obtain the processed data, the hardware unit 3 returns the task message 2 to the kernel-mode driver 3, and the kernel-mode driver 3 returns the task message 2 to the kernel-mode scheduler 3. The task message 2 may include the processed data.

[0167] Step S63: The kernel-mode scheduler 3 receives the task message 2 (i.e., the second task message) returned by the kernel-mode driver 3. In addition to the processed data, the task message 2 may include the task type.

[0168] Exemplarily, when the application program sends the task message 1, the task message 1 may include the task type. In this way, the task message 2 includes the task type. After receiving the task message 2, the kernel-mode scheduler 3 can parse the task type from the task message 2, and the task type is a user-mode sequential call.

[0169] Step S64: If the task type is user-mode sequential call, the kernel-mode scheduler 3 sends task message 2 to the user-mode API 3, so that the application can obtain task message 2 from the user-mode API 3 and obtain the processed data from task message 2. Thus, the processing process of the user-mode sequential call task is completed.

[0170] Case 3. For the task processing method of the kernel-mode sequential call type, the following steps may be included:

[0171] Step S71: The application calls the user-mode API 4 to send task message 1 to the kernel-mode driver 4. After receiving task message 1, the kernel-mode driver 4 sends task message 1 to the corresponding hardware unit 4 of this kernel-mode driver 4. The hardware unit 4 processes the data to be processed in task message 1 to obtain the processed data, and returns task message 2 to the kernel-mode driver 4. Task message 2 includes the processed data.

[0172] Step S72: The kernel-mode driver 4 obtains the first identity identifier (such as the unique identifier of the kernel-mode driver 4) corresponding to the requested hardware unit (i.e., hardware unit 4) from task message 2. The kernel-mode driver 4 determines whether the second identity identifier corresponding to this kernel-mode driver 4 is the same as the first identity identifier. If so, the kernel-mode driver 4 determines whether the auxiliary hardware unit (i.e., hardware unit 3) has completed the task processing. If not, the kernel-mode driver 4 sends task message 2 to the kernel-mode scheduler 3 corresponding to the auxiliary hardware unit.

[0173] Step S73: After receiving task message 2 (i.e., the first task message), the kernel-mode scheduler 3 sends task message 2 to the kernel-mode driver 3, and the kernel-mode driver 3 sends task message 2 to the corresponding hardware unit 3 of this kernel-mode driver 3. The hardware unit 3 processes the data to be processed in task message 2 to obtain the processed data. The hardware unit 3 returns task message 3 to the kernel-mode driver 3, and the kernel-mode driver 3 returns task message 3 to the kernel-mode scheduler 3. Task message 3 may include the processed data.

[0174] Step S74: The kernel-mode scheduler 3 receives task message 3 (the second task message) returned by the kernel-mode driver 3. In addition to the processed data, this task message 3 may also include the task type.

[0175] Exemplarily, when the application sends task message 1, task message 1 may include the task type. In this way, both task message 2 and task message 3 include the task type. The kernel-mode scheduler 3 can parse the task type from task message 3, and this task type is the kernel-mode sequential call.

[0176] Step S75. If the task type is kernel-mode sequential call, the kernel-mode scheduler 3 obtains the first identity identifier corresponding to the requested hardware unit (hardware unit 4) from the task message 3, and determines whether the second identity identifier corresponding to the kernel-mode scheduler 3 (such as the unique identifier of the kernel-mode driver 3) is the same as the first identity identifier.

[0177] If not, the kernel-mode scheduler 3 sends the task message 3 to the kernel-mode scheduler or kernel-mode driver corresponding to the requested hardware unit. For example, if the requested hardware unit is a first type of hardware unit, the task message 3 is sent to the kernel-mode scheduler corresponding to the requested hardware unit; if the requested hardware unit is a second type of hardware unit, the task message 3 is sent to the kernel-mode driver corresponding to the requested hardware unit.

[0178] In this example, since the second identity identifier is different from the first identity identifier, the kernel-mode scheduler 3 can send the task message 3 to the kernel-mode driver 4 corresponding to the hardware unit 4.

[0179] If so, the kernel-mode scheduler 3 determines that the hardware unit 3 is the requested hardware unit, and can determine whether the auxiliary hardware unit has completed the task processing. If so, the kernel-mode scheduler 3 sends the task message 3 to the user-mode API 3 so that the application can obtain the task message 3 from the user-mode API 3; if not, the kernel-mode scheduler 3 sends the task message 3 to the kernel-mode scheduler or kernel-mode driver corresponding to the auxiliary hardware unit.

[0180] Step S76. The kernel-mode driver 4 receives the task message 3 returned by the kernel-mode scheduler 3, and obtains the first identity identifier corresponding to the requested hardware unit from the task message 3. The kernel-mode driver 4 determines whether the second identity identifier corresponding to this kernel-mode driver 4 is the same as the first identity identifier. If so, the kernel-mode driver 4 determines whether the auxiliary hardware unit has completed the task processing. If so, the kernel-mode driver 4 sends the task message 3 to the user-mode API 4, and the application can obtain the task message 3 from the user-mode API 4.

[0181] In a possible implementation manner, when the kernel-mode scheduler 3 simultaneously supports kernel-mode bound call, user-mode sequential call, and kernel-mode sequential call, the kernel-mode scheduler 3 can receive multiple first task messages. For example, a first task message for kernel-mode bound call, a first task message for user-mode sequential call, and a first task message for kernel-mode sequential call.

[0182] Based on this, when the kernel-mode scheduler 3 receives multiple first task messages, for each first task message, it can obtain the task priority of the first task message from the first task message. Based on the task priorities of each first task message, the kernel-mode scheduler 3 selects the first task message with the highest task priority and sends the first task message with the highest task priority to the kernel-mode driver 3. When receiving the second task message (i.e., the response of the first task message) returned by the kernel-mode driver 3, the kernel-mode scheduler 3 selects the first task message with the second highest task priority and sends the first task message with the second highest task priority to the kernel-mode driver 3, and so on, until all the first task messages are sent to the kernel-mode driver 3.

[0183] To implement the above process and make the kernel-mode scheduler 3 support task priorities, when sending the first task message for tasks in different directions, the task priority needs to be carried in the first task message. For example, when sending the first task message for a task of kernel-mode bound call, the first task message includes task priority 3 (indicating the highest priority). When sending the first task message for a task of kernel-mode sequential call, the first task message includes task priority 2 (indicating the intermediate priority). When sending the first task message for a task of user-mode sequential call, the first task message includes task priority 1 (indicating the lowest priority).

[0184] As can be seen from the above technical solutions, in the embodiments of the present application, by deploying the kernel-mode scheduler and the kernel-mode driver in the kernel mode, during the task processing, the task processing can be completed in the kernel mode without the need to repeatedly switch between the user mode and the kernel mode, saving resource overhead and switching overhead, and meeting the requirements of high real-time tasks. By deploying the kernel-mode scheduler, the hardware unit supports user-mode sequential calls, kernel-mode bound calls, and kernel-mode sequential calls at the same time, making the hardware unit balance flexibility and real-time performance, enabling the hardware unit to efficiently respond to real-time tasks and also being able to respond to occasional tasks with flexibility.

[0185] Based on the same application concept as the above method, in the embodiments of the present application, an electronic device is proposed. The electronic device includes a processor and multiple hardware units. For example, the multiple hardware units may include a first type of hardware unit, and the electronic device further includes a kernel-mode scheduler and a kernel-mode driver corresponding to the first type of hardware unit in the kernel mode of the processor; where:

[0186] The kernel-mode scheduler is configured to send the first task message to the kernel-mode driver when receiving the first task message, and the first task message includes data to be processed;

[0187] The kernel-mode driver is configured to send the first task message to the first type of hardware unit;

[0188] The first type of hardware unit is used to process the data to be processed to obtain processed data, and send a second task message to the kernel-mode driver, where the second task message includes the processed data and the task type;

[0189] The kernel-mode driver is further used to send the second task message to the kernel-mode scheduler;

[0190] The kernel-mode scheduler is further used to receive the second task message; if the task type is a kernel-mode bound call, obtain the working order of multiple hardware units from the second task message, and determine whether the first type of hardware unit is the last hardware unit based on the working order; if so, store the processed data in a specified storage medium; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first type of hardware unit;

[0191] Wherein, the kernel-mode bound call means calling the kernel-mode driver of each hardware unit in sequence based on the working order to call the hardware unit to work.

[0192] Exemplarily, the first type of hardware unit corresponds to a user-mode API in the user mode of the processor, and the kernel-mode scheduler is further used to, if the task type is a user-mode sequential call, send the second task message to the user-mode API, so that the application program obtains the second task message from the user-mode API and obtains the processed data from the second task message; wherein, the user-mode sequential call means calling the hardware unit to work by calling the user-mode API of the hardware unit.

[0193] Exemplarily, the kernel-mode scheduler is further used to, if the task type is a kernel-mode sequential call, obtain the first identity identifier corresponding to the requested hardware unit from the second task message; if the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the requested hardware unit; if the second identity identifier is the same as the first identity identifier, determine whether the auxiliary hardware unit has completed task processing; if so, send the second task message to the user-mode API; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the auxiliary hardware unit;

[0194] Wherein, the kernel-mode sequential call means calling the hardware unit to work by calling the kernel-mode driver of the hardware unit; wherein, when a hardware unit expects another hardware unit to jointly complete the same task, the one hardware unit is the requested hardware unit, and the other hardware unit is the auxiliary hardware unit.

[0195] Based on the same application concept as the above method, an electronic device is proposed in an embodiment of the present application. Refer to Figure 8 As shown, the electronic device includes: a processor 81 and a machine-readable storage medium 82, and the machine-readable storage medium 82 stores machine-executable instructions that can be executed by the processor 81; the processor 81 is configured to execute the machine-executable instructions to implement the task processing method disclosed in the above examples of the present application.

[0196] Based on the same application concept as the above method, an embodiment of the present application further provides a machine-readable storage medium, and a number of computer instructions are stored on the machine-readable storage medium. When the computer instructions are executed by a processor, the task processing method disclosed in the above examples of the present application can be implemented.

[0197] Among them, the above machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as a hard disk drive), solid-state drive, any type of storage disk (such as an optical disk, DVD, etc.), or a similar storage medium, or a combination thereof.

[0198] Based on the same application concept as the above method, an embodiment of the present application further provides a computer program product, and the computer program product may include a computer program. When the computer program is executed by a processor, the task processing method disclosed in the above examples of the present application can be implemented.

[0199] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0200] The above are only the embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A task processing method, characterized in that: Applied to an electronic device including a processor and a plurality of hardware units, the plurality of hardware units including a first type of hardware unit, the first type of hardware unit corresponding to a kernel state scheduler and a kernel state driver in a kernel state of the processor, the method comprising: When receiving the first task message, the kernel state scheduler sends the first task message to the kernel state driver, and the kernel state driver sends the first task message to the first type of hardware unit; The kernel-mode scheduler receives a second task message returned by the kernel-mode driver, wherein the second task message includes processed data and a task type; wherein the processed data is obtained after the first type of hardware unit processes the data to be processed carried by the first task message; If the task type is a kernel-mode binding call, the kernel-mode scheduler obtains the working order of multiple hardware units from the second task message, and determines whether the first-type hardware unit is the last hardware unit based on the working order; wherein the kernel-mode binding call means calling the kernel-mode driver of each hardware unit in turn based on the working order to call the hardware unit to work; If so, the kernel-mode scheduler stores the processed data to a designated storage medium; If not, the kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit.

2. The method according to claim 1, characterized in that The first type of hardware unit corresponds to the user state API in the user state of the processor, and after the kernel state scheduler receives the second task message returned by the kernel state driver, the method further includes: If the task type is user-mode successive calls, the kernel-mode scheduler sends the second task message to the user-mode API, so that the application obtains the second task message from the user-mode API and obtains processed data from the second task message; wherein, the user-mode successive calls represent calling the hardware unit to perform work by calling the user-mode API of the hardware unit.

3. The method according to claim 1, characterized in that: After the kernel-mode scheduler receives the second task message returned by the kernel-mode driver, the method further includes: If the task type is kernel-mode sequential calls, the kernel-mode scheduler obtains the first identity corresponding to the requesting hardware unit from the second task message; wherein the kernel-mode sequential calls represent calling the hardware unit to perform work by calling the kernel-mode driver of the hardware unit; If the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, the kernel state scheduler determines that the first type of hardware unit is an auxiliary hardware unit, and sends the second task message to the kernel state scheduler or kernel state driver corresponding to the requesting hardware unit; When one hardware unit expects another hardware unit to complete the same task together, the one hardware unit is a requesting hardware unit, and the other hardware unit is an assisting hardware unit.

4. The method according to claim 3, characterized in that After the kernel state scheduler obtains the first identity corresponding to the requesting hardware unit from the second task message, the method further includes: If the second identity identifier is the same as the first identity identifier, the kernel-mode scheduler determines that the first type of hardware unit is a requesting hardware unit, and determines whether the auxiliary hardware unit has completed task processing; wherein, if the auxiliary hardware unit has returned a task message, the auxiliary hardware unit has completed task processing; if the auxiliary hardware unit has not returned a task message, the auxiliary hardware unit has not completed task processing; If so, the kernel-state scheduler sends the second task message to the user-state API so that the application obtains the second task message from the user-state API; if not, the kernel-state scheduler sends the second task message to the kernel-state scheduler or kernel-state driver corresponding to the auxiliary hardware unit.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When receiving a plurality of first task messages, the kernel state scheduler acquires the task priority of each first task message from the first task message; Based on the task priority of each first task message, the kernel state scheduler selects the first task message with the highest task priority, and sends the first task message with the highest task priority to the kernel state driver.

6. The method according to any one of claims 1 to 4, characterized in that: The plurality of hardware units further include a second type of hardware unit, wherein the second type of hardware unit corresponds to a kernel state driver in the kernel state of the processor, and the second type of hardware unit corresponds to a user state API in the user state of the processor; Wherein, when sending the second task message to the kernel-state scheduler or kernel-state driver corresponding to the next hardware unit of the first-category hardware unit, if the next hardware unit is a first-category hardware unit, the kernel-state scheduler sends the second task message to the kernel-state scheduler corresponding to the next hardware unit; if the next hardware unit is a second-category hardware unit, the kernel-state scheduler sends the second task message to the kernel-state driver corresponding to the next hardware unit; The first type of hardware units include NPU units and / or GPU units, and the second type of hardware units include at least one of ASIC units, FPGA units and CPLD units.

7. A task processing method, characterized in that: Applied to an electronic device, the electronic device includes a processor and a plurality of hardware units, for each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor, and for each kernel state driver, the method includes: When receiving the first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, wherein the second task message includes processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message; The kernel-mode driver obtains a working order of a plurality of hardware units from the second task message, and determines whether the hardware unit is the last hardware unit based on the working order; If so, the kernel-mode driver stores the processed data to a designated storage medium; If not, the kernel-mode driver determines a subsequent hardware unit of the hardware unit based on the working order, and sends the second task message to the kernel-mode driver corresponding to the subsequent hardware unit.

8. A task processing method, characterized in that: Applied to an electronic device including a processor and a plurality of hardware units, for each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor and corresponds to a user state API in the user state of the processor, and for each kernel state driver, the method includes: When receiving the first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, wherein the second task message includes processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message; The kernel-mode driver obtains the first identity corresponding to the requesting hardware unit from the second task message; wherein, when a hardware unit expects another hardware unit to jointly complete the same task, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit; If the second identity identifier corresponding to the kernel-state driver is different from the first identity identifier, the kernel-state driver sends the second task message to the kernel-state driver corresponding to the requesting hardware unit; If the second identity identifier is the same as the first identity identifier, the kernel-state driver determines whether the auxiliary hardware unit has completed task processing; if so, the second task message is sent to the user-state API; if not, the second task message is sent to the kernel-state driver corresponding to the auxiliary hardware unit.

9. An electronic device, characterized in that: The electronic device includes a processor and a plurality of hardware units, the plurality of hardware units include a first type of hardware unit, and the electronic device further includes a kernel state scheduler and a kernel state driver corresponding to the first type of hardware unit in the kernel state of the processor; wherein: The kernel state scheduler is used to send the first task message to the kernel state driver when receiving the first task message, wherein the first task message includes data to be processed; The kernel state driver is used to send the first task message to the first type of hardware unit; The first type of hardware unit is used to process the data to be processed to obtain processed data, and send a second task message to the kernel state driver, wherein the second task message includes the processed data and the task type; The kernel state driver is further used to send the second task message to the kernel state scheduler; The kernel-mode scheduler is further configured to receive the second task message; if the task type is a kernel-mode binding call, obtain the working order of multiple hardware units from the second task message, and determine whether the first-type hardware unit is the last hardware unit based on the working order; if so, store the processed data in a designated storage medium; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit; The kernel-mode binding call indicates calling the kernel-mode driver of each hardware unit in sequence based on the working order to call the hardware unit to work.

10. The electronic device according to claim 9, characterized in that: in, The first type of hardware unit corresponds to a user state API in the user state of the processor, and the kernel state scheduler is further used to send the second task message to the user state API if the task type is a user state successive call, so that the application obtains the second task message from the user state API and obtains processed data from the second task message; wherein the user state successive call means calling the hardware unit to perform work by calling the user state API of the hardware unit; Wherein, the kernel-mode scheduler is further used for, if the task type is kernel-mode sequential calls, obtaining the first identity identifier corresponding to the requesting hardware unit from the second task message; if the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, sending the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the requesting hardware unit; if the second identity identifier is the same as the first identity identifier, determining whether the auxiliary hardware unit has completed the task processing; if so, sending the second task message to the user-mode API; if not, sending the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the auxiliary hardware unit; Among them, the kernel state successive calls represent calling the hardware unit to work by calling the kernel state driver of the hardware unit; among them, when a hardware unit expects another hardware unit to complete the same task together, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit.

Images