Task processing method and device

Abstract

The invention provides a task processing method and equipment, and the method comprises the steps: a kernel mode scheduler sends a first task message to a kernel mode driver when receiving the first task message, and the kernel mode driver sends the first task message to a first type of hardware unit; the kernel mode scheduler receives a second task message returned by the kernel mode driver; if the task type is kernel mode binding calling, the kernel mode scheduler obtains the working sequence of the multiple hardware units from the second task message, and whether the first type of hardware unit is the last hardware unit or not is determined based on the working sequence; if yes, the kernel mode scheduler stores the processed data to a specified storage medium; and if not, the kernel mode scheduler sends the second task message to the kernel mode scheduler or the kernel mode driver corresponding to the next hardware unit of the first type of hardware units. Through the technical scheme of the invention, the resource overhead can be saved.

Description

Technical Field

[0001] The present application relates to the field of edge computing technology, and in particular to a task processing method and device. Background Art

[0002] The operating system is the core of managing computer hardware and software resources. It is responsible for allocating and scheduling computer resources and providing various services to support the operation of applications. In the operating system, there are user state and kernel state, which define the interaction between applications and the operating system.

[0003] In an operating system, kernel state is the state in which operating system programs are run and hardware is operated, and it has the highest privileges. Kernel state is also called kernel mode. In this mode, all instructions can be executed, all memory addresses can be accessed, all interrupt requests can be responded to, and hardware events and system calls can be processed. User state is the state in which user programs are run, and its privileges are limited. User state is also called user mode. In this mode, applications have limited access to system resources and can only run in a specific space designated by the operating system. They cannot directly access hardware devices, and all access to hardware devices must be performed through the operating system.

[0004] The switch between kernel state and user state is mainly achieved through interrupts and system calls. When a process in user state needs to execute kernel state code, it will fall into kernel state through a system call. This process involves saving the context of user state (such as stack information) to kernel state, and then loading the new kernel state context to start execution. When the kernel state code is executed, the control will be returned to user state and the original context will be restored, completing the switch from user state to kernel state. Similarly, when an interrupt occurs, it will automatically switch to kernel state, execute the corresponding interrupt handler, and return to user state after processing.

[0005] During the task processing, it is necessary to switch repeatedly between user state and kernel state. The repeated switching between user state and kernel state will generate a lot of CPU overhead and consume a lot of resources. Summary of the invention

[0006] The present application provides a task processing method, which is applied to an electronic device including a processor and a plurality of hardware units, wherein the plurality of hardware units include a first type of hardware unit, and the first type of hardware unit corresponds to a kernel state scheduler and a kernel state driver in the kernel state of the processor, and the method includes: When receiving the first task message, the kernel state scheduler sends the first task message to the kernel state driver, and the kernel state driver sends the first task message to the first type of hardware unit; The kernel-mode scheduler receives a second task message returned by the kernel-mode driver, wherein the second task message includes processed data and a task type; wherein the processed data is obtained after the first type of hardware unit processes the data to be processed carried by the first task message; If the task type is a kernel-mode binding call, the kernel-mode scheduler obtains the working order of multiple hardware units from the second task message, and determines whether the first-type hardware unit is the last hardware unit based on the working order; wherein the kernel-mode binding call means calling the kernel-mode driver of each hardware unit in turn based on the working order to call the hardware unit to work; If so, the kernel-mode scheduler stores the processed data to a designated storage medium; If not, the kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit.

[0007] The present application provides a task processing method, which is applied to an electronic device, wherein the electronic device includes a processor and a plurality of hardware units, and for each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor, and for each kernel state driver, the method includes: When receiving the first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, wherein the second task message includes processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message; The kernel-mode driver obtains a working order of a plurality of hardware units from the second task message, and determines whether the hardware unit is the last hardware unit based on the working order; If so, the kernel-mode driver stores the processed data to a designated storage medium; If not, the kernel-mode driver determines a subsequent hardware unit of the hardware unit based on the working order, and sends the second task message to the kernel-mode driver corresponding to the subsequent hardware unit.

[0008] The present application provides a task processing method, which is applied to an electronic device including a processor and a plurality of hardware units. For each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor and corresponds to a user state API in the user state of the processor. For each kernel state driver, the method includes: When receiving the first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, wherein the second task message includes processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message; The kernel-mode driver obtains the first identity corresponding to the requesting hardware unit from the second task message; wherein, when a hardware unit expects another hardware unit to jointly complete the same task, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit; If the second identity identifier corresponding to the kernel-state driver is different from the first identity identifier, the kernel-state driver sends the second task message to the kernel-state driver corresponding to the requesting hardware unit; If the second identity identifier is the same as the first identity identifier, the kernel-state driver determines whether the auxiliary hardware unit has completed task processing; if so, the second task message is sent to the user-state API; if not, the second task message is sent to the kernel-state driver corresponding to the auxiliary hardware unit.

[0009] The present application provides an electronic device, the electronic device comprising a processor and a plurality of hardware units, the plurality of hardware units comprising a first type of hardware unit, the electronic device further comprising a kernel state scheduler and a kernel state driver corresponding to the first type of hardware unit in the kernel state of the processor; wherein: The kernel state scheduler is used to send the first task message to the kernel state driver when receiving the first task message, wherein the first task message includes data to be processed; The kernel state driver is used to send the first task message to the first type of hardware unit; The first type of hardware unit is used to process the data to be processed to obtain processed data, and send a second task message to the kernel state driver, wherein the second task message includes the processed data and the task type; The kernel state driver is further used to send the second task message to the kernel state scheduler; The kernel-mode scheduler is further configured to receive the second task message; if the task type is a kernel-mode binding call, obtain the working order of multiple hardware units from the second task message, and determine whether the first-type hardware unit is the last hardware unit based on the working order; if so, store the processed data in a designated storage medium; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit; The kernel-mode binding call indicates calling the kernel-mode driver of each hardware unit in sequence based on the working order to call the hardware unit to work.

[0010] The present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned task processing method is implemented.

[0011] The present application provides an electronic device, which includes: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; wherein the processor is used to execute the machine-executable instructions to implement the above-mentioned task processing method.

[0012] The present application provides a machine-readable storage medium, which stores machine-executable instructions that can be executed by a processor; wherein the processor is used to execute the machine-executable instructions, and implement the above-mentioned task processing method when the machine-executable instructions are executed.

[0013] It can be seen from the above technical solutions that in the embodiments of the present application, by deploying a kernel state scheduler and a kernel state driver in the kernel state of the processor, during the task processing process, the task processing is completed in the kernel state without the need to repeatedly switch between the user state and the kernel state, thereby saving CPU overhead, saving resource overhead, and being able to save switching overhead between the user state and the kernel state (such as time overhead and resource overhead). By completing task processing in the kernel state, the requirements of high real-time performance of the task can be met, and the requirements of mutual calls between hardware units for collaborative work can be met. By deploying a kernel state scheduler, the hardware unit simultaneously supports user state successive calls, kernel state binding calls, and kernel state successive calls, so that the hardware unit has both flexibility and efficiency, can efficiently respond to real-time tasks, and respond to occasional tasks with flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1A is a flowchart of a task processing method in one embodiment of the present application; Figure 1B is a flowchart of a task processing method in one embodiment of the present application; Figure 1C is a flowchart of a task processing method in one embodiment of the present application; Figure 2 It is a schematic diagram of an application scenario of a CPU and multiple hardware units in one embodiment of the present application; Figure 3 is a schematic diagram of the collaborative work between multiple hardware units in one implementation of the present application; Figure 4is a schematic diagram of the collaborative work between multiple hardware units in one implementation of the present application; Figure 5 is a schematic diagram of the collaborative work between multiple hardware units in one implementation of the present application; Figure 6 It is a schematic diagram of successive calls to the kernel state in one implementation of the present application; Figure 7 is a schematic diagram of the collaborative work between multiple hardware units in one implementation of the present application; Figure 8 It is a hardware structure diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0015] In an embodiment of the present application, a task processing method is proposed. The method can be applied to an electronic device (i.e., a device for implementing task processing). The electronic device may include a processor and multiple hardware units. The multiple hardware units include a first type of hardware unit. The first type of hardware unit corresponds to a kernel state scheduler and a kernel state driver in the kernel state of the processor. Figure 1A FIG. 4 is a flow chart of the method, which includes: Step 101: When receiving a first task message, the kernel-state scheduler sends the first task message to a kernel-state driver, and the kernel-state driver sends the first task message to a first-type hardware unit.

[0016] Step 102: The kernel-mode scheduler receives a second task message returned by the kernel-mode driver, where the second task message may include processed data and a task type; wherein the processed data may be obtained by the first type of hardware unit processing the data to be processed carried by the first task message.

[0017] Step 103: If the task type is a kernel-mode binding call, the kernel-mode scheduler obtains the working order of the multiple hardware units from the second task message, and determines whether the first type of hardware unit is the last hardware unit based on the working order. Exemplarily, the kernel-mode binding call means calling the kernel-mode driver of each hardware unit in turn based on the working order to call the hardware unit to work.

[0018] If yes, then step 104 may be executed; if no, then step 105 may be executed.

[0019] Step 104: The kernel-mode scheduler stores the processed data to a designated storage medium.

[0020] Step 105: The kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-category hardware unit.

[0021] Exemplarily, the first type of hardware unit corresponds to the user-state API in the user state of the processor. After the kernel-state scheduler receives the second task message returned by the kernel-state driver, if the task type is user-state successive calls, the kernel-state scheduler sends the second task message to the user-state API, so that the application obtains the second task message from the user-state API and obtains processed data from the second task message; wherein, user-state successive calls means calling the hardware unit to perform work by calling the user-state API of the hardware unit.

[0022] Exemplarily, after the kernel-state scheduler receives the second task message returned by the kernel-state driver, if the task type is kernel-state sequential call, the kernel-state scheduler obtains the first identity corresponding to the requesting hardware unit from the second task message; wherein kernel-state sequential call means calling the hardware unit to work by calling the kernel-state driver of the hardware unit. If the second identity corresponding to the first type of hardware unit is different from the first identity, the kernel-state scheduler determines that the first type of hardware unit is an auxiliary hardware unit, and sends the second task message to the kernel-state scheduler or kernel-state driver corresponding to the requesting hardware unit.

[0023] When one hardware unit expects another hardware unit to complete the same task together, the one hardware unit is a requesting hardware unit, and the other hardware unit is an assisting hardware unit.

[0024] Exemplarily, after the kernel-state scheduler obtains the first identity identifier corresponding to the requesting hardware unit from the second task message, if the second identity identifier is the same as the first identity identifier, the kernel-state scheduler determines that the first type of hardware unit is the requesting hardware unit, and determines whether the auxiliary hardware unit has completed the task processing; wherein, if the auxiliary hardware unit has returned the task message, the auxiliary hardware unit has completed the task processing; if the auxiliary hardware unit has not returned the task message, the auxiliary hardware unit has not completed the task processing.

[0025] If so, the kernel-state scheduler can send the second task message to the user-state API so that the application can obtain the second task message from the user-state API; if not, the kernel-state scheduler can send the second task message to the kernel-state scheduler or kernel-state driver corresponding to the auxiliary hardware unit.

[0026] Exemplarily, when the kernel-state scheduler receives multiple first task messages, it can obtain the task priority of each first task message from the first task message; based on the task priority of each first task message, the kernel-state scheduler selects the first task message with the highest task priority, and sends the first task message with the highest task priority to the kernel-state driver.

[0027] Exemplarily, the plurality of hardware units may further include a second type of hardware unit, the second type of hardware unit corresponds to a kernel-state driver in the kernel state of the processor, and the second type of hardware unit corresponds to a user-state API in the user state of the processor. Wherein, when sending the second task message to the kernel-state scheduler or kernel-state driver corresponding to the next hardware unit of the first type of hardware unit, if the next hardware unit is the first type of hardware unit, the kernel-state scheduler sends the second task message to the kernel-state scheduler corresponding to the next hardware unit; if the next hardware unit is the second type of hardware unit, the kernel-state scheduler sends the second task message to the kernel-state driver corresponding to the next hardware unit. Wherein, the first type of hardware unit may include but is not limited to an NPU (Neural-network Processing Unit) unit and / or a GPU (Graphics Processing Unit) unit, and the second type of hardware unit may include but is not limited to at least one of an ASIC (Application Specific Intergrated Circuits) unit, an FPGA (Field Programmable Gate Array) unit, and a CPLD (Complex Programmable logic device) unit.

[0028] It can be seen from the above technical solutions that in the embodiments of the present application, by deploying a kernel state scheduler and a kernel state driver in the kernel state of the processor, during the task processing process, the task processing is completed in the kernel state without the need to repeatedly switch between the user state and the kernel state, thereby saving CPU overhead, saving resource overhead, and being able to save switching overhead between the user state and the kernel state (such as time overhead and resource overhead). By completing task processing in the kernel state, the requirements of high real-time performance of the task can be met, and the requirements of mutual calls between hardware units for collaborative work can be met. By deploying a kernel state scheduler, the hardware unit simultaneously supports user state successive calls, kernel state binding calls, and kernel state successive calls, so that the hardware unit has both flexibility and efficiency, can efficiently respond to real-time tasks, and respond to occasional tasks with flexibility.

[0029] In an embodiment of the present application, a task processing method is proposed, which can be applied to an electronic device (i.e., a device for implementing task processing), and the electronic device may include a processor and multiple hardware units. For each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor. Figure 1B FIG. 1 is a flow chart of the task processing method, which may include: Step 111. For each kernel-state driver, when receiving a first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, where the second task message may include processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message.

[0030] Step 112: The kernel-mode driver obtains the working order of the multiple hardware units from the second task message, and determines whether the hardware unit is the last hardware unit based on the working order.

[0031] If yes, step 113 may be executed; if no, step 114 may be executed.

[0032] Step 113: The kernel-mode driver stores the processed data to a designated storage medium.

[0033] Step 114: The kernel-mode driver determines a subsequent hardware unit of the hardware unit based on the working order, and sends the second task message to the kernel-mode driver corresponding to the subsequent hardware unit.

[0034] As can be seen from the above technical solutions, in the embodiment of the present application, by deploying a kernel state scheduler and a kernel state driver in the kernel state of the processor, during the task processing process, the task processing is completed in the kernel state without repeatedly switching between the user state and the kernel state, thereby saving CPU overhead, saving resource overhead, and being able to save switching overhead between the user state and the kernel state (such as time overhead and resource overhead). By completing task processing in the kernel state, the requirements of high real-time performance of the task can be met, the requirements of mutual calls between hardware units for collaborative work can be met, and real-time tasks can be responded to efficiently.

[0035] In an embodiment of the present application, a task processing method is proposed, which can be applied to an electronic device (i.e., a device for implementing task processing), and the electronic device may include a processor and multiple hardware units. For each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor, and the hardware unit corresponds to a user state API (Application Programming Interface) in the user state of the processor. Figure 1C FIG. 1 is a flow chart of the task processing method, which may include: Step 121. For each kernel-state driver, when receiving a first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, where the second task message may include processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message.

[0036] Step 122, the kernel-mode driver obtains the first identity identifier corresponding to the requesting hardware unit from the second task message; wherein, when one hardware unit expects another hardware unit to complete the same task together, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit.

[0037] Step 123: Determine whether the second identity identifier corresponding to the kernel-mode driver is the same as the first identity identifier.

[0038] If not, that is, the second identity identifier is different from the first identity identifier, step 124 may be executed.

[0039] If yes, that is, the second identity identifier is the same as the first identity identifier, step 125 may be executed.

[0040] Step 124: The kernel-mode driver sends the second task message to the kernel-mode driver corresponding to the requesting hardware unit.

[0041] Step 125, the kernel-state driver determines whether the auxiliary hardware unit has completed task processing; if so, that is, the task processing has been completed, the second task message is sent to the user-state API; if not, that is, the task processing has not been completed, the second task message is sent to the kernel-state driver corresponding to the auxiliary hardware unit.

[0042] It can be seen from the above technical solutions that in the embodiments of the present application, by deploying a kernel state scheduler and a kernel state driver in the kernel state of the processor, during the task processing process, the task processing is completed in the kernel state without repeatedly switching between the user state and the kernel state, saving CPU overhead, saving resource overhead, and saving switching overhead between the user state and the kernel state (such as time overhead and resource overhead). By completing task processing in the kernel state, the requirements of high real-time performance of the task can be met, and the requirements of mutual calls between hardware units for collaborative work can be met, so that the hardware units have both flexibility and efficiency, and can respond to occasional tasks with flexibility.

[0043] The above technical solutions of the embodiments of the present application are described below in combination with specific application scenarios.

[0044] The task processing device in this embodiment can be an electronic device, such as an edge device. The edge device is a variety of computing devices deployed on edge nodes, such as smart phones, tablets, smart watches, smart speakers, smart homes, routers, cameras, etc. There is no restriction on the type of this electronic device.

[0045] Exemplarily, the electronic device may include a processor (such as a CPU (Central Processing Unit, central processing unit), etc.) and multiple hardware units, and the multiple hardware units may be homogeneous hardware units, that is, all hardware units are of the same type, such as all hardware units are NPU units, or all hardware units are GPU units, or all hardware units are ASIC units, or all hardware units are FPGA units, or all hardware units are CPLD units. Alternatively, the multiple hardware units may also be heterogeneous hardware units, that is, all hardware units may include at least two types of NPU units, GPU units, ASIC units, FPGA units, and CPLD units. For example, the multiple hardware units include at least one NPU unit and at least one ASIC unit, or the multiple hardware units include at least one NPU unit and at least one FPGA unit, or the multiple hardware units include at least one NPU unit, at least one ASIC unit, and at least one FPGA unit. In this embodiment, the types of the multiple hardware units are not limited.

[0046] For example, heterogeneous hardware units refer to electronic devices that include NPU units, ASIC units, FPGA units, and other heterogeneous units in addition to the CPU. These heterogeneous units are controlled by the CPU and can work together around the CPU. For example, ASIC units, NPU units, and CPUs form a heterogeneous system. For the NPU unit in the heterogeneous hardware unit, the NPU unit is a hardware module specifically used to accelerate neural network calculations, aiming to improve the processing efficiency and performance of machine learning and deep learning tasks.

[0047] In the scenario of a CPU (i.e., processor) and multiple hardware units, the multiple hardware units are controlled by the CPU. The CPU and operating system (such as the Linux operating system) distinguish between kernel state and user state. For each hardware unit, the hardware unit corresponds to a driver in the kernel state of the CPU. For the sake of distinction, it can be recorded as a kernel state driver in the following. The hardware unit corresponds to an API in the user state of the CPU. For the sake of distinction, it can be recorded as a user state API in the following. See Figure 2 The figure shows an application scenario diagram of a CPU and multiple hardware units. Taking hardware unit 1 and hardware unit 2 as examples, hardware unit 1 corresponds to kernel state driver 1 in the kernel state of the CPU, and hardware unit 1 corresponds to user state API 1 in the user state of the CPU. Hardware unit 2 corresponds to kernel state driver 2 in the kernel state of the CPU, and hardware unit 2 corresponds to user state API 2 in the user state of the CPU.

[0048] For example, a kernel-mode driver is a program running in kernel mode, which is used to control the operation of a hardware unit, and may include configuration registers, signals for commanding the hardware unit to start, and interrupt signals for receiving the hardware unit. A user-mode API is an interface provided to the user, and the kernel-mode driver of the hardware unit is called internally by the user-mode API, that is, the user controls the hardware unit by calling the user-mode API. For ease of understanding, the user-mode API is used in this embodiment to represent the user-mode program and user-mode program interface of the hardware unit as a whole.

[0049] When the application needs to call hardware unit 1 to process data, it can call user state API 1 to send task message 1 to kernel state driver 1 (kernel state driver 1 is a driver). User state API 1 is used to convert task message 1 into a message format that kernel state driver 1 can understand. Kernel state driver 1 sends task message 1 to hardware unit 1. Kernel state driver 1 is used to convert task message 1 into a message format that hardware unit 1 can understand. After receiving task message 1, hardware unit 1 can process the data to be processed in task message 1 and return task message 2 to kernel state driver 1. Task message 2 includes processed data (i.e., the processing result of the data to be processed). Kernel state driver 1 returns task message 2 to user state API 1. Kernel state driver 1 is used to convert task message 2 into a message format that the application can understand. The application can obtain task message 2 from user state API 1 and then obtain the processed data.

[0050] When the application needs to call hardware unit 2 to process data (such as processed data returned by hardware unit 1), it can call user state API 2 to send task message 3 to kernel state driver 2, and kernel state driver 2 sends task message 3 to hardware unit 2. After receiving task message 3, hardware unit 2 can process the data to be processed in task message 3 and return task message 4 to kernel state driver 2, and task message 4 includes processed data. Kernel state driver 2 returns task message 4 to user state API 2, and the application can obtain task message 4 from user state API 2 and then obtain processed data.

[0051] In the above scenario, kernel-mode driver 1 is used to start hardware unit 1, send task messages of user-mode API 1 to hardware unit 1, and send task messages of hardware unit 1 to user-mode API 1. User-mode API 1 is an interface provided to applications for calling, and user-mode API 1 calls kernel-mode driver 1 to control the hardware unit. Kernel-mode driver 2 is used to start hardware unit 2, send task messages of user-mode API 2 to hardware unit 2, and send task messages of hardware unit 2 to user-mode API 2. User-mode API 2 is an interface provided to applications for calling, and user-mode API 2 calls kernel-mode driver 2 to control the hardware unit.

[0052] In one possible implementation, the same task can be implemented by multiple hardware units, that is, multiple hardware units need to work together. For example, a task can be divided into three subtasks, and subtask 1 is used to pre-process the image (ISP processing), such as white balance processing, noise elimination, color correction processing, automatic exposure control, etc. Subtask 2 is used to perform artificial intelligence processing on the image, such as detecting the image based on the network model, classifying the image based on the network model, and segmenting the image based on the network model. Subtask 3 is used to perform fusion processing on the image, such as fusing the artificial intelligence processing results with the original image. Of course, the above is just an example of multiple hardware units implementing the same task.

[0053] On this basis, three hardware units can be used to process the three subtasks respectively. Hardware unit 1 is used to process subtask 1, hardware unit 2 is used to process subtask 2, and hardware unit 3 is used to process subtask 3. On this basis, see Figure 3 As shown, it is a schematic diagram of the collaborative work between multiple hardware units, taking hardware unit 1, hardware unit 2 and hardware unit 3 as examples, such as hardware unit 1 is ASIC unit 1, hardware unit 2 is ASIC unit 2, and hardware unit 3 is NPU unit.

[0054] See also Figure 3 As shown, hardware unit 1 corresponds to kernel-state driver 1 in the kernel state of the CPU, and hardware unit 1 corresponds to user-state API 1 in the user state of the CPU. Hardware unit 2 corresponds to kernel-state driver 2 in the kernel state of the CPU, and hardware unit 2 corresponds to user-state API 2 in the user state of the CPU. Hardware unit 3 corresponds to kernel-state driver 3 in the kernel state of the CPU, and hardware unit 3 corresponds to user-state API 3 in the user state of the CPU.

[0055] The application calls the user-state API1 to send task message 1 to the kernel-state driver 1. Task message 1 includes the image to be processed a1 (i.e., the original image). The kernel-state driver 1 sends task message 1 to the hardware unit 1. The hardware unit 1 performs pre-processing (ISP processing) on ​​the image to be processed a1 in the task message 1 to obtain the processed image a2 (i.e., the pre-processed image), and returns task message 2 to the kernel-state driver 1. Task message 2 includes the processed image a2. The kernel-state driver 1 returns task message 2 to the user-state API1. The application obtains task message 2 from the user-state API1 and then obtains the processed image a2.

[0056] The application calls the user-state API2 to send a task message 3 to the kernel-state driver 2. The task message 3 includes the processed image a2. The kernel-state driver 2 sends the task message 3 to the hardware unit 2. The hardware unit 2 performs artificial intelligence processing on the processed image a2 in the task message 3, such as detecting the processed image a2 based on the network model to obtain the image detection result a3 (such as a rectangular frame of the license plate area). The hardware unit 2 returns the task message 4 to the kernel-state driver 2. The task message 4 includes the processed image a2 and the image detection result a3. The kernel-state driver 2 returns the task message 4 to the user-state API2. The application obtains the task message 4 from the user-state API2 and then obtains the processed image a2 and the image detection result a3.

[0057] The application calls the user-state API3 to send a task message 5 to the kernel-state driver 3. The task message 5 includes the processed image a2 and the image detection result a3. The kernel-state driver 3 sends the task message 5 to the hardware unit 3. The hardware unit 3 performs a fusion process on the processed image a2 and the image detection result a3 in the task message 5 to obtain the image fusion result a4 (such as superimposing the image detection result a3 on the processed image a2). The hardware unit 3 returns a task message 6 to the kernel-state driver 3. The task message 6 includes the image fusion result a4. The kernel-state driver 3 returns the task message 6 to the user-state API3. The application obtains the task message 6 from the user-state API3 and then obtains the image fusion result a4. At this point, the processing process of the same task is completed.

[0058] Obviously, under the above collaborative working mode, every time the application calls the user-mode API, a switch between the user-mode and kernel-mode will occur, and this state switching will generate a lot of CPU overhead, thereby consuming a lot of resources. If the user-mode API is called frequently, the CPU overhead caused by the state switching will increase significantly. For example, the state switching refers to the state switching between the user-mode and kernel-mode. When calling a hardware unit through the user-mode API, the process of calling the hardware unit needs to go through the kernel-mode driver.

[0059] In view of the above findings, a task processing method is proposed in an embodiment of the present application. By creating a task of the kernel state binding call type, the kernel state binding call of multiple hardware units can be implemented. The kernel state binding call means calling the kernel state driver of each hardware unit in turn based on the working order to call the hardware unit to work, and the working order is determined by the application according to actual needs. For the kernel state binding call, the working order between the hardware units has been created in the initialization stage, and then the kernel state driver controls the work of each hardware unit according to the working order, and does not enter the user state during the work. By implementing the kernel state binding call of multiple hardware units, the user state is not entered during the task processing, and repeated switching between the user state and the kernel state can be avoided, saving the switching overhead between states, thereby reducing the CPU overhead and saving resources.

[0060] For example, a task can be divided into three subtasks. Subtask 1 is used for image pre-processing, subtask 2 is used for image artificial intelligence processing, and subtask 3 is used for image fusion processing. The three subtasks are processed by three hardware units respectively. Hardware unit 1 is used to process subtask 1, hardware unit 2 is used to process subtask 2, and hardware unit 3 is used to process subtask 3. On this basis, see Figure 4 As shown, it is a schematic diagram of the collaborative work between multiple hardware units, taking hardware unit 1, hardware unit 2 and hardware unit 3 as examples. For example, hardware unit 1 is ASIC unit 1, hardware unit 2 is ASIC unit 2, and hardware unit 3 is NPU unit.

[0061] See also Figure 4 As shown, hardware unit 1 corresponds to kernel-state driver 1 in the kernel state of the CPU, and hardware unit 1 corresponds to user-state API 1 in the user state of the CPU. Hardware unit 2 corresponds to kernel-state driver 2 in the kernel state of the CPU, and hardware unit 2 corresponds to user-state API 2 in the user state of the CPU. Hardware unit 3 corresponds to kernel-state driver 3 in the kernel state of the CPU, and hardware unit 3 corresponds to user-state API 3 in the user state of the CPU.

[0062] In order to implement kernel-mode binding calls, the application needs to determine the working order of multiple hardware units, such as the working order can be represented by hardware unit 1-hardware unit 2-hardware unit 3. In this way, the kernel-mode binding call represents that based on the working order, hardware unit 1 is first called to work through kernel-mode driver 1 of hardware unit 1, then hardware unit 2 is called to work through kernel-mode driver 2 of hardware unit 2, and then hardware unit 3 is called to work through kernel-mode driver 3 of hardware unit 3.

[0063] In the above application scenario, the task processing method may include the following steps: Step S11, the application starts a kernel-mode binding call type task and sends a task message 1 to the kernel-mode driver 1. For example, the application can call the user-mode API 1 to send the task message 1 to the kernel-mode driver 1, or can directly send the task message 1 to the kernel-mode driver 1, and there is no restriction on this.

[0064] The task message 1 may include data to be processed and a work order. The data to be processed may be an image a1 to be processed (ie, an original image), and the work order may represent hardware unit 1 - hardware unit 2 - hardware unit 3.

[0065] Step S12 : After receiving the task message 1 (ie, the first task message), the kernel state driver 1 sends the task message 1 to the hardware unit 1 corresponding to the kernel state driver 1 .

[0066] Step S13, the hardware unit 1 pre-processes the image a1 to be processed in the task message 1 to obtain the processed image a2 (i.e., processed data), and returns the task message 2 to the kernel state driver 1. The task message 2 may include the processed image a2 and the working order (obtained from the task message 1).

[0067] Step S14: kernel-mode driver 1 receives task message 2 (i.e., second task message) returned by hardware unit 1, obtains the working order of multiple hardware units from task message 2, and determines whether hardware unit 1 is the last hardware unit based on the working order. Since the working order indicates hardware unit 1-hardware unit 2-hardware unit 3, hardware unit 1 is not the last hardware unit, and step S15 is executed.

[0068] Step S15 , kernel-state driver 1 determines the next hardware unit (ie, hardware unit 2 ) of hardware unit 1 based on the working order, and sends task message 2 to kernel-state driver 2 corresponding to hardware unit 2 .

[0069] Obviously, kernel state driver 1 directly sends task message 2 to kernel state driver 2, instead of sending task message 2 to user state API 1 first. The application obtains task message 2 from user state API 1 and then sends task message 2 to kernel state driver 2. In this way, sending task message 2 to user state can be avoided, repeated switching between user state and kernel state can be avoided, and the switching overhead between states can be saved.

[0070] Step S16 : After receiving the task message 2 (ie, the first task message), the kernel state driver 2 sends the task message 2 to the hardware unit 2 corresponding to the kernel state driver 2 .

[0071] Step S17, the hardware unit 2 performs artificial intelligence processing on the processed image a2 in the task message 2 (the processed image a2 is the data to be processed for the hardware unit 2), obtains the image detection result a3 (such as the rectangular frame of the license plate area, etc.), and returns the task message 3 to the kernel state driver 2. The task message 3 may include the processed image a2, the image detection result a3 and the working order. For example, the processed image a2 and the image detection result a3 are used as the processed data together, and the image detection result a3 may also be used as the processed data alone.

[0072] Step S18, kernel-mode driver 2 receives task message 3 (i.e., second task message) returned by hardware unit 2, obtains the working order of multiple hardware units from task message 3, and determines whether hardware unit 2 is the last hardware unit based on the working order. Since the working order indicates hardware unit 1-hardware unit 2-hardware unit 3, hardware unit 2 is not the last hardware unit, and step S19 is executed.

[0073] Step S19 , the kernel-state driver 2 determines the next hardware unit (ie, hardware unit 3 ) of the hardware unit 2 based on the working order, and sends the task message 3 to the kernel-state driver 3 corresponding to the hardware unit 3 .

[0074] Step S20 : ​​After receiving the task message 3 (ie, the first task message), the kernel state driver 3 sends the task message 3 to the hardware unit 3 corresponding to the kernel state driver 3 .

[0075] Step S21, the hardware unit 3 performs fusion processing on the processed image a2 and the image detection result a3 in the task message 3 to obtain the image fusion result a4, the processed image a2 and the image detection result a3 are the data to be processed, and the image fusion result a4 is the processed data. The hardware unit 3 returns the task message 4 to the kernel state driver 3, and the task message 4 may include the image fusion result a4 and the working order.

[0076] Step S22: kernel-mode driver 3 receives task message 4 (i.e., second task message) returned by hardware unit 3, obtains the working order of multiple hardware units from task message 4, and determines whether hardware unit 3 is the last hardware unit based on the working order. Since the working order indicates hardware unit 1-hardware unit 2-hardware unit 3, hardware unit 3 is the last hardware unit, and step S23 is executed.

[0077] Step S23 : the kernel-mode driver 3 stores the image fusion result a4 (ie, the processed data) in the task message 4 to a designated storage medium (eg, a memory, a hard disk, or other storage medium).

[0078] Obviously, when hardware unit 3 is the last hardware unit, kernel-mode driver 3 ends the task message processing process and can store image fusion result a4 to the specified storage medium instead of returning the task message to the user-mode API. In addition, kernel-mode driver 3 can send a task end message to the application, indicating that the task has been processed. When the application needs to obtain image fusion result a4, the application can directly read image fusion result a4 from the specified storage medium without going through the user-mode API.

[0079] Obviously, in the above collaborative working mode, the application is only responsible for task initiation, and then kernel state driver 1, kernel state driver 2 and kernel state driver 3 will complete the task sequentially in kernel state, and will not return to user state API during task processing. When the task is completed, the image fusion result a4 will be stored in the specified storage medium. In this way, there is no need to switch repeatedly between user state and kernel state, which can save CPU overhead. Moreover, when tasks are completed sequentially in kernel state, the high real-time requirements of the task can also be met.

[0080] In a possible implementation, for tasks of kernel-mode binding call type, multiple hardware units process the tasks one by one, thereby flexibly calling multiple hardware units one by one, and the working order is specified by the application, that is, the user perceives the working order of multiple hardware units. Kernel-mode binding call is suitable for tasks with a relatively fixed working order, many hardware units, frequent hardware unit calls, and high real-time requirements.

[0081] For example, when a task is used to encode images captured by a camera, since it involves the image acquisition process, image pre-processing process, image artificial intelligence processing process, and image encoding process, the task can be completed through the coordination of multiple hardware units. Since the camera captures images in real time, a large number of images need to be encoded, and the working order of these images is the same, the hardware units need to be called frequently. In summary, it can be seen that such tasks have a fixed working order, many hardware units, frequent hardware unit calls, and high real-time requirements. Therefore, such tasks can be implemented through kernel state binding calls.

[0082] In addition to the kernel state binding call type task, a kernel state sequential call type task is proposed in the embodiment of the present application, which can realize the kernel state sequential call of multiple hardware units. Kernel state sequential call means calling the hardware unit to work sequentially by calling the kernel state driver of the hardware unit. For kernel state sequential call, it is necessary to meet the requirements of multiple hardware units calling each other to work together.

[0083] Different from kernel-mode binding calls, multiple hardware units include a requesting hardware unit and at least one auxiliary hardware unit. For example, when hardware unit 1 expects hardware unit 2 to complete the same task together, hardware unit 1 is the requesting hardware unit and hardware unit 2 is the auxiliary hardware unit. For another example, when hardware unit 1 expects hardware unit 2 and hardware unit 3 to complete the same task together, hardware unit 1 is the requesting hardware unit and hardware unit 2 and hardware unit 3 are auxiliary hardware units.

[0084] For kernel-mode sequential calls, the application does not need to specify the working order, but the kernel-mode driver corresponding to the requesting hardware unit pre-configures the working order of the auxiliary hardware unit, so that the application / user does not perceive the working order of the auxiliary hardware unit. In addition, for kernel-mode sequential call type tasks, multiple hardware units process the tasks one by one, but the task message of the auxiliary hardware unit can only be returned to the requesting hardware unit, and the task message cannot be sent to another auxiliary hardware unit.

[0085] Kernel-state successive calls are suitable for occasional hardware unit calls with high real-time requirements. For example, for occasional tasks such as human detection and vehicle detection, that is, human detection and vehicle detection are only performed on a single frame or a few frames of images, such tasks can be implemented through kernel-state successive calls. For example, during the JPEG encoding process, the JPEG encoding hardware unit expects to call the NPU hardware unit to assist in the encoding work, and expects to hide this intermediate calling process from the user. Such tasks are implemented through kernel-state successive calls, that is, the JPEG encoding hardware unit initiates successive calls to the NPU hardware unit through the NPU kernel-state driver.

[0086] For kernel-mode sequential calls, the kernel-mode driver corresponding to the requested hardware unit pre-configures the working order of the auxiliary hardware unit, and then the kernel-mode driver controls the work of each hardware unit according to the working order, and does not enter the user mode during the work. By implementing kernel-mode sequential calls of multiple hardware units, the user mode is not entered during task processing, which can avoid repeated switching between user mode and kernel mode, and can automatically complete the mutual calls between hardware units, avoiding the user (application) from being aware, and facilitating the user to call conveniently. It can keep the interface compatible with users when the function is upgraded. For example, the kernel-mode implementation of the first version of JPEG encoding is to call only the JPEG hardware encoding unit, and the kernel-mode implementation of the second version of JPEG encoding is to call the JPEG hardware encoding unit and the NPU hardware encoding unit, and the user-mode APIs of these two versions are completely consistent, that is, when the JPEG encoding function is upgraded, the workload of modifying the code of the application product will not be increased, and it is only necessary to make the kernel-mode driver support kernel-mode sequential calls.

[0087] For example, hardware unit 1 is a requesting hardware unit, and hardware unit 1 expects hardware unit 2 and hardware unit 3 to complete the same task together, that is, hardware unit 2 and hardware unit 3 are auxiliary hardware units. Figure 5 As shown, it is a schematic diagram of the collaborative work between multiple hardware units, taking hardware unit 1, hardware unit 2 and hardware unit 3 as examples. Hardware unit 1 corresponds to kernel state driver 1 in the kernel state of the CPU, and hardware unit 1 corresponds to user state API 1 in the user state of the CPU. Hardware unit 2 corresponds to kernel state driver 2 in the kernel state of the CPU, and hardware unit 2 corresponds to user state API 2 in the user state of the CPU. Hardware unit 3 corresponds to kernel state driver 3 in the kernel state of the CPU, and hardware unit 3 corresponds to user state API 3 in the user state of the CPU.

[0088] In order to realize kernel-mode sequential calls, the working order of multiple auxiliary hardware units (such as hardware unit 2 and hardware unit 3) can be pre-configured in the kernel-mode driver 1 of the requesting hardware unit (hardware unit 1). There is no restriction on this configuration process. For example, the working order can represent hardware unit 2-hardware unit 3.

[0089] In the above application scenario, with respect to the task processing method, the method may include the following steps: Step S31 : The application program calls the user state API 1 to send a task message 1 to the kernel state driver 1 .

[0090] Step S32 : After receiving the task message 1 (ie, the first task message), the kernel state driver 1 sends the task message 1 to the hardware unit 1 corresponding to the kernel state driver 1 .

[0091] Step S33: Hardware unit 1 processes the data to be processed in task message 1 to obtain processed data, and returns task message 2 to kernel state driver 1, where task message 2 includes the processed data.

[0092] Step S34 , the kernel-mode driver 1 receives the task message 2 (ie, the second task message) returned by the hardware unit 1 , and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 2 .

[0093] For example, when kernel-mode driver 1 receives task message 1 from user-mode API 1, since task message 1 originates from user-mode API 1, kernel-mode driver 1 determines that hardware unit 1 corresponding to kernel-mode driver 1 is the requesting hardware unit, and adds the first identity corresponding to the requesting hardware unit in task message 1. The first identity may be a unique identifier of kernel-mode driver 1 (such as a driver identifier). Hardware unit 1 may obtain the first identity from task message 1 and add the first identity to task message 2. Based on this, kernel-mode driver 1 may obtain the first identity from task message 2.

[0094] Step S35, the kernel state driver 1 determines whether the second identity identifier (such as the unique identifier of the kernel state driver 1) corresponding to the kernel state driver 1 is the same as the first identity identifier. If so, the kernel state driver 1 determines whether the auxiliary hardware unit has completed the task processing. If not, the kernel state driver 1 sends the task message 2 to the kernel state driver corresponding to the auxiliary hardware unit. If the auxiliary hardware unit has returned the task message, the task processing has been completed; if the auxiliary hardware unit has not returned the task message, the task processing has not been completed.

[0095] For example, kernel-state driver 1 pre-configures the working order of multiple auxiliary hardware units. For example, the working order can be represented by hardware unit 2-hardware unit 3. Since neither hardware unit 2 nor hardware unit 3 returns a task message, neither hardware unit 2 nor hardware unit 3 completes task processing. Kernel-state driver 1 sends task message 2 to kernel-state driver 2 corresponding to hardware unit 2 (i.e., the auxiliary hardware unit).

[0096] In a possible implementation, when the kernel-mode driver 1 preconfigures the working order of multiple auxiliary hardware units, the corresponding relationship between the task identifier and the working order can be configured. For example, the task identifier of task message 1 can be known in advance, and then the corresponding relationship between the task identifier and the working order can be configured. When the application sends task message 1 to the kernel-mode driver 1, task message 1 includes the task identifier, and task message 2 also includes the task identifier. In this way, the kernel-mode driver 1 can parse the task identifier from task message 2, and then query the working order corresponding to the task identifier, such as the working order represents hardware unit 2-hardware unit 3.

[0097] Step S36 : After receiving the task message 2 (ie, the first task message), the kernel state driver 2 sends the task message 2 to the hardware unit 2 corresponding to the kernel state driver 2 .

[0098] Step S37, the hardware unit 2 processes the data to be processed in the task message 2 to obtain processed data, and returns the task message 3 to the kernel state driver 2, where the task message 3 includes the processed data.

[0099] Step S38 , the kernel-mode driver 2 receives the task message 3 (ie, the second task message) returned by the hardware unit 2 , and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 3 .

[0100] For example, hardware unit 2 may obtain the first identity identifier from task message 2 , add the first identity identifier to task message 3 , and kernel-mode driver 2 may obtain the first identity identifier from task message 3 .

[0101] Step S39: Kernel state driver 2 determines whether the second identity identifier (such as the unique identifier of kernel state driver 2) corresponding to kernel state driver 2 is the same as the first identity identifier. If not, kernel state driver 2 sends task message 3 to kernel state driver 1 corresponding to the requesting hardware unit (i.e., hardware unit 1).

[0102] For example, the kernel-state driver 2 determines that the hardware unit 1 corresponding to the second identity identifier is the requesting hardware unit, and the kernel-state driver 2 sends the task message 3 to the kernel-state driver 1 corresponding to the second identity identifier.

[0103] Step S40 , kernel-mode driver 1 receives task message 3 (ie, second task message) returned by kernel-mode driver 2 , and obtains a first identity identifier corresponding to the requesting hardware unit from task message 3 .

[0104] Step S41, kernel state driver 1 determines whether the second identity identifier corresponding to kernel state driver 1 is the same as the first identity identifier. If so, kernel state driver 1 determines whether the auxiliary hardware unit has completed task processing. If the auxiliary hardware unit has not completed task processing, kernel state driver 1 sends task message 3 to the kernel state driver corresponding to the auxiliary hardware unit. For example, since hardware unit 2 has returned a task message, but hardware unit 3 has not returned a task message, hardware unit 3 has not completed task processing, and kernel state driver 1 can send task message 3 to kernel state driver 3 corresponding to hardware unit 3.

[0105] Step S42 : After receiving the task message 3 (ie, the first task message), the kernel state driver 3 sends the task message 3 to the hardware unit 3 corresponding to the kernel state driver 3 .

[0106] Step S43: The hardware unit 3 processes the data to be processed in the task message 3 to obtain processed data, and returns the task message 4 to the kernel state driver 3, wherein the task message 4 includes the processed data.

[0107] Step S44 , the kernel-mode driver 3 receives the task message 4 (ie, the second task message) returned by the hardware unit 3 , and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 4 .

[0108] Step S45, kernel state driver 3 determines whether the second identity identifier (such as the unique identifier of kernel state driver 3) corresponding to kernel state driver 3 is the same as the first identity identifier. If not, kernel state driver 3 sends task message 4 to kernel state driver 1 corresponding to the requesting hardware unit (i.e., hardware unit 1).

[0109] Step S46 , the kernel-mode driver 1 receives the task message 4 (ie, the second task message) returned by the kernel-mode driver 3 , and obtains the first identity identifier corresponding to the requesting hardware unit from the task message 4 .

[0110] Step S47, kernel state driver 1 determines whether the second identity identifier corresponding to kernel state driver 1 is the same as the first identity identifier. If so, kernel state driver 1 determines whether the auxiliary hardware unit has completed task processing. If so, kernel state driver 1 sends task message 4 to user state API 1. For example, hardware unit 2 and hardware unit 3 have returned task messages, and all auxiliary hardware units have completed task processing.

[0111] After sending the task message 4 to the user state API 1, the application can obtain the task message 4 from the user state API 1, and then obtain the processed data. At this point, the processing process of the same task is completed.

[0112] In one possible implementation, see Figure 6 The figure shows a schematic diagram of kernel-mode sequential calls. The application can call the ASIC hardware unit through the ASIC user-mode API of the ASIC hardware unit to complete a task. In order to improve the work effect, the ASIC hardware unit hopes to use the NPU hardware unit to assist the work, and there is no need to explicitly expose the process of the ASIC hardware unit calling the NPU hardware unit to the user of the ASIC user-mode API. In this way, kernel-mode sequential calls can be used.

[0113] In the above process, tasks involving kernel-mode binding call types (corresponding to Figure 4 ), user-mode sequential call type tasks (corresponding to Figure 2 ), kernel state sequential call type tasks (corresponding to Figure 5 ), in order to support at least two of the three tasks at the same time, this embodiment proposes a task processing method, in which the hardware unit simultaneously supports kernel-state binding calls, user-state successive calls, and kernel-state successive calls, forming a heterogeneous system with flexibility, real-time and high efficiency. Kernel-state binding calls refer to calling the kernel-state driver of each hardware unit in sequence based on the working order to call the hardware unit to work. User-state successive calls refer to calling the hardware unit to work one by one by calling the user-state API of the hardware unit. For example, a user calls the user-state API of the NPU once to complete a neural network inference. Kernel-state successive calls refer to calling the hardware unit to work by calling the kernel-state driver of the hardware unit.

[0114] Exemplarily, multiple hardware units are divided into first-class hardware units and second-class hardware units, and the number of second-class hardware units may be 0 or at least one. The first-class hardware unit corresponds to the kernel-state scheduler and kernel-state driver in the kernel state of the processor, and the first-class hardware unit corresponds to the user-state API in the user state of the processor. The second-class hardware unit corresponds to the kernel-state driver in the kernel state of the processor, and the second-class hardware unit corresponds to the user-state API in the user state of the processor. Obviously, compared with the second-class hardware unit, the first-class hardware unit needs to correspond to the kernel-state scheduler in the kernel state, that is, the kernel-state scheduler is additionally deployed, and the task scheduling is performed with the kernel-state scheduler as the core. In summary, on the basis of the kernel-state driver and the user-state API, a kernel-state scheduler can be developed, and the kernel-state scheduler simultaneously undertakes tasks of the kernel-state binding call type, tasks of the user-state successive call type, and tasks of the kernel-state successive call type.

[0115] Exemplarily, the first type of hardware unit may include but is not limited to an NPU unit and / or a GPU unit, and the second type of hardware unit may include but is not limited to at least one of an ASIC unit, an FPGA unit, and a CPLD unit. For example, if an NPU unit exists, the NPU unit may be used as the first type of hardware unit, and if a GPU unit exists, the GPU unit may be used as the first type of hardware unit. If an ASIC unit exists, the ASIC unit may be used as the second type of unit, if an FPGA unit exists, the FPGA unit may be used as the second type of unit, and if a CPLD unit exists, the CPLD unit may be used as the second type of unit.

[0116] See also Figure 7 As shown, it is a schematic diagram of the collaborative work between multiple hardware units, taking hardware unit 1, hardware unit 2, hardware unit 3 and hardware unit 4 as examples, if hardware unit 1 is ASIC unit 1, hardware unit 2 is ASIC unit 2, hardware unit 3 is NPU unit, and hardware unit 4 is ASIC unit 3.

[0117] exist Figure 7 In the example, hardware unit 3 is a first-class hardware unit, and hardware unit 1, hardware unit 2 and hardware unit 4 are second-class hardware units. Figure 7 This is just an example. For example, hardware unit 1 and hardware unit 3 are first-class hardware units, or hardware unit 1 and hardware unit 2 are first-class hardware units, or all hardware units are first-class hardware units. There is no limitation on this.

[0118] exist Figure 7In the figure, hardware unit 1 corresponds to kernel-state driver 1 in the kernel state of the CPU, and corresponds to user-state API 1 in the user state of the CPU. Hardware unit 2 corresponds to kernel-state driver 2 in the kernel state of the CPU, and corresponds to user-state API 2 in the user state of the CPU. Hardware unit 3 corresponds to kernel-state driver 3 and kernel-state scheduler 3 in the kernel state of the CPU, and hardware unit 3 corresponds to user-state API 3 in the user state of the CPU. Hardware unit 4 corresponds to kernel-state driver 4 in the kernel state of the CPU, and corresponds to user-state API 4 in the user state of the CPU.

[0119] exist Figure 7 In the embodiment, the kernel-mode binding call type task can be completed by hardware unit 1, hardware unit 2 and hardware unit 3, and the application needs to determine the working order of multiple hardware units, such as the working order can be expressed as hardware unit 1-hardware unit 2-hardware unit 3. In addition, the user-mode successive call type task can be completed by hardware unit 3. In addition, the kernel-mode successive call type task can be completed by hardware unit 3 and hardware unit 4, with hardware unit 3 acting as the requesting hardware unit and hardware unit 4 acting as the auxiliary hardware unit, or hardware unit 4 acting as the requesting hardware unit and hardware unit 3 acting as the auxiliary hardware unit. In the subsequent process, hardware unit 4 is taken as the requesting hardware unit and hardware unit 3 is taken as the auxiliary hardware unit as an example.

[0120] Case 1: The task processing method for the kernel-mode binding call type may include the following steps: Step S51, the application starts a task of kernel-mode binding call type, and sends task message 1 to kernel-mode driver 1. After receiving task message 1, kernel-mode driver 1 sends task message 1 to hardware unit 1 corresponding to kernel-mode driver 1. Hardware unit 1 processes the data to be processed in task message 1, obtains processed data, and returns task message 2 to kernel-mode driver 1, and task message 2 may include processed data. Task message 1 and task message 2 may also include a work order.

[0121] Step S52: kernel-state driver 1 receives task message 2 returned by hardware unit 1, obtains the working order of multiple hardware units from task message 2, and determines whether hardware unit 1 is the last hardware unit based on the working order. Since hardware unit 1 is not the last hardware unit, kernel-state driver 1 determines the next hardware unit (i.e., hardware unit 2) of hardware unit 1 based on the working order, and kernel-state driver 1 sends task message 2 to kernel-state driver 2 corresponding to hardware unit 2.

[0122] Step S53, the kernel state driver 2 sends the task message 2 to the hardware unit 2 corresponding to the kernel state driver 2. The hardware unit 2 processes the data to be processed in the task message 2 to obtain processed data, and returns the task message 3 to the kernel state driver 2. The task message 3 may include the processed data and the working order.

[0123] Step S54: kernel-mode driver 2 obtains the working order of multiple hardware units from task message 3, and determines whether hardware unit 2 is the last hardware unit based on the working order. Since hardware unit 2 is not the last hardware unit, kernel-mode driver 2 determines the next hardware unit (hardware unit 3) of hardware unit 2 based on the working order, and sends task message 3 to kernel-mode scheduler 3 corresponding to hardware unit 3.

[0124] Exemplarily, since the hardware unit 3 corresponds to the kernel state scheduler 3 in the kernel state, the kernel state driver 2 sends the task message 3 to the kernel state scheduler 3 instead of to the kernel state driver 3 .

[0125] Step S55: After receiving the task message 3 (i.e., the first task message), the kernel state scheduler 3 sends the task message 3 to the kernel state driver 3, and the kernel state driver 3 sends the task message 3 to the hardware unit 3 corresponding to the kernel state driver 3. The hardware unit 3 processes the data to be processed in the task message 3 to obtain processed data, and returns the task message 4 to the kernel state driver 3. The kernel state driver 3 returns the task message 4 to the kernel state scheduler 3, and the task message 4 may include the processed data and the working order.

[0126] Step S56 , the kernel-mode scheduler 3 receives the task message 4 (second task message) returned by the kernel-mode driver 3 . In addition to the processed data and the working sequence, the task message 4 may include the task type.

[0127] Exemplarily, when the application sends task message 1 to kernel-mode driver 1, task message 1 may include a task type, so that task message 2, task message 3, and task message 4 all include a task type. Other entities (such as kernel-mode drivers, etc.) except kernel-mode scheduler 3 do not pay attention to the task type, while kernel-mode scheduler 3 pays attention to the task type. For example, after receiving task message 4, kernel-mode scheduler 3 may parse the task type from task message 4. Since situation 1 is a task processing method for kernel-mode binding call type, the task type is kernel-mode binding call.

[0128] Step S57, if the task type is a kernel-mode binding call, the kernel-mode scheduler 3 obtains the working order of multiple hardware units from the task message 4, and determines whether the hardware unit 3 (first-class hardware unit) is the last hardware unit based on the working order. If so, the kernel-mode scheduler 3 stores the processed data in the specified storage medium. If not, the kernel-mode scheduler 3 sends the task message 4 to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit. If the next hardware unit is a first-class hardware unit, the task message 4 is sent to the kernel-mode scheduler corresponding to the next hardware unit. If the next hardware unit is a second-class hardware unit, the task message 4 is sent to the kernel-mode driver corresponding to the next hardware unit.

[0129] In this example, since the working order represents hardware unit 1-hardware unit 2-hardware unit 3, that is, hardware unit 3 is the last hardware unit, kernel state scheduler 3 can end the processing of the task message and store the processed data in task message 4 to the specified storage medium. In addition, kernel state scheduler 3 can send a task end message to the application, indicating that the task has been processed. When the application needs to obtain the processed data, it can directly read the processed data from the specified storage medium.

[0130] Case 2: For the task processing method of user-mode sequential call type, the following steps may be included: Step S61 , the application calls the user state API 1 to send a task message 1 to the kernel state scheduler 3 .

[0131] Step S62: After receiving the task message 1 (i.e., the first task message), the kernel state scheduler 3 sends the task message 1 to the kernel state driver 3, and the kernel state driver 3 sends the task message 1 to the hardware unit 3 corresponding to the kernel state driver 3. The hardware unit 3 processes the data to be processed in the task message 1 to obtain processed data, and the hardware unit 3 returns the task message 2 to the kernel state driver 3, and the kernel state driver 3 returns the task message 2 to the kernel state scheduler 3, and the task message 2 may include the processed data.

[0132] Step S63 : The kernel-mode scheduler 3 receives the task message 2 (ie, the second task message) returned by the kernel-mode driver 3 . In addition to the processed data, the task message 2 may include the task type.

[0133] Exemplarily, when the application sends task message 1, task message 1 may include a task type, and thus task message 2 includes the task type. After receiving task message 2, kernel state scheduler 3 may parse the task type from task message 2, and the task type is user state sequential call.

[0134] Step S64: If the task type is user-mode sequential call, the kernel-mode scheduler 3 sends the task message 2 to the user-mode API 3, so that the application obtains the task message 2 from the user-mode API 3 and obtains the processed data from the task message 2. At this point, the processing process of user-mode sequential call tasks is completed.

[0135] Case 3, for the task processing method of kernel state sequential call type, the following steps may be included: Step S71, the application calls the user state API 4 to send a task message 1 to the kernel state driver 4. After receiving the task message 1, the kernel state driver 4 sends the task message 1 to the hardware unit 4 corresponding to the kernel state driver 4. The hardware unit 4 processes the data to be processed in the task message 1 to obtain processed data, and returns a task message 2 to the kernel state driver 4, and the task message 2 includes the processed data.

[0136] Step S72, kernel-mode driver 4 obtains the first identity (such as the unique identity of kernel-mode driver 4) corresponding to the requesting hardware unit (i.e., hardware unit 4) from task message 2. Kernel-mode driver 4 determines whether the second identity corresponding to kernel-mode driver 4 is the same as the first identity. If so, kernel-mode driver 4 determines whether the auxiliary hardware unit (i.e., hardware unit 3) has completed task processing. If not, kernel-mode driver 4 sends task message 2 to kernel-mode scheduler 3 corresponding to the auxiliary hardware unit.

[0137] Step S73: after receiving the task message 2 (i.e., the first task message), the kernel state scheduler 3 sends the task message 2 to the kernel state driver 3, and the kernel state driver 3 sends the task message 2 to the hardware unit 3 corresponding to the kernel state driver 3. The hardware unit 3 processes the data to be processed in the task message 2 to obtain processed data, and the hardware unit 3 returns the task message 3 to the kernel state driver 3, and the kernel state driver 3 returns the task message 3 to the kernel state scheduler 3, and the task message 3 may include the processed data.

[0138] Step S74 , the kernel state scheduler 3 receives the task message 3 (second task message) returned by the kernel state driver 3 . In addition to the processed data, the task message 3 may also include a task type.

[0139] Exemplarily, when the application sends task message 1, task message 1 may include a task type, so that both task message 2 and task message 3 include a task type. Kernel state scheduler 3 may parse the task type from task message 3, and the task type is kernel state sequential call.

[0140] Step S75, if the task type is kernel-state sequential calls, the kernel-state scheduler 3 obtains the first identity identifier corresponding to the requesting hardware unit (hardware unit 4) from the task message 3, and determines whether the second identity identifier corresponding to the kernel-state scheduler 3 (such as the unique identifier of the kernel-state driver 3) is the same as the first identity identifier.

[0141] If not, the kernel state scheduler 3 sends the task message 3 to the kernel state scheduler or kernel state driver corresponding to the requesting hardware unit. For example, if the requesting hardware unit is a first-class hardware unit, the task message 3 is sent to the kernel state scheduler corresponding to the requesting hardware unit; if the requesting hardware unit is a second-class hardware unit, the task message 3 is sent to the kernel state driver corresponding to the requesting hardware unit.

[0142] In this example, since the second identity identifier is different from the first identity identifier, the kernel state scheduler 3 can send the task message 3 to the kernel state driver 4 corresponding to the hardware unit 4 .

[0143] If yes, the kernel state scheduler 3 determines that the hardware unit 3 is the requesting hardware unit, and can determine whether the auxiliary hardware unit has completed the task processing. If yes, the kernel state scheduler 3 sends the task message 3 to the user state API 3, so that the application obtains the task message 3 from the user state API 3; if no, the kernel state scheduler 3 sends the task message 3 to the kernel state scheduler or kernel state driver corresponding to the auxiliary hardware unit.

[0144] Step S76, the kernel state driver 4 receives the task message 3 returned by the kernel state scheduler 3, and obtains the first identity corresponding to the requesting hardware unit from the task message 3. The kernel state driver 4 determines whether the second identity corresponding to the kernel state driver 4 is the same as the first identity. If so, the kernel state driver 4 determines whether the auxiliary hardware unit has completed the task processing. If so, the kernel state driver 4 sends the task message 3 to the user state API 4, and the application can obtain the task message 3 from the user state API 4.

[0145] In a possible implementation, when the kernel-state scheduler 3 simultaneously supports kernel-state binding calls, user-state successive calls, and kernel-state successive calls, the kernel-state scheduler 3 can receive multiple first task messages, for example, a first task message for kernel-state binding calls, a first task message for user-state successive calls, and a first task message for kernel-state successive calls.

[0146] Based on this, when the kernel-state scheduler 3 receives multiple first task messages, for each first task message, it can obtain the task priority of the first task message from the first task message. Based on the task priority of each first task message, the kernel-state scheduler 3 selects the first task message with the highest task priority, and sends the first task message with the highest task priority to the kernel-state driver 3. When receiving the second task message returned by the kernel-state driver 3 (i.e., the response to the first task message), the kernel-state scheduler 3 selects the first task message with the second highest task priority, and sends the first task message with the second highest task priority to the kernel-state driver 3, and so on, until all the first task messages are sent to the kernel-state driver 3.

[0147] In order to implement the above process and make the kernel state scheduler 3 support task priority, when tasks in different directions send the first task message, it is necessary to carry the task priority in the first task message. For example, when sending the first task message for the task called by kernel state binding, the first task message includes task priority 3 (indicating the highest priority). When sending the first task message for the task called successively by kernel state, the first task message includes task priority 2 (indicating the middle priority). When sending the first task message for the task called successively by user state, the first task message includes task priority 1 (indicating the lowest priority).

[0148] It can be seen from the above technical solutions that in the embodiment of the present application, by deploying a kernel state scheduler and a kernel state driver in the kernel state, during the task processing process, the task processing can be completed in the kernel state without repeatedly switching between the user state and the kernel state, saving resource overhead and switching overhead, and meeting the high real-time requirements of the task. By deploying the kernel state scheduler, the hardware unit supports user state successive calls, kernel state binding calls, and kernel state successive calls at the same time, so that the hardware unit takes into account flexibility and real-time performance, so that the hardware unit can respond to real-time tasks efficiently, and can also respond to occasional tasks with flexibility.

[0149] Based on the same application concept as the above method, an electronic device is proposed in an embodiment of the present application, the electronic device includes a processor and multiple hardware units. For example, the multiple hardware units may include a first type of hardware unit, and the electronic device also includes a kernel state scheduler and a kernel state driver corresponding to the first type of hardware unit in the kernel state of the processor; wherein: The kernel state scheduler is used to send the first task message to the kernel state driver when receiving the first task message, wherein the first task message includes data to be processed; The kernel state driver is used to send the first task message to the first type of hardware unit; The first type of hardware unit is used to process the data to be processed to obtain processed data, and send a second task message to the kernel state driver, wherein the second task message includes the processed data and the task type; The kernel state driver is further used to send the second task message to the kernel state scheduler; The kernel-mode scheduler is further configured to receive the second task message; if the task type is a kernel-mode binding call, obtain the working order of multiple hardware units from the second task message, and determine whether the first-type hardware unit is the last hardware unit based on the working order; if so, store the processed data in a designated storage medium; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit; The kernel-mode binding call indicates calling the kernel-mode driver of each hardware unit in sequence based on the working order to call the hardware unit to work.

[0150] Exemplarily, the first type of hardware unit corresponds to the user state API in the user state of the processor, and the kernel state scheduler is also used to send the second task message to the user state API if the task type is user state successive call, so that the application obtains the second task message from the user state API and obtains processed data from the second task message; wherein, the user state successive call means calling the hardware unit to perform work by calling the user state API of the hardware unit.

[0151] Exemplarily, the kernel-mode scheduler is further used to, if the task type is kernel-mode sequential calls, obtain the first identity identifier corresponding to the requesting hardware unit from the second task message; if the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the requesting hardware unit; if the second identity identifier is the same as the first identity identifier, determine whether the auxiliary hardware unit has completed task processing; if so, send the second task message to the user-mode API; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the auxiliary hardware unit; Among them, the kernel state successive calls represent calling the hardware unit to work by calling the kernel state driver of the hardware unit; among them, when a hardware unit expects another hardware unit to complete the same task together, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit.

[0152] Based on the same application concept as the above method, an electronic device is proposed in the embodiment of the present application, see Figure 8As shown, the electronic device includes: a processor 81 and a machine-readable storage medium 82, wherein the machine-readable storage medium 82 stores machine-executable instructions that can be executed by the processor 81; the processor 81 is used to execute the machine-executable instructions to implement the task processing method disclosed in the above example of this application.

[0153] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the task processing method disclosed in the above example of the present application can be implemented.

[0154] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0155] Based on the same application concept as the above method, an embodiment of the present application also provides a computer program product, which may include a computer program, wherein when the computer program is executed by a processor, it can implement the task processing method disclosed in the above example of the present application.

[0156] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0157] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A task processing method, characterized in that: Applied to an electronic device including a processor and a plurality of hardware units, the plurality of hardware units including a first type of hardware unit, the first type of hardware unit corresponding to a kernel state scheduler and a kernel state driver in a kernel state of the processor, the method comprising: When receiving the first task message, the kernel state scheduler sends the first task message to the kernel state driver, and the kernel state driver sends the first task message to the first type of hardware unit; The kernel-mode scheduler receives a second task message returned by the kernel-mode driver, wherein the second task message includes processed data and a task type; wherein the processed data is obtained after the first type of hardware unit processes the data to be processed carried by the first task message; If the task type is a kernel-mode binding call, the kernel-mode scheduler obtains the working order of multiple hardware units from the second task message, and determines whether the first-type hardware unit is the last hardware unit based on the working order; wherein the kernel-mode binding call means calling the kernel-mode driver of each hardware unit in turn based on the working order to call the hardware unit to work; If so, the kernel-mode scheduler stores the processed data to a designated storage medium; If not, the kernel-mode scheduler sends the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit.

2. The method according to claim 1, characterized in that The first type of hardware unit corresponds to the user state API in the user state of the processor, and after the kernel state scheduler receives the second task message returned by the kernel state driver, the method further includes: If the task type is user-mode successive calls, the kernel-mode scheduler sends the second task message to the user-mode API, so that the application obtains the second task message from the user-mode API and obtains processed data from the second task message; wherein, the user-mode successive calls represent calling the hardware unit to perform work by calling the user-mode API of the hardware unit.

3. The method according to claim 1, characterized in that After the kernel-mode scheduler receives the second task message returned by the kernel-mode driver, the method further includes: If the task type is kernel-mode sequential calls, the kernel-mode scheduler obtains the first identity corresponding to the requesting hardware unit from the second task message; wherein the kernel-mode sequential calls represent calling the hardware unit to perform work by calling the kernel-mode driver of the hardware unit; If the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, the kernel state scheduler determines that the first type of hardware unit is an auxiliary hardware unit, and sends the second task message to the kernel state scheduler or kernel state driver corresponding to the requesting hardware unit; When one hardware unit expects another hardware unit to complete the same task together, the one hardware unit is a requesting hardware unit, and the other hardware unit is an assisting hardware unit.

4. The method according to claim 3, characterized in that After the kernel state scheduler obtains the first identity corresponding to the requesting hardware unit from the second task message, the method further includes: If the second identity identifier is the same as the first identity identifier, the kernel-mode scheduler determines that the first type of hardware unit is a requesting hardware unit, and determines whether the auxiliary hardware unit has completed task processing; wherein, if the auxiliary hardware unit has returned a task message, the auxiliary hardware unit has completed task processing; if the auxiliary hardware unit has not returned a task message, the auxiliary hardware unit has not completed task processing; If so, the kernel-state scheduler sends the second task message to the user-state API so that the application obtains the second task message from the user-state API; if not, the kernel-state scheduler sends the second task message to the kernel-state scheduler or kernel-state driver corresponding to the auxiliary hardware unit.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When receiving a plurality of first task messages, the kernel state scheduler acquires the task priority of each first task message from the first task message; Based on the task priority of each first task message, the kernel state scheduler selects the first task message with the highest task priority, and sends the first task message with the highest task priority to the kernel state driver.

6. The method according to any one of claims 1 to 4, characterized in that: The plurality of hardware units further include a second type of hardware unit, wherein the second type of hardware unit corresponds to a kernel state driver in the kernel state of the processor, and the second type of hardware unit corresponds to a user state API in the user state of the processor; Wherein, when sending the second task message to the kernel-state scheduler or kernel-state driver corresponding to the next hardware unit of the first-category hardware unit, if the next hardware unit is a first-category hardware unit, the kernel-state scheduler sends the second task message to the kernel-state scheduler corresponding to the next hardware unit; if the next hardware unit is a second-category hardware unit, the kernel-state scheduler sends the second task message to the kernel-state driver corresponding to the next hardware unit; The first type of hardware units include NPU units and / or GPU units, and the second type of hardware units include at least one of ASIC units, FPGA units and CPLD units.

7. A task processing method, characterized in that: Applied to an electronic device, the electronic device includes a processor and a plurality of hardware units, for each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor, and for each kernel state driver, the method includes: When receiving the first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, wherein the second task message includes processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message; The kernel-mode driver obtains a working order of a plurality of hardware units from the second task message, and determines whether the hardware unit is the last hardware unit based on the working order; If so, the kernel-mode driver stores the processed data to a designated storage medium; If not, the kernel-mode driver determines a subsequent hardware unit of the hardware unit based on the working order, and sends the second task message to the kernel-mode driver corresponding to the subsequent hardware unit.

8. A task processing method, characterized in that: Applied to an electronic device including a processor and a plurality of hardware units, for each hardware unit, the hardware unit corresponds to a kernel state driver in the kernel state of the processor and corresponds to a user state API in the user state of the processor, and for each kernel state driver, the method includes: When receiving the first task message, the kernel-state driver sends the first task message to the hardware unit corresponding to the kernel-state driver, and receives a second task message returned by the hardware unit, wherein the second task message includes processed data; wherein the processed data is obtained after the hardware unit processes the data to be processed carried by the first task message; The kernel-mode driver obtains the first identity corresponding to the requesting hardware unit from the second task message; wherein, when a hardware unit expects another hardware unit to jointly complete the same task, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit; If the second identity identifier corresponding to the kernel-state driver is different from the first identity identifier, the kernel-state driver sends the second task message to the kernel-state driver corresponding to the requesting hardware unit; If the second identity identifier is the same as the first identity identifier, the kernel-state driver determines whether the auxiliary hardware unit has completed task processing; if so, the second task message is sent to the user-state API; if not, the second task message is sent to the kernel-state driver corresponding to the auxiliary hardware unit.

9. An electronic device, characterized in that: The electronic device includes a processor and a plurality of hardware units, the plurality of hardware units include a first type of hardware unit, and the electronic device further includes a kernel state scheduler and a kernel state driver corresponding to the first type of hardware unit in the kernel state of the processor; wherein: The kernel state scheduler is used to send the first task message to the kernel state driver when receiving the first task message, wherein the first task message includes data to be processed; The kernel state driver is used to send the first task message to the first type of hardware unit; The first type of hardware unit is used to process the data to be processed to obtain processed data, and send a second task message to the kernel state driver, wherein the second task message includes the processed data and the task type; The kernel state driver is further used to send the second task message to the kernel state scheduler; The kernel-mode scheduler is further configured to receive the second task message; if the task type is a kernel-mode binding call, obtain the working order of multiple hardware units from the second task message, and determine whether the first-type hardware unit is the last hardware unit based on the working order; if so, store the processed data in a designated storage medium; if not, send the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the next hardware unit of the first-type hardware unit; The kernel-mode binding call indicates calling the kernel-mode driver of each hardware unit in sequence based on the working order to call the hardware unit to work.

10. The electronic device according to claim 9, characterized in that: in, The first type of hardware unit corresponds to a user state API in the user state of the processor, and the kernel state scheduler is further used to send the second task message to the user state API if the task type is a user state successive call, so that the application obtains the second task message from the user state API and obtains processed data from the second task message; wherein the user state successive call means calling the hardware unit to perform work by calling the user state API of the hardware unit; Wherein, the kernel-mode scheduler is further used for, if the task type is kernel-mode sequential calls, obtaining the first identity identifier corresponding to the requesting hardware unit from the second task message; if the second identity identifier corresponding to the first type of hardware unit is different from the first identity identifier, sending the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the requesting hardware unit; if the second identity identifier is the same as the first identity identifier, determining whether the auxiliary hardware unit has completed the task processing; if so, sending the second task message to the user-mode API; if not, sending the second task message to the kernel-mode scheduler or kernel-mode driver corresponding to the auxiliary hardware unit; Among them, the kernel state successive calls represent calling the hardware unit to work by calling the kernel state driver of the hardware unit; among them, when a hardware unit expects another hardware unit to complete the same task together, the one hardware unit is the requesting hardware unit, and the other hardware unit is the auxiliary hardware unit.

Images