Kernel driving method and device based on multi-core collaboration and electronic equipment

Abstract

The invention provides a kernel driving method and device based on multi-kernel collaboration and electronic equipment. The method comprises the following steps: receiving a calling task of a user mode by a service driver; the service driver sends the calling task to a task queue of an IP driver of a corresponding type according to the IP type corresponding to the calling task; selecting a calling task from the task queue as a target task by the IP driver; if the IP corresponding to the target task is of a multi-core structure, the IP driver sends the target task to a corresponding multi-core cooperation unit; selecting, by the multi-core cooperation unit, a target core from the plurality of cores of the IP according to a priority policy; the multi-core cooperation unit schedules the target task to the target core; and executing the target task by the target core. In this way, the task processing capacity of the system is greatly improved, and the overall efficiency is improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of IP hardware drivers, and in particular, relates to a kernel driver method and device for IP multi-core collaboration, and an electronic device. Background Art

[0002] Take video encoding and decoding as an example. In the current digital era, the explosive growth of video data has put forward higher requirements on the performance of video encoding and decoding technology. Traditional single-core processors are unable to handle ultra-high-definition and 8K videos, and the emergence of IP multi-core processors has made it possible to solve this problem. However, simply allocating video encoding and decoding tasks to multiple cores does not guarantee optimal performance. In addition, there are also problems such as load imbalance and data dependency between multiple cores. Summary of the invention

[0003] The purpose of the disclosed embodiments is to provide a kernel driver method and device based on multi-core collaboration, and an electronic device. The calling task is dynamically allocated to multiple isomorphic cores of the IP through a multi-core collaboration strategy to ensure efficient use of hardware resources and load balancing of each core. In addition, the correctness of access to data by calling tasks with dependencies is ensured by sharing a memory mapping table.

[0004] In the first aspect, the embodiment of the present disclosure provides a kernel driver method based on multi-core collaboration. The method includes: receiving a user-mode call task by a service driver; sending the call task to a task queue of an IP driver of a corresponding type according to the IP type corresponding to the call task by the service driver; selecting a call task from the task queue as a target task by the IP driver; if the IP corresponding to the target task is a multi-core structure, sending the target task to a corresponding multi-core collaboration unit by the IP driver; selecting a target core from multiple cores of the IP by the multi-core collaboration unit according to a priority strategy; scheduling the target task to the target core by the multi-core collaboration unit; and executing the target task by the target core.

[0005] In an implementation of the first aspect, the method further includes: if the IP is a single-core structure, the IP driver sends the target task to the single-core core of the IP; and the single-core core executes the target task.

[0006] In an implementation of the first aspect, the method further includes: during the firmware loading phase when the device is powered on, the service driver and the IP driver load the corresponding service driver program and IP driver program in the firmware into the operating system according to the node configuration information in the dtsi, wherein the service driver receives the user-state call task, including: the service driver receives the call task through a character device file, wherein the character device file is created when the service driver program is loaded and serves as the total entry for all IP drivers.

[0007] In an implementation of the first aspect, the node configuration information in the dtsi includes: configuration information of the service node; node configuration information of each IP core; if the IP is a multi-core structure, there is also configuration information of a multi-core collaboration node.

[0008] In an implementation of the first aspect, the method also includes: if the IP is a multi-core structure, the first core is identified as the main core and the other cores are secondary cores, the node configuration information of the main core references the service node and the multi-core collaboration node, and the node configuration information of the secondary core only references the multi-core collaboration node; and if the IP is a single-core structure, the node configuration information of the IP core only references the service node.

[0009] In an implementation of the first aspect, loading the corresponding IP driver in the firmware into the operating system includes: parsing the node configuration information of the IP core one by one according to the node order in the dtsi; matching the IP driver according to the node name of the IP core; if the node of the IP core references the service node, instantiating a task queue as the task queue of the IP driver, and registering the IP driver under the service driver; if the node of the IP core references the multi-core collaboration node, pointing the IP core to the corresponding multi-core collaboration unit; and if the node of the IP core is configured with IOMMU parameters, instantiating a memory mapping table for the IOMMU corresponding to the IP core.

[0010] In an implementation of the first aspect, the task queue includes: a first-in-first-out serialized single queue; and a priority-based multi-queue.

[0011] In an implementation manner of the first aspect, the multiple cores of the IP refer to that the IP contains multiple cores with the same functions and structures, and the cores include cores with the same performance, or large and small cores with different performances.

[0012] In an implementation of the first aspect, the priority strategy includes: the multi-core collaboration unit prioritizes the idle cores among the multiple cores using a directed polling method; and the multi-core collaboration unit prioritizes the idle cores among the multiple cores using a cumulative task load method.

[0013] In an implementation of the first aspect, selecting a target core from multiple cores of the IP includes: selecting a core or multiple cores, including: if the target task is a simple task, selecting an idle core as the target core; if the target task is a complex task, selecting multiple idle cores as the target core, and splitting the target task into a corresponding number of target subtasks.

[0014] In an implementation of the first aspect, the multi-core collaboration unit uses a directional polling method to prioritize the idle cores among the multiple cores, including: starting from the starting core, querying each of the cores in turn in a circular manner along a preset direction to see if they are idle, and the order of the idle cores is the priority order, wherein the starting core of the first polling is the preset core, and the starting cores of the remaining pollings are the (core_id+1)%Nth core among the multiple cores, core_id is the number of the last core of the target core in the previous polling, and N is the number of the multiple cores.

[0015] In an implementation of the first aspect, the multi-core collaboration unit prioritizes the idle cores among the multiple cores by using a cumulative task load method, including: starting from the main core, querying whether each of the cores is idle in turn along a preset direction, and sorting the idle cores from small to large according to the cumulative task load, and the sorted order is the priority order, wherein the cumulative task load of all cores is initialized to 0 for the first time, and after the target core completes the task, the cumulative task load of the corresponding core needs to be added to the task load of the currently executing task.

[0016] In an implementation of the first aspect, the task load includes: a load obtained according to the amount of calculation required for executing the task; and a load obtained by querying a corresponding load table according to the attributes of the executing task.

[0017] In an implementation of the first aspect, the memory mapping table is used to map a virtual address visible to the driver to a real physical address of the DDR, so that the driver accesses the storage data on the DDR through the IOMMU and the memory mapping table.

[0018] In an implementation manner of the first aspect, the method further includes: if there is a front-end dependency relationship among the calling tasks in the user state, causing the target core that executes all the calling tasks in the user state to use the same memory mapping table.

[0019] In an implementation manner of the first aspect, the method further includes: if the complex task is split into multiple subtasks, causing the target core that executes the subtasks to use a same memory mapping table.

[0020] In an implementation of the first aspect, the method also includes: the application program splits a complex task into multiple user-mode calling tasks, so as to send these calling tasks to the cores of the corresponding IP types through character device files for execution; and after all the calling tasks are executed, the application program merges the results of the multiple calling tasks.

[0021] In an implementation of the first aspect, the method further includes: after the execution of multiple target subtasks on the target core is completed, the multi-core collaboration unit merges the results of the execution of the target core.

[0022] In an implementation of the first aspect, the information contained in the calling task includes: the type of IP used, register configuration information, previous and next dependency identifiers, a multi-core collaboration identifier, and task complexity.

[0023] In a second aspect, an embodiment of the present disclosure provides a kernel driver device based on multi-core collaboration, the device comprising: a task queue, configured to store scheduled tasks; a character device file unit, configured to receive scheduled tasks in user mode and send the scheduled tasks to the task queue; multiple homogeneous cores, including a main core and a secondary core, the main core being the first core among the multiple homogeneous cores, and the secondary core being the other cores among the multiple homogeneous cores except the first core; and a multi-core collaboration unit, configured to select a scheduled task from the task queue as a target task, and select a target core from the multiple homogeneous cores to schedule the target task to the target core, so that the target core executes the target task.

[0024] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory; and one or more processors coupled to the memory and configured to execute the method provided in the first aspect of the embodiment of the present disclosure.

[0025] The disclosed embodiment provides a kernel driver method based on multi-core collaboration. The service driver sends the task to the task queue of the corresponding IP driver according to the IP type of the calling task. If the IP is a multi-core structure, the IP driver selects a calling task from the task queue as the target task and sends the target task to the multi-core collaboration unit. The multi-core collaboration unit selects an idle core as the target core according to the result of the priority strategy, and the target core executes the target task. In this way, multiple isomorphic cores can execute their respective target tasks in parallel, realizing multi-core collaborative processing of the target task. If the IP is a single-core structure, the IP driver directly sends the target task to the core of the IP for execution.

[0026] In the embodiments of the present disclosure, developers can adapt IP chips with different numbers of cores by simply configuring the node information of each IP core of dtsi and the corresponding multi-core collaboration node information according to the single-core / multi-core situation of the IP contained in the chip.

[0027] In the embodiment of the present disclosure, if the IP is a multi-core structure, the application layer can split a complex task into multiple calling tasks, and merge the results after all the calling tasks are completed. This method reduces the granularity of the calling tasks, increases the parallelism of the IP multi-core operation, and improves the utilization rate of the IP core.

[0028] In the disclosed embodiment, if the IP is a multi-core structure, the multi-core collaboration unit will sort the idle cores according to the priority strategy, determine the target core according to the number of idle cores and the complexity of the target task, split the target task into multiple subtasks, and merge the execution results after all subtasks are completed. Compared with the user-mode scheduling task splitting, this method can adaptively split the task granularity according to the actual usage of the IP core, further improving the utilization rate of the IP multi-core.

[0029] In the disclosed embodiment, the multi-core collaboration unit shields the hardware differences between multi-core and single-core. For the application layer, only one set of the same calling interface is required, thus reducing the complexity of development.

[0030] In the disclosed embodiment, if the IP core uses IOMMU, the problem of data synchronization caused by the front-to-back dependency of multiple calling tasks and the splitting of calling tasks is solved by multiple cores sharing a memory mapping table. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Shown is a schematic diagram of IP multi-core homogeneous mode.

[0032] Figure 2 Shown is an IP driver loading flow chart in an embodiment of the present disclosure.

[0033] Figure 3Shown is a schematic diagram of the kernel driver structure in an embodiment of the present disclosure.

[0034] Figure 4 Shown is a flowchart of the kernel driving method in an embodiment of the present disclosure.

[0035] Figure 5A Shown is a schematic diagram of an IP multi-core executing an application layer split scheduling task in an embodiment of the present disclosure.

[0036] Figure 5B Shown is a schematic diagram of an IP multi-core executing a multi-core collaboration unit splitting target task in an embodiment of the present disclosure.

[0037] Figure 5C Shown is a flow chart of a multi-core collaboration unit executing a target task in an embodiment of the present disclosure.

[0038] Fig. 6A Shown is a schematic diagram of a dedicated memory mapping table used by each IP core in an embodiment of the present disclosure.

[0039] Figure 6B It is a schematic diagram showing that all IP cores in an embodiment of the present disclosure share a memory mapping table.

[0040] Figure 6C It is a schematic diagram showing that some IP cores in an embodiment of the present disclosure share a memory mapping table.

[0041] Figure 7 Shown is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0043] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present disclosure. Therefore, the drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0044] Taking video encoding and decoding as an example, multiple homogeneous cores can be stacked in a video encoding and decoding device to improve the encoding and decoding capabilities. For example, four homogeneous encoding and decoding cores with 4K capabilities can be used to form a 4-core homogeneous multi-core codec, and the multiple cores work together to achieve 8K encoding and decoding capabilities.

[0045] Figure 1 is an example diagram showing a stack of multiple homogeneous cores. Figure 1 As shown, VEPU (Video Encode Processing Unit) and JPEGE (JPEG Encoder) adopt homogeneous multi-core mode, while VDPU (Video Decode Processing Unit) and AV1D (AV1 Decoder) adopt single-core mode. Specifically, VEPU includes 4 cores Core-00 to Core-03, which are used to improve the processing capability of VEPU; JPEGE includes 4 cores Core-10 to Core-13, which are used to improve the processing capability of JPEGE; VDPU includes a single core Core-2, and AV1D includes a single core Core-3.

[0046] In order to achieve collaboration among multiple homogeneous cores, tasks must be reasonably scheduled among multiple homogeneous cores. At least for this problem, the present disclosure provides a kernel driving method based on multi-core collaboration. The method provided by the present disclosure will be described in detail in the following by means of specific embodiments combined with the accompanying drawings.

[0047] In some implementations, the chip contains multiple IP types, some of which are single-core and some are multi-core. The composition of these IPs will be described in the form of core nodes in dtsi. If the IP is a multi-core structure, the first core is conventionally called the main core and the other cores are called secondary cores. For IP single-core nodes and main core nodes in IP multi-cores, the core nodes need to reference service nodes to register with the service when the IP driver is loaded. In addition, for IP multi-cores, all core nodes under its IP need to reference the corresponding multi-core collaboration nodes, and the IP driver will point all core nodes to the corresponding multi-core collaboration units when it is loaded. Among them, multi-core collaboration nodes are used for collaboration and task scheduling between multi-cores.

[0048] Exemplarily, the IP types included in the chip are: VEPU, JPEGE, VDPU and AV1D, wherein VEPU and JPEGE each have 4 homogeneous cores, and VDPU and AV1D are single cores.

[0049] The following is an example of a dtsi description:

[0050] mpp_service: mpp_service / / service node

[0051] vepu-ccu: vepu-ccu / / VEPU multi-core collaboration node

[0052] vepu0: vepu-core0@ff350000 / / VEPU main core

[0053] rc,srv=mpp_service; / / reference service node

[0054] rc,ccu=vepu-ccu; / / reference VEPU multi-core collaboration node

[0055] vepu1: vepu-core1@ff351000

[0056] rc,ccu=vepu-ccu; / / reference VEPU multi-core collaboration node

[0057] vepu2:vepu-core2@ff352000

[0058] rc,ccu=vepu-ccu; / / reference VEPU multi-core collaboration node

[0059] vepu3:vepu-core3@ff353000

[0060] rc,ccu=vepu-ccu; / / reference VEPU multi-core collaboration node

[0061] jpege-ccu: jpege-ccu / / JPEGE multi-core collaboration node

[0062] jpege0:jpege-core0@ff354000 / / JPEGE main core

[0063] rc,srv=mpp_service; / / reference service node

[0064] rc,ccu=vepu-ccu; / / reference JPEGE multi-core collaboration node

[0065] jpege1: jpege-core1@ff355000

[0066] rc,ccu=vepu-ccu; / / reference JPEGE multi-core collaboration node

[0067] jpege2: jpege-core2@ff356000

[0068] rc,ccu=vepu-ccu; / / reference JPEGE multi-core collaboration node

[0069] jpege3: jpege-core3@ff357000

[0070] rc,ccu=vepu-ccu; / / reference JPEGE multi-core collaboration node

[0071] vdpu: vdpu-core@ff358000 / / VDPU single core

[0072] rc,srv=mpp_service; / / reference service node

[0073] av1d:av1dec-core@ff359000 / / AV1D single core

[0074] rc,srv=mpp_service; / / reference service node

[0075] Among them, mpp-service is the service node; vepu0 is the main core of VEPU, vepu1, vepu2 and vepu3 are the secondary cores of VEPU, vepu-ccu is the multi-core cooperation node of VEPU; jpege0 is the main core of JPEGE, jpege1, jpege2 and jpege3 are the secondary cores of JPEGE, jpege-ccu is the multi-core cooperation node of JPEGE; vdpu is the single-core node of VDPU; av1d is the single-core node of AV1D.

[0076] Figure 2 FIG. 1 is a flowchart showing an IP driver loading process according to an embodiment of the present disclosure. Figure 2 , the method includes the following steps S21 to S252.

[0077] Step S21, parsing the node configuration information of the IP core one by one according to the node order in the dtsi.

[0078] Step S22, matching the IP driver according to the node name of the IP core.

[0079] Step S231: If the node of the IP core references the service node, go to step S232; otherwise, jump to S241.

[0080] Step S232, instantiate a task queue as the task queue driven by IP. Since the node of IP core references the service node, it means that the current node is the main core of IP multi-core or IP is a single core.

[0081] Step S233, register the IP driver to the service driver, wherein the service driver is responsible for the task scheduling and distribution of all IP drivers.

[0082] Step S241: If the node of the IP core references the multi-core collaboration node, go to step S242; otherwise, jump to S251.

[0083] Step S242: Point the IP core to the corresponding multi-core collaboration unit. A bidirectional link is established between the IP core and the multi-core collaboration unit.

[0084] Step S251: If the IP core node is configured with IOMMU parameters, the next step is entered. IOMMU (Input-Output Memory Management Unit) is a memory management device used for data access between the IP core and DDR (Double Data Rate).

[0085] Step S252, instantiate a memory mapping table for the IOMMU corresponding to the IP core. The mapping table stores the mapping relationship between the virtual address used by the driver and the DDR physical address. Therefore, the IP driver can read and write DDR data using the virtual address through the IOMMU and the mapping table.

[0086] Figure 3 1 is a schematic diagram showing the structure of the kernel driver according to an embodiment of the present disclosure. The schematic diagram includes a service driver, a task queue, a multi-core collaboration unit (central control unit, CCU), an IP driver, and an IP core. The task queue is instantiated by the IP main core or the IP single core, and is used to receive the call task distributed by the service driver. The commonly used task queue can be a first-in-first-out one-way queue or a one-way multi-queue with priority. If the IP is multi-core, the IP driver will select a call task in the task queue as the target task and forward it to the multi-core collaboration unit. The multi-core collaboration unit selects the target core, splits the task, and distributes the target task to the target core. The IP core executes the target task, and the multi-core collaboration unit merges the execution results.

[0087] For example, in Figure 3In the embodiment, IP is a multi-core structure, C0 corresponds to m cores C01 to C0m, where C01 to C1m are cores with the same functions, C01 is the main core, and the other cores are secondary cores. In some embodiments, C01 to C0m can be completely the same cores. In other embodiments, C01 to C0m have the same structure and function, but the support capabilities of different cores may be different. For example, C01 can support 1080p resolution, C02 can support 4K resolution, and C03 can be a soft-core with the same structure and function.

[0088] For example, in Figure 3 In the example, task queue 0 is instantiated when the main core C01 is loaded, and is used to receive the call task distributed by the service driver. CCU0 is the multi-core collaboration unit of C0, and all point to cores C01 to C1m. When the service driver receives a call task of type C0, it distributes the call to task queue 0; the IP driver selects a call task from task queue 0 as the target task and forwards it to CCU0; CCU0 selects one or more idle cores from C01 to C1m as the target core according to the multi-core collaboration strategy; the target core executes the target task.

[0089] For example, in Figure 3 In the example, C2 is an IP single-core structure, and task queue 2 is instantiated when core C2 is loaded. When the service driver receives a call task of type C2, it distributes the call task to task queue 2; the IP driver selects a call task from task queue 2 as the target task; the target task is directly executed on C2.

[0090] Figure 4 is a flowchart showing a kernel driving method according to an embodiment of the present disclosure. Figure 4 , the method includes the following steps S41 to S46.

[0091] Step S41, the service driver receives the calling task in the user state. Exemplarily, the service driver can receive the calling task through a character device file, wherein the character device file is created when the service driver is loaded, and is an interface for interaction between the user state and the kernel state. Exemplarily, for example, / dev / mpp_service.

[0092] Step S42, the service driver sends the calling task to the task queue of the corresponding type of IP driver according to the IP type corresponding to the calling task. Exemplarily, it can be determined whether the device_type parameter in the calling task is the same as the device_type in the IP driver.

[0093] Step S43, the IP driver selects a calling task from the task queue as the target task. The task queue is generally a serial queue. For example, a serial single queue based on the first-in-first-out principle.

[0094] Step S441, the IP driver determines whether the IP corresponding to the target task is a multi-core structure, if so, it goes to step S442, otherwise it jumps to step S45.

[0095] Step S442: The IP driver sends the target task to the corresponding multi-core collaboration unit.

[0096] Step S443: The multi-core collaboration unit selects a target core from the multiple cores of the IP according to the priority strategy. Exemplarily, the target core may be one or more idle cores.

[0097] Step S444: the multi-core collaboration unit schedules the target task to the target core, and jumps to step S46.

[0098] Step S45: If the IP is a single core, the IP driver sends the target task to the single core of the IP.

[0099] Step S46, the target task is executed by the target core of the IP multi-core or the IP single-core core.

[0100] According to the above description, in the embodiment of the present disclosure, if the IP is a multi-core structure, the IP driver will send the target task to the multi-core collaboration unit, and the multi-core collaboration unit will select one or more cores as the target core according to the priority strategy, the number of idle cores and the complexity of the task. In this way, a complex target task can be split into multiple target subtasks, and multiple cores are executed in parallel to achieve multi-core collaborative processing of scheduling tasks. If the IP is a single core, the IP driver directly sends the target task to the core of the IP to execute the target task.

[0101] It can be seen that the method provided by the embodiment of the present disclosure can support both IP multi-core collaboration and IP single-core compatibility. By simply configuring the nodes of each IP core in dtsi, the chip containing IP multi-core and IP single-core can be adapted, which improves the reusability, adaptability and flexibility of the driver. For the application layer, the hardware differences between multi-core and single-core are shielded, reducing the complexity of development.

[0102] In some implementations, the kernel driver method provided by the embodiment of the present disclosure may also include: the service driver and the IP driver load the corresponding service driver and the IP driver in the firmware into the operating system according to the node configuration information in the dtsi during the firmware loading phase of the device startup. The service driver receiving the calling task in the user state includes: the service driver receiving the calling task through the character device file, which is created when the service driver is loaded and serves as the general entry for all IP drivers.

[0103] In some implementations, the node configuration information in the dtsi may include: configuration information of the service node; node configuration information of each IP core; if the IP is a multi-core structure, there is also configuration information of a multi-core collaboration node.

[0104] In some implementations, the kernel driving method provided by the embodiment of the present disclosure may further include: if the IP is a multi-core structure, the first core is identified as the main core and the other cores are secondary cores, the node configuration information of the main core references the service node and the multi-core collaboration node, and the node configuration information of the secondary core only references the multi-core collaboration node. If the IP is a single-core structure, the node configuration information of the IP core only references the service node.

[0105] In some implementations, taking video encoding as an example, the application layer can divide a frame of image into M tiles, each tile corresponds to a coding task, where M is a positive integer greater than 1. If the corresponding encoding IP has multiple cores, the multi-core collaboration unit may assign these tasks to different idle cores and execute multiple coding tasks in parallel. After the coding tasks of the M tiles are completed, the application layer merges all the execution results to obtain the coding result of a frame of image.

[0106] Figure 5A It is a schematic diagram showing the IP multi-core executing the application layer split scheduling task in the embodiment of the present disclosure. Figure 5A As shown in the figure, the encoding IP has 4 homogeneous cores C0 to C3, with C0 being the main core. The application layer divides a frame of image into 4 tiles, such as tile0 to tile3 in the figure, and generates 4 call tasks. After receiving the call task, the service driver sends it to the task queue. The multi-core collaboration unit takes the call task from the task queue and sends it to C0 to C3 in order for execution. After all the call tasks are completed, the application layer merges the execution results of tile0 to tile3.

[0107] In some implementations, the IP core contains multiple homogeneous cores. After receiving the target task, the multi-core collaboration unit first determines whether there are idle cores, and uses a priority strategy to sort the idle cores, and then selects one or more idle cores based on the number of idle cores and the complexity of the target task. If multiple cores are selected, the target task is split into a corresponding number of target subtasks and sent to the selected cores for execution. After all target subtasks are executed, the results are merged.

[0108] Figure 5B It is a schematic diagram showing the IP multi-core executing the multi-core collaboration unit splitting target task in the embodiment of the present disclosure. Figure 5B As shown in the figure, the encoding IP has 4 homogeneous cores C0~C3, with C0 as the main core. After the service driver receives a frame encoding call task from the application layer, it sends it to the task queue. The multi-core collaboration unit divides a frame of image into 4 target subtasks of tile0~tile3, and sends them to C0~C3 for execution respectively. After the target subtask is executed, all execution results are merged, and finally the result of a frame encoding is output.

[0109] Figure 5C The flowchart of the multi-core collaboration unit executing the target task in the embodiment of the present disclosure is shown. Figure 5C , the method includes the following steps S51c to S57c.

[0110] Step S51c: the multi-core collaboration unit receives a target task.

[0111] Step S521c, determine whether there is an idle core, if so adjust to step S53c, otherwise go to step S522c.

[0112] Step S522c, the current thread is dormant and waits to be awakened.

[0113] Step S53c: Use a priority strategy to sort the idle cores. For example, the priority strategy may be directional polling or cumulative task load.

[0114] Step S54c, judging whether splitting is required according to the task complexity in the target task, if splitting is required, proceeding to step S551c, otherwise jumping to step S56c. Exemplarily, the task complexity can be judged according to the resolution information contained in the target task.

[0115] Step S551c, split the target task according to the number of idle cores.

[0116] Step S552c, sending the split target task to the hollow core for execution.

[0117] Step S553c, merging the execution results of all split target tasks.

[0118] Step S56c, sending the target task to the first idle core after sorting for execution.

[0119] Step S57c, the target task is executed and the dormant thread is awakened.

[0120] In some implementations, selecting a target core from multiple cores of the IP includes: selecting a core or multiple cores, including: if the target task is a simple task, selecting an idle core as the target core; if the target task is a complex task, selecting multiple idle cores as the target cores, and splitting the target task into a corresponding number of target subtasks.

[0121] In order to ensure load balancing of each core while meeting the requirements of efficient utilization of hardware resources, in the embodiments of the present disclosure, the priority strategy may include: the multi-core collaboration unit uses a directed polling method to prioritize the idle cores among the multiple cores; the multi-core collaboration unit uses a cumulative task load method to prioritize the idle cores among the multiple cores.

[0122] In some implementations, the multi-core collaboration unit uses a directional polling method to prioritize idle cores among multiple cores, including: starting from the starting core, querying each core in turn along a preset direction in a circular manner to see if it is idle, and the order of the idle cores is the priority order. The starting core of the first polling is the preset core, and the starting cores of the remaining pollings are the (core_id+1)%N cores in the multi-core cores, core_id is the number of the last core of the target core in the previous polling, and N is the number of the multi-core cores.

[0123] Exemplarily, the IP has four isomorphic cores C0-C3, the preset starting core is C0, the direction is C0-C1-C2-C3 forward cycle, and one core is selected as the target core each time.

[0124] In the first round, the starting core is C0, and the idle cores are polled according to C0-C1-C2-C3. The idle cores C0, C1, C2, and C3 are found, and the priority order is C0-C1-C2-C3. C0 is selected as the target core.

[0125] In the second round, the core_id of the target core in the previous round is 0, the starting core is C1, and the idle cores are polled according to C1-C2-C3-C0. The idle cores C1, C2, and C3 are found, and the priority order is C1-C2-C3. C1 is selected as the target core.

[0126] In the third round, the core_id of the target core of the previous round is 1, the starting core is C2, and the idle cores are polled according to C2-C3-C0-C1. The idle cores C2 and C3 are found, and the priority order is C2-C3. C2 is selected as the target core.

[0127] In the fourth round, the core_id of the target core in the previous round is 2, the starting core is C3, and the idle cores are polled according to C3-C0-C1-C2. The idle core C3 is found, the priority is C3, and C3 is selected as the target core.

[0128] In the fifth round, the core_id of the target core in the previous round is 3, the starting core is C0, and the idle cores are polled according to C0-C1-C2-C3. No idle core is found, and the core goes into sleep mode and waits to be awakened.

[0129] In some implementations, the multi-core collaboration unit prioritizes idle cores among multiple cores by using the cumulative task load method: starting from the main core, querying each core in turn along a preset direction to see if it is idle, and sorting the idle cores from small to large according to the cumulative task load, and the sorted order is the priority order. Among them, the cumulative task load of all cores is initialized to 0 for the first time, and after the target core completes the task, the cumulative task load of the corresponding core needs to be added to the task load of the currently executed task.

[0130] In some implementations, the task load may include: a load obtained based on the amount of computation required to execute the task; and a load obtained by querying a corresponding load table based on properties of the task to be executed.

[0131] For example, the IP has 4 isomorphic cores C0-C3, the main core is C0, the direction is C0-C1-C2-C3, and one core is selected as the target core each time. Assume that the task load of the target task is 2, where the task load can be calculated by resolution, calculation amount, number of runs, or other parameters that can characterize the task characteristics.

[0132] Initialize the cumulative task load of the four cores sum0=sum1=sum2=sum3=0.

[0133] In the first round, the idle cores are polled according to C0-C1-C2-C3, and the idle cores C0, C1, C2, and C3 are found. Sum0, sum1, sum2, and sum3 are sorted from small to large, with a priority of C0-C1-C2-C3. C0 is selected as the target core. After the target task is executed, sum0=2, sum1=0, sum2=0, and sum3=0.

[0134] In the second round, the idle cores are polled according to C0-C1-C2-C3, and the idle cores C1, C2, and C3 are found. Sum1, sum2, and sum3 are sorted from small to large, with a priority of C1-C2-C3. C1 is selected as the target core. After the target task is executed, sum0=2, sum1=2, sum2=0, and sum3=0.

[0135] In the third round, the idle cores are polled according to C0-C1-C2-C3, and the idle cores C2 and C3 are found. Sum2 and sum3 are sorted from small to large with a priority of C2-C3. C2 is selected as the target core. After the target task is executed, sum0=2, sum1=2, sum2=2, sum3=0.

[0136] In the fourth round, the idle cores are polled according to C0-C1-C2-C3, and the idle core C3 is found. C3 is selected as the target core. After the target task is executed, sum0=2, sum1=2, sum2=2, sum3=2.

[0137] In the fifth round, the idle cores are polled according to C0-C1-C2-C3. If no idle cores are found, the system goes into sleep mode and waits to be awakened.

[0138] In some implementations, multiple isomorphic core structures contained in an IP core have the same functions, but different support capabilities, and are usually referred to as large and small cores. When selecting a target core, the multi-core collaboration unit needs to further screen it based on the core's support capabilities. If the priority strategy uses the cumulative task load method, the task load of the target task executed by the small core can be multiplied by a coefficient greater than 1.

[0139] For example, IP has 4 isomorphic cores C0-C3, C0 is the main core, C0 and C1 are large cores with a support capacity of 4k, C2 and C3 are small cores with a support capacity of 2k, and the cumulative task load method is adopted, and the task load coefficient of the small core relative to the large core is 2. Assume that there are 3 calling tasks 4k, 2k, and 2k in the task queue, and the task load is in units of computing amount.

[0140] Initialize the cumulative task load of the four cores sum0=sum1=sum2=sum3=0.

[0141] In the first round, for 4k tasks, the priority of idle cores is C0-C1-C2-C3. Unsupported cores C2 and C3 are eliminated, and C0 is selected as the target core. The cumulative task load sum0=4k, sum1=0, sum2=0, sum3=0.

[0142] In the second round, there are 2k tasks, and the priority of idle cores is C1-C2-C3. C1 is selected as the target core, and the cumulative task load sum0=4k, sum1=2k, sum2=0, sum3=0.

[0143] In the third round, there are 2k tasks, and the priority of the idle cores is C2-C3. C2 is selected as the target core, and the cumulative task load sum0=4k, sum1=2k, sum2=4k=2x2k, sum3=0.

[0144] The IP core can access data on DDR through IOMMU and the corresponding memory mapping table. If the IP is a multi-core structure, in order to avoid contention between cores in accessing data, each core can have a dedicated IOMMU, and the IP driver loading will also instantiate a dedicated memory mapping table for the corresponding IOMMU.

[0145] In some implementations, all calling tasks in the application layer are independent of each other, have no dependencies, and are simple tasks. If the IP is a multi-core structure, since each calling task is unrelated, each core uses a dedicated memory mapping table to independently access DDR data. This method has a higher efficiency. Fig. 6A As shown, the encoding IP has 4 cores C0~C3, and the CCU sends the target task to any idle core. The idle core can access DDR data through the corresponding IOMMU and the dedicated memory mapping table mmu_table.

[0146] In some implementations, there is a dependency between the calling tasks sent by the application layer. Take video encoding as an example. For example, when the current frame is not an I frame, encoding and decoding the current frame requires reading back the previous output data as a reference frame. Fig. 6A As shown, if the IP is a multi-core structure, assume that the IP has 4 cores C0~C3. The first image is an I frame, and the CCU sends the target task to C0. C0 writes the reconstructed frame data to the DDR through mmu_talbe0. The second image is not an I frame, and it is necessary to read the reconstructed data written by the first I frame as a reference frame. If the CCU sends the target task to C1 for execution at this time, C1 reads the I frame data from the DDR through mmu_talbe1. If the contents of mmu_table0 and mmu_table1 are different, C1 cannot find the correct physical address of the output data of the I frame, which leads to data errors. To address this problem, in the disclosed embodiment, multiple isomorphic cores can be configured to use the same shared memory mapping table. As Figure 6B As shown, C1 to C3 share the memory mapping table of C0, so no matter which core the calling task is sent to by the CCU for execution, it is guaranteed that the correct data can be read.

[0147] In some implementations, if the IP is multi-core and the calling task sent by the application layer is a complex task, the multi-core collaboration unit may split the calling task into multiple subtasks and send them to multiple cores for execution. Fig. 6A As shown in the figure, different memory mapping tables are used, and there is data dependency between subtasks, then data reading errors may occur due to inconsistent memory mapping tables. To address this problem, in the embodiment of the present disclosure, some isomorphic cores can be configured to use the same shared memory mapping table. Figure 6C As shown, CCU splits the task into two subtasks and distributes them to C0 and C1 for execution. CCU can share the memory mapping table of C0's corresponding IOMMU with C1's corresponding IOMMU. In this way, it can be ensured that C0 and C1 can correctly access each other's data.

[0148] In some implementations, the information included in the calling task includes: the type of IP used, register configuration information, context dependency identifiers, multi-core collaboration identifiers, and task complexity.

[0149] To summarize, in the embodiments of the present disclosure, the service driver receives the calling task in the user state through the character device file; the service driver sends the calling task to the task queue of the corresponding type of IP driver according to the IP type corresponding to the calling task; the IP driver selects a calling task from the task queue as the target task; if the IP is a multi-core structure, the IP driver sends the target task to the corresponding multi-core collaboration unit; the multi-core collaboration unit selects the target core from the multiple cores of the IP according to the priority strategy; the multi-core collaboration unit schedules the target task to the target core for execution. In this way, multiple isomorphic cores can execute their respective target tasks in parallel, realizing multi-core collaborative processing of the target tasks. In addition, in the embodiments of the present disclosure, if the IP is a single-core, the IP driver will be directly scheduled to execute on the corresponding IP core.

[0150] In addition, in the embodiment of the present disclosure, the hardware differences between multi-core and single-core are shielded by the multi-core collaboration unit. Different chip versions can be adapted by simply configuring the corresponding dtsi information and multi-core collaboration node information for the nodes of each IP core according to the composition of the chip. There is no need to develop different driver codes for each type of chip, which improves the flexibility and reusability of the code. In addition, the application layer does not need to distinguish whether the IP is a multi-core structure or a single-core structure, which reduces the complexity of code development.

[0151] Furthermore, in the embodiment of the present disclosure, multiple homogeneous cores can solve the information synchronization problem between different cores by sharing a memory mapping table.

[0152] The protection scope of the kernel driver method provided by the embodiment of the present disclosure is not limited to the execution order of the steps listed in the embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0153] The disclosed embodiment also provides a kernel driver device based on multi-core collaboration. The driver device includes: a task queue, configured to store scheduling tasks; a character device file unit, configured to receive user-mode scheduling tasks and send the scheduling tasks to the task queue; multiple homogeneous cores, including a main core and a secondary core, the main core is the first core among the multiple homogeneous cores, and the secondary core is the other core among the multiple homogeneous cores except the first core; a multi-core collaboration unit, configured to select a scheduling task from the task queue as a target task, and select a target core from the multiple homogeneous cores to schedule the target task to the target core, so that the target core executes the target task.

[0154] It should be understood that the contents related to the driving device have been described in detail in the above method embodiment, and for the sake of brevity, they will not be repeated here.

[0155] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0156] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. For example, the functional modules / units in the various embodiments of the present disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0157] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0158] The embodiment of the present disclosure also provides an electronic device. Figure 7 The diagram is a schematic diagram of the structure of an electronic device 7 in an embodiment of the present disclosure. Figure 7 As shown, the electronic device 7 in this embodiment includes a memory 71 and one or more processors 72. The processor 72 is coupled to the memory 71 and is configured to execute the driving method provided in the embodiment of the present disclosure.

[0159] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0160] In addition, the above embodiments are only illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in the present disclosure should still be covered by the claims of the present disclosure.

Claims

1. A kernel driving method based on multi-core collaboration, characterized in that: include: The service driver receives the calling task from the user mode; The service driver sends the calling task to a task queue of an IP driver of a corresponding type according to the IP type corresponding to the calling task; The IP driver selects a calling task from the task queue as a target task; If the IP corresponding to the target task is a multi-core structure, the IP driver sends the target task to the corresponding multi-core collaboration unit; The multi-core collaboration unit selects a target core from the multiple cores of the IP according to a priority strategy; The multi-core collaboration unit schedules the target task to the target core; as well as The target task is executed by the target core.

2. The method according to claim 1, characterized in that Also includes: If the IP is a single-core structure, the IP driver sends the target task to the single-core core of the IP; as well as The target task is executed by the single core.

3. The method according to claim 1, characterized in that Also includes: The service driver and the IP driver load the corresponding service driver and IP driver in the firmware into the operating system according to the node configuration information in the dtsi during the firmware loading phase when the device is powered on. The receiving of the calling task in user mode by the service driver includes: the service driver receives the calling task through a character device file, wherein the character device file is created when the service driver is loaded and serves as a general entry for all IP drivers.

4. The method according to claim 3, characterized in that: The node configuration information in the dtsi includes: Configuration information of the service node; Node configuration information of each IP core; If the IP is a multi-core structure, there is also configuration information of a multi-core collaboration node.

5. The method according to claim 4, characterized in that Also includes: If the IP is a multi-core structure, the first core is identified as the primary core and the other cores are secondary cores, the node configuration information of the primary core references the service node and the multi-core collaboration node, and the node configuration information of the secondary core only references the multi-core collaboration node; as well as If the IP is a single-core structure, the node configuration information of the IP core only references the service node.

6. The method according to claim 4, characterized in that Loading the corresponding IP driver from the firmware into the operating system involves: According to the node order in dtsi, parse the node configuration information of IP core one by one; Matching an IP driver according to the node name of the IP core; If the node of the IP core references the service node, instantiate a task queue as the task queue of the IP driver, and register the IP driver under the service driver; If the node of the IP core references the multi-core collaboration node, pointing the IP core to the corresponding multi-core collaboration unit; and If the node of the IP core is configured with IOMMU parameters, a memory mapping table is instantiated for the IOMMU corresponding to the IP core.

7. The method according to claim 1, characterized in that The task queue includes: First-in, first-out serialized single queue; Priority-based multiple queues.

8. The method according to claim 1, characterized in that The multiple cores of the IP refer to that the IP contains multiple cores with the same functions and structures, and the cores include cores with the same performance, or large and small cores with different performances.

9. The method according to claim 1, characterized in that: The priority strategies include: The multi-core cooperation unit prioritizes idle cores among the multiple cores by adopting a directional polling method; and The multi-core collaboration unit prioritizes the idle cores among the multiple cores by accumulating task loads.

10. The method according to claim 1, characterized in that Selecting a target core from among the plurality of cores of the IP comprises: Selecting a core or multiple cores includes: if the target task is a simple task, selecting an idle core as the target core; if the target task is a complex task, selecting multiple idle cores as the target core, and splitting the target task into a corresponding number of target subtasks.

11. The method according to claim 9, characterized in that The multi-core collaboration unit prioritizes idle cores among the multiple cores in a directional polling manner, including: Starting from the starting core, each core is checked in turn along the preset direction in a circular manner to see if it is idle. The order of the idle cores is the priority order. The starting core of the first polling is the preset core, and the starting cores of the remaining pollings are the (core_id+1)%Nth cores among the multiple cores, core_id is the number of the last core of the target core in the previous polling, and N is the number of the multiple cores.

12. The method according to claim 9, characterized in that The multi-core collaboration unit prioritizes idle cores among the multiple cores by accumulating task loads, including: Starting from the main core, query each core in turn along the preset direction to see if it is idle, sort the idle cores from small to large according to the cumulative task load, and the sorted order is the priority order. The first time, the accumulated task loads of all cores are initialized to 0. After the target core completes the task, the accumulated task load of the corresponding core needs to be added with the task load of the currently executed task.

13. The method according to claim 12, characterized in that The task load includes: A load amount obtained according to the amount of computation required to perform the task; and The load amount is obtained by querying the corresponding load amount table according to the attribute of the execution task.

14. The method according to claim 6, characterized in that The memory mapping table is used to map the virtual address visible to the driver to the real physical address of the DDR, so that the driver accesses the storage data on the DDR through the IOMMU and the memory mapping table.

15. The method according to claim 1, characterized in that Also includes: If there is a front-to-back dependency relationship between the calling tasks in the user state, the target core that executes all the calling tasks in the user state is made to use the same memory mapping table.

16. The method according to claim 10, characterized in that Also includes: If the complex task is split into multiple subtasks, the target cores executing the subtasks are made to use the same memory mapping table.

17. The method according to claim 1, characterized in that Also includes: The application program splits a complex task into multiple user-mode calling tasks, and sends these calling tasks to the core of the corresponding IP type through character device files for execution; as well as After all the calling tasks are executed, the application program will merge the results of the multiple calling tasks.

18. The method according to claim 10, characterized in that Also includes: After the execution of multiple target subtasks on the target core is completed, the multi-core collaboration unit merges the execution results of the target core.

19. The method according to claim 1, characterized in that The information contained in the calling task includes: the IP type used, register configuration information, previous and next dependency identifiers, multi-core collaboration identifiers, and task complexity.

20. A kernel driver device based on multi-core collaboration, characterized in that: include: Task queue, configured to store scheduled tasks; A character device file unit is configured to receive a user-mode scheduled task and send the scheduled task to the task queue; A plurality of homogeneous cores, including a primary core and secondary cores, wherein the primary core is a first core among the plurality of homogeneous cores, and the secondary cores are other cores among the plurality of homogeneous cores except the first core; as well as The multi-core collaboration unit is configured to select a scheduling task from the task queue as a target task, and select a target core from the multiple homogeneous cores to schedule the target task to the target core so that the target core executes the target task.

21. An electronic device, characterized in that: include: Memory; as well as One or more processors, coupled to the memory and configured to execute the method according to any one of claims 1 to 19.

Images