Heterogeneous computing method, device, electronic device and storage medium
By receiving and analyzing heterogeneous computing flowcharts and assigning multiple processors to perform tasks, the problem that the prior art cannot be universal among different heterogeneous computing scenarios is solved, and the universality and flexibility of heterogeneous computing is realized.
Patent Information
- Application Number
- CN202110554407.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-20
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2041-05-20
AI Technical Summary
Existing heterogeneous computing methods and components cannot be generalized between different heterogeneous computing scenarios and cannot be applied to other heterogeneous computing tasks, such as image processing and deep learning tasks.
By receiving and analyzing heterogeneous computing flowcharts, the execution method and data transfer relationship of the task are determined, and multiple types of processors are allocated to execute tasks, realizing generalized processing of tasks.
It realizes the versatility of completing any task under heterogeneous computing method, and is suitable for any computing scenario, improving the flexibility and efficiency of computing.
Smart Images
Figure CN113377439B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of heterogeneous computing, and specifically to heterogeneous computing methods, devices, electronic devices and storage media. Background Art
[0002] Heterogeneous computing, in which multiple types of processors participate in computing, is widely used to complete tasks such as image processing tasks and machine learning tasks. Before using heterogeneous computing to complete a task, it is necessary to define relevant information such as the tasks to be performed in a configuration file.
[0003] At present, machine learning frameworks such as Tensorflow provide components for defining relevant information in specific heterogeneous computing scenarios, that is, heterogeneous computing scenarios for completing machine learning tasks in a heterogeneous computing manner. The components for defining relevant information in specific heterogeneous computing scenarios cannot be applied to other heterogeneous computing scenarios, such as heterogeneous computing scenarios for completing image processing tasks and deep learning tasks in a heterogeneous computing manner. Summary of the invention
[0004] Embodiments of the present application provide a heterogeneous computing method, device, electronic device, and storage medium.
[0005] The present application provides a heterogeneous computing method, including:
[0006] Receive a heterogeneous computing flow chart, the heterogeneous computing flow chart comprising: a task node and a control node, wherein the control node represents an execution mode of a task corresponding to each task node connected to the control node, and a connection identifier between two task nodes having a connection relationship represents a data transfer relationship between two tasks corresponding to the two task nodes;
[0007] Parsing the heterogeneous computing flow chart to determine description information of each of all tasks, wherein the description information of the task includes at least one of the following: an identifier of an execution mode of the task, and a data transfer relationship between the task and other tasks;
[0008] Assign tasks to at least some processors in a processor set, and send tasks of corresponding processors to corresponding processors so that the corresponding processors execute corresponding tasks based on description information of corresponding tasks, wherein the processor set includes: processors of multiple processor types.
[0009] The present application provides a heterogeneous computing device, including:
[0010] A receiving unit is configured to receive a heterogeneous computing flow chart, wherein the heterogeneous computing flow chart includes: a task node, a control node, and a storage node, wherein the control node indicates an execution mode of a task corresponding to each task node connected to the control node, and a connection identifier between two task nodes having a connection relationship indicates a data transfer relationship between two tasks corresponding to the two task nodes;
[0011] A parsing unit is configured to parse the heterogeneous computing flow chart to determine description information of each task in all tasks, wherein the description information of the task includes at least one of the following: an identifier of an execution mode of the task, and a data transfer relationship between the task and other tasks;
[0012] The allocation and distribution unit is configured to allocate tasks to at least part of the processors in the processor set, and distribute the tasks of the corresponding processors to the corresponding processors so that the corresponding processors can execute the corresponding tasks based on the description information of the corresponding tasks, wherein the processor set includes: processors of multiple processor types.
[0013] The heterogeneous computing method and device provided in the embodiments of the present application realize the use of a heterogeneous computing flowchart to describe each task that needs to be executed and the execution method of each task that needs to be executed when completing a task in a heterogeneous computing manner, and is applicable to any computing scenario that completes any task in a heterogeneous computing manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0015] Figure 1 A flow chart of a heterogeneous computing method provided by an embodiment of the present application is shown;
[0016] Figure 2 A structural block diagram of a heterogeneous computing device provided in an embodiment of the present application is shown;
[0017] Figure 3 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0018] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0020] Figure 1 A flowchart of a heterogeneous computing method provided in an embodiment of the present application is shown, and the method includes:
[0021] Step 101, receiving a heterogeneous computing flow chart.
[0022] In the present application, the input data of a task may be data of a single image or data of each image in an image set. For each task, executing the task includes: processing the input data of the task.
[0023] In the present application, the heterogeneous computing flow chart includes: a task node corresponding to each task.
[0024] When constructing a computational flowchart, multiple types of task nodes may be displayed to the user. For each task, the user may perform an operation to add a task node for the task. For example, the operation to add a task node for the task is to drag a task node of the displayed type of the task to the computational flowchart construction interface, and add the task node corresponding to the task to the heterogeneous computational flowchart construction interface.
[0025] For example, all tasks include multiple image filtering tasks. For each image filtering task, the user can drag the displayed image filtering type task node to the calculation flowchart construction interface, thereby adding the task node corresponding to each image filtering task to the heterogeneous calculation flowchart construction interface.
[0026] In the present application, when constructing a computational flowchart, multiple types of control nodes may be displayed to the user. The user may perform a control node addition operation, for example, the control node addition operation is to drag any type of displayed control node to the computational flowchart construction interface.
[0027] For each control node in the heterogeneous computing flow chart, the control node is connected to at least one task node, and the control node is used to indicate the execution mode of the task corresponding to each task node connected to the control node.
[0028] For a task, if the execution mode of the task is serial, that is, when the task is completed, the result data obtained by executing the task is used as the input data of at least one other task, then the task node corresponding to the task may not be connected to the control node, and the task node corresponding to the task is connected to the task node corresponding to the at least one other task.
[0029] For a task, if the execution mode of the task is loop execution, the heterogeneous computing flow chart includes: a loop type control node connected to the task node corresponding to the task. Loop execution means that during the execution of the task, multiple preset operations are executed cyclically, and the data targeted by the first preset operation is the input data of the task. For each preset operation executed after the first preset operation, the data targeted by the preset operation is the data obtained by the last preset operation executed.
[0030] For multiple tasks that need to be executed in parallel, the heterogeneous computing flowchart includes: multiple task nodes corresponding to the multiple tasks that need to be executed in parallel. Each of the multiple task nodes corresponds to a task in the multiple tasks that need to be executed in parallel. The heterogeneous computing flowchart includes: a parallel type control node that simultaneously connects the multiple task nodes.
[0031] For each task corresponding to a task node connected to the parallel control node, the parallel control node indicates that the execution mode of the task is: start executing the task at the same execution start time. The execution start time of each task in the multiple tasks that need to be executed in parallel, that is, the time when the task is started, is the same execution start time. Thus, the multiple tasks that need to be executed in parallel start executing at the same time, and the multiple tasks are executed in parallel.
[0032] In the present application, for any two task nodes having a connection relationship, there is a connection identifier between the two task nodes, and the connection identifier between the two task nodes represents the data transfer relationship between the two tasks corresponding to the two task nodes. The data transfer relationship between the two tasks corresponding to the two task nodes can be: the result data of one of the two tasks is used as the input data of the other task in the two tasks. The result data of one task is the data obtained after processing the input data of the task.
[0033] For example, the connection identifier between task nodes with a connection relationship can be a line with an arrow at one end, and the result data of the task corresponding to the task node connected at the end without an arrow in the line is used as the input data of the task corresponding to the task node connected by the arrow.
[0034] Step 102, parse the heterogeneous computing flow chart to determine the description information of each task in all tasks.
[0035] In the present application, when parsing a heterogeneous computing flowchart, each node in the heterogeneous computing flowchart can be accessed in sequence.
[0036] When a task node is accessed during the parsing of a heterogeneous computing flowchart, it can be determined that a task of this type needs to be executed based on the type of the task node, thereby determining a task corresponding to the accessed task node.
[0037] For example, when accessing a task node of a filtering type in the parsing heterogeneous computing flowchart, based on the type of the task node, i.e., the filtering type, it can be determined that a task of the filtering type, i.e., the filtering task, needs to be executed, thereby determining a filtering task corresponding to the task node.
[0038] In the present application, by accessing each task node in turn, the task corresponding to each task node can be determined, thereby determining all tasks.
[0039] After all tasks are determined, description information of each task may be determined. For each task, the description information of the task includes at least one of the following: an identifier of the execution mode of the task, and a data transmission relationship between the task and other tasks.
[0040] In the present application, when a task node that is not connected to a control node is accessed during the parsing of a heterogeneous computing flowchart, it can be determined that the execution mode of the task corresponding to the task node is serial, and the description information of the task corresponding to the task node includes: a serial identifier.
[0041] In the present application, when a control node is accessed during the parsing of a heterogeneous computing flowchart, each task node connected to the control node can be determined, and based on the type of the control node, the execution mode of the task corresponding to each task node connected to the control node can be determined. The descriptive information of the execution mode of the task corresponding to each task node includes an identifier of the execution mode of the task corresponding to the task node.
[0042] In the present application, for a task, whether the task has a data transfer relationship with other tasks can be determined based on whether the task node corresponding to the task has a connection relationship with the task nodes corresponding to other tasks.
[0043] In some embodiments, a heterogeneous computing flow chart includes: a storage node, for each storage node, the storage node represents a storage space associated with a task corresponding to a task node connected to the storage node, and the description information of the task corresponding to the task node connected to the storage node includes: the location of the storage space associated with the task corresponding to the task node connected to the storage node.
[0044] In the present application, in the heterogeneous computing flow chart, each task node can be connected to a storage node. For each storage node, the storage node represents the storage space related to the task corresponding to the task node connected to the storage node.
[0045] When a storage node is accessed during the process of parsing a heterogeneous computing flowchart, the task node connected to the storage node can be determined, and at the same time, the location information corresponding to the storage node can be obtained. The location information corresponding to the storage node includes: the starting address and the ending address of the storage space of the task corresponding to the task node connected to the storage node, thereby determining the storage space related to the task corresponding to the task node connected to the storage node.
[0046] Step 103: assign tasks to at least some of the processors in the processor set, and send the tasks of corresponding processors to corresponding processors.
[0047] The processor set includes: processors of multiple processor types. In other words, for each processor type in the multiple processor types, the processor set includes: at least one processor of the processor type.
[0048] In the present application, multiple processor types may include: central processing unit (CPU), digital signal processor (DSP), graphics processing unit (GPU), embedded neural network processor (NPU). The processor set may include: at least one CPU, at least one DSP, at least one GPU, at least one NPU.
[0049] In the present application, each processor in the processor set may be used to execute all tasks or some processors in the processor set may be used to execute all tasks. After all tasks are determined, tasks are assigned to at least some processors in the processor set. For any processor, the task assigned to the processor is called the task of the processor, and the processor executes the task of the processor based on the description information of the task of the processor.
[0050] In the present application, for any processor, the processor executing the task of the processor includes: processing input data of the task of the processor.
[0051] For a task of a processor, the task is executed according to the execution mode of the task. If the description information of the task includes the data transmission relationship between the task and other tasks, the source of the input data of the task can be determined according to the data transmission relationship between the task and other tasks, that is, the input data of the task is determined to be the result data of the other tasks. If the description information of the task does not include the data transmission relationship between the task and other tasks, the input data of the task pre-bound to the task can be obtained.
[0052] When the description information of the task includes the location of the storage space associated with the task, the storage space associated with the task can be determined based on the location of the storage space associated with the task. During the execution of the task, the input data of the task is stored in the storage space associated with the task, and the processing thread of the processor reads the input data of the task from the storage space associated with the task and processes the input data of the task.
[0053] In some embodiments, allocating tasks to at least some of the processors in the processor set includes allocating tasks to at least some of the processors in the processor set based on priorities of processor types and busyness of the processors in the processor set.
[0054] Each processor type can correspond to an allocation ratio. The higher the priority of the processor type, the greater the allocation ratio corresponding to the processor type. For example, the order of processor type priority from high to low is: NPU, GPU, DSP, CPU.
[0055] For any processor, the busyness of the processor may be calculated by dividing the amount of currently occupied resources among the computing resources of the processor by the amount of computing resources of the processor.
[0056] If all tasks include multiple tasks executed in parallel by multiple processors, for each of the multiple processors, the task assigned to the processor among the multiple tasks is one of the multiple tasks. Among all tasks, the tasks other than the multiple tasks executed in parallel by multiple processors are the remaining tasks. For each processor type, the number of all remaining tasks can be multiplied by the ratio corresponding to the processor type, and the result can be integerized to obtain the number of all remaining tasks assigned to the processor type. For each processor of a processor type, the lower the busyness of the processor, the greater the number of remaining tasks assigned to the processor among all remaining tasks assigned to the processor type.
[0057] If all tasks do not include tasks executed in parallel by multiple processors, the total number of tasks can be multiplied by the ratio corresponding to the processor type, and the result can be rounded to an integer to obtain the number of all tasks assigned to the processor type. For each processor of a processor type, the lower the busyness of the processor, the greater the number of tasks assigned to the processor among all tasks assigned to the processor type.
[0058] In some embodiments, the processor set includes at least one central processing unit; sending the tasks of the corresponding processor to the corresponding processor includes: sending the tasks of the central processing unit to the global task pool of the central processing unit, wherein the processing thread of the central processing unit is configured to obtain the tasks of the central processing unit from the global task pool of the central processing unit; and adding the obtained tasks of the central processing unit to the task pool of the processing thread of the central processing unit.
[0059] For each CPU in the processor set, the processing thread of the CPU is a thread running on the CPU for executing the task of the CPU.
[0060] For each CPU in the processor set, the global task pool of the CPU is used to store all tasks assigned to the CPU, and each processing thread of the CPU has a task pool.
[0061] For each CPU in the processor set, when the tasks of the CPU are sent to the global task pool of the CPU, one task of the CPU may be sent to the global task pool of the CPU at a time.
[0062] For a processing thread of a CPU, the processing thread of the CPU can be configured to obtain the task of the CPU from the global task pool of the CPU, and add the obtained task of the CPU to the task pool of the processing thread of the CPU. After the processing thread of the CPU obtains a task of the CPU from the global task pool of the CPU each time, the obtained task of the CPU in the global task pool is deleted.
[0063] For a processing thread of a CPU, the processing thread of the CPU can obtain a task of the CPU from the task pool of the processing thread of the CPU each time and can split the task of the CPU into multiple subtasks, and execute each subtask separately.
[0064] In the present application, when the processor set includes at least one DSP, for each DSP, each time when the number of tasks of the DSP that have not been issued is greater than a preset number, the number of tasks assigned to the DSP that is greater than the preset number is issued to the task queue of the DSP. The processing thread of the DSP obtains a task of the DSP from the task queue of the DSP each time and processes the obtained task of the DSP.
[0065] In some embodiments, the processor set includes at least one image processor; sending the tasks of the corresponding processor to the corresponding processor includes: sending the tasks of the image processor to a second-level task pool of the image processor, wherein the scheduling thread of the image processor is configured to obtain the tasks of the image processor from the second-level task pool of the image processor, and add the obtained tasks of the image processor to the first-level task pool of the image processor, the upper limit of the number of tasks corresponding to the second-level task pool is greater than the upper limit of the number of tasks corresponding to the first-level task pool, and the processing thread of the image processor is configured to obtain the tasks of the image processor from the first-level task pool and process the obtained tasks of the image processor.
[0066] In the present application, for each GPU in a processor set, the scheduling thread of the GPU is a thread running on the GPU, which is used to obtain the tasks of the GPU from the second-level task pool of the GPU and add the obtained tasks of the GPU to the first-level task pool of the GPU, and the processing thread of the GPU is a thread running on the GPU, which is used to execute the tasks of the GPU.
[0067] For each GPU in the processor set, the task assigned to the GPU can be sent to the second-level task pool of the GPU. One task of the GPU can be sent to the second-level task pool of the GPU at a time.
[0068] For each GPU in the processor set, the scheduling thread of the GPU may be configured to obtain the task of the GPU from the second-level task pool of the GPU, and add the obtained task of the GPU to the first-level task pool of the GPU.
[0069] For each GPU in the processor set, the upper limit of the number of tasks corresponding to the second-level task pool of the GPU represents the maximum number of tasks of the GPU stored in the second-level task pool of the GPU. The upper limit of the number of tasks corresponding to the first-level task pool of the GPU represents the maximum number of tasks of the GPU stored in the first-level task pool of the GPU. The upper limit of the number of tasks corresponding to the second-level task pool of the GPU is greater than the upper limit of the number of tasks corresponding to the first-level task pool of the GPU.
[0070] For each GPU in the processor set, the processing thread of the GPU may be configured to obtain the task of the GPU from the first-level task pool of the GPU and process the obtained task of the GPU.
[0071] In some embodiments, it also includes: for each of a plurality of tasks executed in parallel by processors in a processor subset, dividing the input data of the task into a plurality of data blocks of the task, wherein the processor subset belongs to a processor set, each processor in the processor subset is of a different type, and each processor in the processor subset corresponds to one of the plurality of tasks; during execution of the task by the processor corresponding to the task, storing one data block of the task each time in a sub-storage space of a storage space associated with the processor subset and allocated to the processor corresponding to the task, wherein the processor corresponding to the task is configured to read one data block of the task from the sub-storage space allocated to the processor corresponding to the task each time and process one data block of the task.
[0072] In the present application, when multiple tasks are executed in parallel using multiple processors of different types in a processor set, the multiple processors of different types may constitute a processor subset, and the multiple tasks are multiple tasks executed in parallel by the processors in the processor subset.
[0073] For example, the processor subset is composed of a CPU, a DSP, a GPU, and an NPU in the processor set. The CPU, the DSP, the GPU, and the NPU execute four tasks in parallel, and each of the CPU, the DSP, the GPU, and the NPU corresponds to one task.
[0074] Each processor in the processor subset corresponds to one of the multiple tasks executed in parallel by the processors in the processor subset. For one of the multiple tasks executed in parallel by the processors in the processor subset, during the period when the processors in the processor subset execute the multiple tasks in parallel, the processor corresponding to the task processes input data of the task.
[0075] In the present application, the processors in a processor subset executing multiple tasks in parallel may mean that: each processor in the processor subset starts to execute the task corresponding to the processor at the same time or each processor in the processor subset starts to execute the task corresponding to the processor within a short time period.
[0076] In the present application, the storage space associated with the processor subset may be divided into a plurality of sub-storage spaces, and for each processor in the processor subset, one of the plurality of sub-storage spaces may be allocated to the processor.
[0077] In the present application, for each of the multiple tasks executed in parallel by the processors in the processor subset, the input data of the task can be divided into multiple data blocks of the task. During the period when the processor corresponding to the task executes the task, i.e., processes the input data of the task, one data block of the task is stored in the sub-storage space assigned to the processor corresponding to the task each time. During the period when the processor corresponding to the task executes the task, the processor corresponding to the task is configured to read one data block of the task from the sub-storage space of the processor corresponding to the task each time and process one data block of the task. The processor corresponding to the task can store one data block of the task read from the sub-storage space of the processor corresponding to the task in the internal memory of the processor corresponding to the task each time, and then, the processing thread of the processor corresponding to the task processes one data block of the task. Each time the processor corresponding to the task completes the processing of one data block of the task, if there is an unprocessed data block of the task, then replace the one data block of the task in the sub-storage space of the processor corresponding to the task that has been processed the most recently.
[0078] In some embodiments, it also includes: for each data block of a target task among multiple tasks executed in parallel by processors in a processor subset, dividing the data block into multiple sub-data blocks of the data block, wherein, during the period when the processor corresponding to the target task processes the data block, the processor corresponding to the target task is configured to read one sub-data block of the data block from the sub-storage space allocated to the target processor in the storage space associated with the processor subset each time and store one sub-data of the read data block in the internal memory of the processor corresponding to the target task.
[0079] In the present application, a target task among the multiple tasks executed in parallel by the processors in the processor subset is a task executed by a specific type of processor, such as a DSP or a GPU.
[0080] For each data block of the target task, during the period when the processor corresponding to the target task processes the data block, the data block can be divided into a plurality of sub-data blocks of the data block. During the period when the processor corresponding to the target task processes a data block of the target task, the processor corresponding to the target task is configured to read a sub-data block of the data block from the sub-storage space allocated to the target processor each time and store the read sub-data of the data block in the internal memory of the processor corresponding to the target task, and the processing thread of the processor corresponding to the target task processes the sub-data of the data block.
[0081] Please refer to Figure 2, which shows a structural block diagram of a heterogeneous computing device provided in an embodiment of the present application. The heterogeneous computing device includes: a receiving unit 201, a parsing unit 202, and an allocating and issuing unit 203.
[0082] The receiving unit 201 is configured to receive a heterogeneous computing flow chart, the heterogeneous computing flow chart comprising: a task node, a control node, and a storage node, wherein the control node represents an execution mode of a task corresponding to each task node connected to the control node, and a connection identifier between two task nodes having a connection relationship represents a data transfer relationship between two tasks corresponding to the two task nodes;
[0083] The parsing unit 202 is configured to parse the heterogeneous computing flow chart to determine the description information of each task in all tasks, wherein the description information of the task includes at least one of the following: an identification of the execution mode of the task, and a data transfer relationship between the task and other tasks;
[0084] The allocation and distribution unit 203 is configured to allocate tasks to at least part of the processors in the processor set, and distribute the tasks of the corresponding processors to the corresponding processors so that the corresponding processors can execute the corresponding tasks based on the description information of the corresponding tasks, wherein the processor set includes: processors of multiple processor types.
[0085] In some embodiments, the heterogeneous computing flowchart includes: a storage node, the storage node represents a storage space associated with a task corresponding to a task node connected to the storage node, and the description information of the task includes: a location of the storage space associated with the task.
[0086] In some embodiments, the allocating and issuing unit 203 is further configured to allocate tasks to at least some of the processors in the processor set based on the priorities of the processor types and the busyness of the processors in the processor set.
[0087] In some embodiments, the processor set includes at least one central processing unit; the allocation and distribution unit 203 is further configured to distribute the tasks of the central processing unit to the global task pool of the central processing unit, wherein the processing thread of the central processing unit is configured to obtain the tasks of the central processing unit from the global task pool of the central processing unit; and add the obtained tasks of the central processing unit to the task pool of the processing thread.
[0088] In some embodiments, the processor set includes at least one image processor; the allocation and distribution unit 203 is further configured to distribute the tasks of the image processor to the second-level task pool of the image processor, wherein the scheduling thread of the image processor is configured to obtain the tasks of the image processor from the second-level task pool of the image processor, and add the obtained tasks of the image processor to the first-level task pool of the image processor, the upper limit of the number of tasks corresponding to the second-level task pool is greater than the upper limit of the number of tasks corresponding to the first-level task pool, and the processing thread of the image processor is configured to obtain the tasks of the image processor from the first-level task pool and process the obtained tasks of the image processor.
[0089] In some embodiments, the apparatus further comprises:
[0090] A storage management unit is configured to divide the input data of each task among a plurality of tasks executed in parallel by processors in a processor subset into a plurality of data blocks of the task, wherein the processor subset belongs to the processor set, each processor in the processor subset is of a different type, and each processor in the processor subset corresponds to one of the plurality of tasks; and during execution of the task by the processor corresponding to the task, each time a data block of the task is stored in a sub-storage space of a storage space associated with the processor subset and allocated to the processor corresponding to the task, wherein the processor corresponding to the task is configured to read each time a data block of the task from the sub-storage space allocated to the processor corresponding to the task and process the data block of the task.
[0091] In some embodiments, the apparatus further comprises:
[0092] The storage management unit is further configured to divide each data block of a target task among the multiple tasks into a plurality of sub-data blocks of the data block, wherein, during the period when the processor corresponding to the target task processes the data block, the processor corresponding to the target task is configured to read one sub-data block of the data block from the sub-storage space allocated to the target processor in the storage space associated with the processor subset each time and store one sub-data of the read data block in the internal memory of the processor corresponding to the target task.
[0093] Figure 33 is a block diagram of a structure of an electronic device provided by this embodiment. The electronic device includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332, which is used to store instructions executable by the processing component 322, such as an application. The application stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 322 is configured to execute instructions to perform the above method.
[0094] The electronic device may also include a power supply component 326 configured to perform power management of the electronic device, a wired or wireless network interface 350 configured to connect the electronic device to a network, and an input / output (I / O) interface 358. The electronic device may operate based on an operating system stored in the memory 332, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™ or the like.
[0095] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by an electronic device to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0096] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0097] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A heterogeneous computing method, characterized in that: The method comprises: Receive a heterogeneous computing flow chart, the heterogeneous computing flow chart comprising: a task node and a control node, wherein the control node represents an execution mode of a task corresponding to each task node connected to the control node, and a connection identifier between two task nodes having a connection relationship represents a data transmission relationship between two tasks corresponding to the two task nodes; The data transfer relationship is as follows: the result data of one of the two tasks is used as the input data of the other of the two tasks, and the result data of the one task is the data obtained after processing the input data of the task; Parsing the heterogeneous computing flow chart to determine description information of each of all tasks, wherein the description information of the task includes at least one of the following: an identifier of an execution mode of the task, and a data transfer relationship between the task and other tasks; Allocating tasks of at least some processors in a processor set, and sending tasks of corresponding processors to corresponding processors, so that the corresponding processors perform corresponding tasks based on description information of corresponding tasks, wherein the processor set includes: processors of multiple processor types; The multiple processor types include: central processing unit, digital signal processor, image processor and embedded neural network processor; For each of a plurality of tasks executed in parallel by processors in a processor subset, input data of the task is divided into a plurality of data blocks of the task, wherein the processor subset belongs to the processor set, each processor in the processor subset is of a different type, and each processor in the processor subset corresponds to one of the plurality of tasks; during execution of the task by the processor corresponding to the task, one data block of the task is stored each time in a sub-storage space of a storage space associated with the processor subset and allocated to the processor corresponding to the task, wherein the processor corresponding to the task is configured to read one data block of the task from the sub-storage space allocated to the processor corresponding to the task each time and process one data block of the task.
2. The method according to claim 1, characterized in that The heterogeneous computing flow chart includes: a storage node, the storage node represents a storage space related to a task corresponding to a task node connected to the storage node, and the description information of the task includes: a location of the storage space related to the task.
3. The method according to claim 1, characterized in that The task of allocating at least some of the processors in the set of processors includes: Based on the priorities of the processor types of the processors in the processor set and the busyness of the processors, tasks are allocated to at least some of the processors in the processor set.
4. The method according to any one of claims 1 to 3, characterized in that The processor set includes at least one central processor; sending the tasks of the corresponding processor to the corresponding processor includes: The tasks of the central processing unit are sent to the global task pool of the central processing unit, wherein the processing thread of the central processing unit is configured to obtain the tasks of the central processing unit from the global task pool of the central processing unit; and the obtained tasks of the central processing unit are added to the task pool of the processing thread.
5. The method according to any one of claims 1 to 3, characterized in that: The processor set includes at least one image processor; Sending the tasks of the corresponding processor to the corresponding processor includes: The tasks of the image processor are sent to the second-level task pool of the image processor, wherein the scheduling thread of the image processor is configured to obtain the tasks of the image processor from the second-level task pool of the image processor, and add the obtained tasks of the image processor to the first-level task pool of the image processor, the upper limit of the number of tasks corresponding to the second-level task pool is greater than the upper limit of the number of tasks corresponding to the first-level task pool, and the processing thread of the image processor is configured to obtain the tasks of the image processor from the first-level task pool and process the obtained tasks of the image processor.
6. The method according to claim 1, characterized in that The method further comprises: For each data block of a target task among the multiple tasks, the data block is divided into a plurality of sub-data blocks of the data block, wherein, during the period when the processor corresponding to the target task processes the data block, the processor corresponding to the target task is configured to read one sub-data block of the data block from the sub-storage space allocated to the target processor in the storage space associated with the processor subset each time and store one sub-data of the read data block in the internal memory of the processor corresponding to the target task.
7. A heterogeneous computing device, characterized in that: The device comprises: A receiving unit is configured to receive a heterogeneous computing flow chart, wherein the heterogeneous computing flow chart includes: a task node, a control node, and a storage node, wherein the control node represents an execution mode of a task corresponding to each task node connected to the control node, and a connection identifier between two task nodes having a connection relationship represents a data transfer relationship between two tasks corresponding to the two task nodes; The data transfer relationship is as follows: the result data of one of the two tasks is used as the input data of the other of the two tasks, and the result data of the one task is the data obtained after processing the input data of the task; A parsing unit is configured to parse the heterogeneous computing flow chart to determine description information of each task in all tasks, wherein the description information of the task includes at least one of the following: an identifier of an execution mode of the task, and a data transfer relationship between the task and other tasks; an allocating and issuing unit, configured to allocate tasks of at least part of the processors in the processor set, and issue the tasks of the corresponding processors to the corresponding processors, so that the corresponding processors perform the corresponding tasks based on the description information of the corresponding tasks, wherein the processor set includes: processors of multiple processor types; The multiple processor types include: central processing unit, digital signal processor, image processor and embedded neural network processor; For each of a plurality of tasks executed in parallel by processors in a processor subset, input data of the task is divided into a plurality of data blocks of the task, wherein the processor subset belongs to the processor set, each processor in the processor subset is of a different type, and each processor in the processor subset corresponds to one of the plurality of tasks; during execution of the task by the processor corresponding to the task, one data block of the task is stored each time in a sub-storage space of a storage space associated with the processor subset and allocated to the processor corresponding to the task, wherein the processor corresponding to the task is configured to read one data block of the task from the sub-storage space allocated to the processor corresponding to the task each time and process one data block of the task.
8. An electronic device comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 6. 9 . A storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 6.