Multitask processing method and device, electronic equipment, chip and medium
通过获取不同算子的运算效率参数并在满足条件时将同类型算子转换分配到不同硬件模块,解决了计算网络中算子集中出现导致的硬件资源闲置问题,提高了计算效率。
Patent Information
- Application Number
- CN202310573073.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-22
AI Technical Summary
In computing networks, due to the evolution of algorithms and the demand for low power consumption and high energy efficiency, similar operators appear in computing networks, resulting in the calculation tasks being centrally allocated to a hardware unit for a certain time period, while other computing units are idle. , hardware resources are not fully utilized and computing efficiency is low.
By obtaining the operation efficiency parameters of different operators, when specific conditions are met, operators of the same type are converted and assigned to different hardware modules for calculation to improve overall efficiency. The specific method includes obtaining the computing efficiency parameters on the first operator and the second operator, and when the computing efficiency of the second computing task on the second operator is higher than the first operator, the second operator is executed on the second operator. Calculate the task and ensure consistency of output data after conversion.
It improves the utilization rate of hardware resources, reduces the overall running time of the computing network, and thus improves the computing efficiency.
Smart Images

Figure CN119003144A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of intelligent computing, and in particular to a multi-tasking processing method, device, electronic device, chip and medium. Background Art
[0002] With the advancement of technology, the demand for high-performance parallel computing in the consumer electronics market is exploding, and there is a great demand for heterogeneous computing systems. At present, a large number of computing networks have various operators interlaced at the same time. In the actual operation process, different hardware units can also be operated in parallel in time to improve the efficiency of computing. However, with the evolution of algorithms, in order to cater to the demand for low power consumption and high energy efficiency of consumer electronic products such as mobile phones, computing networks have a tendency to develop in the direction of miniaturization, and similar operators have appeared in the computing network. In the actual computing process, there will be a situation where computing tasks are concentratedly assigned to one of the hardware units in a certain period of time, while other computing units are idle, hardware resources are not fully utilized, and computing efficiency is low. Summary of the invention
[0003] The present disclosure provides a multi-tasking processing method, device, electronic device, chip and medium to solve the problems in the related technology, utilize idle hardware resources, distribute the same type of operators to different hardware modules for calculation through partial conversion, so as to achieve the purpose of improving overall efficiency.
[0004] A first aspect embodiment of the present disclosure proposes a multi-task processing method, the method comprising: obtaining a computing efficiency parameter of a second computing task on a first operator on the first operator and a second operator; when a preset condition is met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, executing the second computing task on the second operator, wherein the preset condition comprises at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
[0005] In some embodiments of the present disclosure, there are multiple second computing tasks, and obtaining the computing efficiency parameters of the second computing task on the first operator on the first operator and the second operator includes: obtaining the overall computing efficiency parameters of the second computing task on the first operator and the second operator.
[0006] In some embodiments of the present disclosure, the preset condition also includes: the output of the second computing task is the input of the third computing task executed on the second operator, and the method also includes: based on the output of the second computing task, executing the third computing task on the second operator.
[0007] In some embodiments of the present disclosure, there are multiple second computing tasks, and the second computing task includes a first subtask and a second subtask, wherein when the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method also includes: determining whether the computing efficiency of the first subtask on the second operator is higher than the computing efficiency on the first operator; if so, executing the first subtask on the second operator, and after the first computing task is completed on the first operator, executing the second subtask on the first operator based on the computing result of the first subtask obtained from the shared memory.
[0008] In some embodiments of the present disclosure, the method further includes: if the computing efficiency of the first subtask on the second operator is lower than or equal to the computing efficiency on the first operator, wait for the first computing task to be completed on the first operator, and then execute the second computing task on the first operator.
[0009] In some embodiments of the present disclosure, a first operator is used to perform a convolution operation, and a second operator is used to perform a vector operation, and executing a second computing task on the second operator includes: obtaining input data and convolution parameters of the second computing task; converting the convolution parameters into a preset format; and executing the second computing task on the second operator based on the input data and the convolution parameters in the preset format, wherein output data of executing the second computing task on the second operator is the same as output data of executing the second computing task on the first operator.
[0010] In some embodiments of the present disclosure, when a preset condition is not met, or the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method further includes: waiting for the first computing task to be completed on the first operator, and then executing the second computing task on the first operator.
[0011] A second aspect of the present disclosure provides a multi-task processing device, which includes: an acquisition module for acquiring a computing efficiency parameter of a second computing task on a first operator on the first operator and the second operator; a processing module for executing a second computing task on the second operator when a preset condition is met and the computing efficiency of the second computing task on the second operator is higher than that on the first operator, wherein the preset condition includes at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
[0012] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect embodiment of the present disclosure.
[0013] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.
[0014] The fifth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive code instructions and send code instructions to the processor, and the processor is used to run the code instructions to execute the method described in the first aspect embodiment of the present disclosure.
[0015] In summary, according to the multi-task processing method proposed in the present disclosure, the method includes: obtaining the operation efficiency parameters of the second computing task on the first operator on the first operator and the second operator; when the preset conditions are met and the operation efficiency of the second computing task on the second operator is higher than the operation efficiency on the first operator, executing the second computing task on the second operator, wherein the preset conditions include at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state. This solution utilizes idle hardware resources to allocate operators of the same type to different hardware modules for calculation through partial conversion, so as to achieve the purpose of improving overall efficiency.
[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0018] Figure 1 A flowchart of a multi-task processing method provided by an embodiment of the present disclosure;
[0019] Figure 2 A flowchart of a multi-task processing method provided by an embodiment of the present disclosure;
[0020] Figure 3 A schematic diagram of a multi-task processing method provided by an embodiment of the present disclosure;
[0021] Figure 4 A flowchart of a multi-task processing method provided by an embodiment of the present disclosure;
[0022] Figure 5 A schematic diagram of a multi-task processing method provided by an embodiment of the present disclosure;
[0023] Figure 6 A schematic diagram of a multi-task processing method provided by an embodiment of the present disclosure;
[0024] Figure 7 A schematic diagram of a multi-task processing method provided by an embodiment of the present disclosure;
[0025] Figure 8 A schematic diagram of a multi-task processing method provided by an embodiment of the present disclosure;
[0026] Fig. 9 A schematic diagram of the structure of a multi-task processing device provided by an embodiment of the present disclosure;
[0027] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar originals or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.
[0029] Heterogeneous computing was born in the mid-1980s. It mainly refers to the computing method of using computing units with different types of instruction sets and architectures to form a system. Common computing unit categories include CPU, GPU, DSP, ASIC, FPGA, etc. A heterogeneous computing platform often contains processors using different instruction set architectures (ISA), which work together to complete computing tasks.
[0030] With the advancement of technology, the demand for high-performance parallel computing in the consumer electronics market is exploding, especially in machine vision, artificial intelligence, cloud computing, AR / VR, software-defined radio and other emerging fields, which have great demands for heterogeneous computing systems. Heterogeneous computing enables computing units on the same system-on-chip to complete their respective computing tasks through programming, thus achieving higher efficiency and lower power consumption than a single computing unit.
[0031] Facing the consumer electronics market dominated by mobile phones, major chip design companies currently generally adopt heterogeneous computing systems to design solutions. For example, the current mainstream processors include the following hardware units: CPU, GPU, DSP. The HVX unit (Hexagon Vector eXtensions) in the DSP accelerates scalar and vector calculations, and the HTA unit (Hexagon Tensor Accelerator) accelerates tensor calculations. Similarly, in NVIDIA's NVDLA chip system, there are Conv units (Convolution Core) designed for convolution calculations, and SDP units (Single Data Point) designed for vector calculations.
[0032] In view of the fact that different hardware in heterogeneous computing systems have the function of accelerating different calculations, the corresponding software will assign different computing tasks to corresponding hardware modules when allocating computing tasks, thereby achieving the goal of improving overall computing efficiency. Dividing the entire target computing network according to the idea of dividing the corresponding computing tasks by hardware modules is a relatively intuitive and easy-to-implement software processing method, and it has also been proven to have good performance and effects. Considering that various types of operators are interlaced in a large number of computing networks at the same time, in the actual operation process, different hardware units can also be allowed to operate in parallel in time, thereby improving the efficiency of computing.
[0033] However, with the evolution of algorithms, in order to cater to the demand for low power consumption and high energy efficiency of consumer electronic products such as mobile phones, computing networks have also tended to develop in the direction of miniaturization, so similar operators have appeared in the computing network. If we still follow the previous ideas, then in the actual computing process, the computing tasks will be concentrated on one of the hardware units in a certain period of time, while other computing units will be idle.
[0034] In order to solve the problems existing in the related art, the present disclosure proposes a multi-task processing solution, which distributes the same type of operators to different hardware modules for calculation through partial conversion, so as to achieve the purpose of improving the overall efficiency.
[0035] The multi-tasking method provided by the present application is described in detail below with reference to the accompanying drawings.
[0036] Figure 1 Flow chart of a multi-task processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the multi-tasking method may include the following steps.
[0037] Step 101: Obtain computing efficiency parameters of a second computing task on a first operator on the first operator and a second operator.
[0038] Step 102: When a preset condition is met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, the second computing task is executed on the second operator.
[0039] The preset condition includes at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
[0040] It should be noted that the entire target computing network can be divided into different types of operators according to the computing logic. Different hardware in the heterogeneous computing system has acceleration functions for different operators. The computing units can complete the computing tasks they are good at. Based on the heterogeneous computing system, different types of operators are currently assigned to corresponding hardware modules.
[0041] In an embodiment of the present disclosure, the first operator and the second operator are operators included in the multi-tasking processing algorithm currently being executed. The first operator and the second operator are operators of different types and have different computing logics. For example, if the current multi-tasking processing executes a deep learning algorithm, the first operator may be an operator for calculating a convolution layer (Convolution Layer), and the second operator may be an operator for calculating a weight summation process in a fully-connected layer (FC layer). Conversely, the first operator may be an operator for calculating a weight summation process in a fully-connected layer, and the second operator may be an operator for calculating a convolution process.
[0042] In an embodiment of the present disclosure, the algorithm currently required to be executed can be divided into different types of operators, and each type of operator has at least one computing task, that is, the first operator includes at least one computing task, and the second operator includes at least one computing task. The first computing task on the first operator is the computing task being executed, and the second computing task on the first operator is considered to be a computing task waiting to be executed after the first computing task in the execution order. For example, the first operator includes four tasks, among which, sorted in order of execution, the first task is being executed, which is the first computing task, and the last three tasks are in a waiting state, which are the second computing tasks.
[0043] It should be noted that in the related art, since different types of operators are assigned to corresponding hardware modules, all computing tasks on the first operator will be assigned to the same hardware computing unit for execution, and all computing tasks on the second operator will be assigned to another hardware computing unit for execution. As a result, when there are more computing tasks on the first operator and the computing time is longer, while there are fewer computing tasks on the second operator, the corresponding hardware computing unit is idle. The present disclosure discloses the idea of allocating the same type of operators to different hardware modules for calculation through partial conversion, but the premise is that the overall computing efficiency can be improved after the conversion, otherwise it will be putting the cart before the horse.
[0044] In an embodiment of the present disclosure, the operation efficiency parameters of the second computing task on the first operator on the first operator and the second operator are obtained to determine whether to perform conversion, wherein the second computing task may be one or more. When the second computing task is multiple, it is necessary to obtain the overall operation efficiency parameters of the second computing task on the first operator and the second operator.
[0045] From the above, it can be seen that in an embodiment of the present disclosure, when the preset conditions are met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, the second computing task is executed on the second operator. The idle hardware resources corresponding to the second operator can be utilized to convert the second computing task originally waiting to be executed on the first operator to be executed on the second operator.
[0046] It can be understood that when executing different algorithms, the first operator and the second operator in this scheme can represent any two different types of operators in the currently executed algorithm. When there are many operators that meet the conversion conditions, steps 101 and 102 can be repeated to achieve higher computing efficiency.
[0047] In summary, according to the multi-task processing method proposed in the present disclosure, the method includes: obtaining the computing efficiency parameters of the second computing task on the first operator on the first operator and the second operator, and when the preset conditions are met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, executing the second computing task on the second operator. This scheme distributes the same type of operators to different hardware modules for calculation through partial conversion to achieve the purpose of improving overall efficiency.
[0048] based on Figure 1 The embodiment shown, Figure 2 The following further illustrates a flow chart of a multi-tasking method proposed by the present disclosure. Figure 2 based on Figure 1 In the illustrated embodiment, step 101 and step 102 are further defined.
[0049] like Figure 2As shown, the method may include the following steps.
[0050] Step 201: Obtain computing efficiency parameters of a second computing task on a first operator on the first operator and a second operator.
[0051] In some embodiments of the present disclosure, there are multiple second computing tasks, and obtaining the computing efficiency parameters of the second computing task on the first operator on the first operator and the second operator includes: obtaining the overall computing efficiency parameters of the second computing task on the first operator and the second operator.
[0052] In one embodiment of the present disclosure, a computing efficiency parameter can be obtained based on the time required for the operation. After the preliminary design of each computing unit of the hardware (such as a processor chip) is completed, a software model can be provided to obtain the theoretical operating efficiency of each hardware module. Based on the software model, the compiler can obtain the time required for each operator to run in the corresponding hardware unit during the static compilation stage. Through the software model, the time required for the second computing task on the first operator to run on the hardware unit corresponding to the first operator is obtained, and the time required for the second computing task on the first operator to run on the hardware unit corresponding to the second operator is obtained. Obviously, when the algorithm remains unchanged, the less time required for the operation, the higher the computing efficiency, and conversely, the more time required for the operation, the lower the computing efficiency.
[0053] In another embodiment of the present disclosure, there are multiple second computing tasks. After the preliminary design of each computing unit of the hardware is completed, the corresponding software model can be used to simulate all situations in which the multiple second computing tasks are respectively run on the first operator and the second operator, including situations in which multiple second computing tasks are all run on the first operator, multiple second computing tasks are all run on the second operator, and some second computing tasks are run on the first operator and some second computing tasks are run on the second operator, etc., so as to obtain the overall computing efficiency parameters of the second computing tasks on the first operator and the second operator under different situations.
[0054] Step 202: When a preset condition is met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, the second computing task is executed on the second operator.
[0055] The preset condition includes at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
[0056] It should be noted that when there are multiple second computing tasks, there may be situations where multiple second computing tasks are all run on the first operator, multiple second computing tasks are all run on the second operator, and some second computing tasks are run on the first operator and some second computing tasks are run on the second operator. At this time, the computing efficiency parameters when multiple second computing tasks are all run on the first operator are compared with the computing efficiency parameters of the remaining situations. When there is a situation that is higher than the computing efficiency when multiple second computing tasks are all run on the first operator, this situation is adopted to execute the second computing task on the second operator.
[0057] For example, when there are four second computing tasks, namely Task 1, Task 2, Task 3, and Task 4, the overall computing efficiency when Task 1 and Task 2 are executed on the second operator is higher than the cloud computing efficiency when Task 1, Task 2, Task 3, and Task 4 are all executed on the first operator, so Task 1 and Task 2 will be executed on the second operator.
[0058] In some embodiments of the present disclosure, when a preset condition is not met, or the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method further includes: waiting for the first computing task to be completed on the first operator, and then executing the second computing task on the first operator.
[0059] Specifically, when any one of the preset conditions is not met and the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, it means that the conversion conditions for converting the second computing task from the first operator to the second operator are met, or the computing efficiency after conversion is lower than or equal to the computing efficiency before conversion. Therefore, there is no need to convert the second computing task to be executed on the second operator, and wait until the first computing task is completed on the first operator before executing the second computing task on the first operator.
[0060] In some embodiments of the present disclosure, there are multiple second computing tasks, and the second computing task includes a first subtask and a second subtask, wherein when the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method also includes: determining whether the computing efficiency of the first subtask on the second operator is higher than the computing efficiency on the first operator; if so, executing the first subtask on the second operator, and after the first computing task is completed on the first operator, executing the second subtask on the first operator based on the computing result of the first subtask obtained from the shared memory.
[0061] In some embodiments of the present disclosure, the method further includes: if the computing efficiency of the first subtask on the second operator is lower than or equal to the computing efficiency on the first operator, wait for the first computing task to be completed on the first operator, and then execute the second computing task on the first operator.
[0062] Among them, on the first operator, the execution order of the first subtask is before the second subtask, and the second subtask needs to be calculated based on the calculation result of the first subtask.
[0063] It can be understood that, in some embodiments, there are multiple second computing tasks. When the preset conditions are met, but when the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, there is a situation where the computing efficiency of some tasks on the second operator is higher than the computing efficiency on the first operator. If it is unreasonable to abandon the conversion of some second computing tasks due to the overall efficiency, it is possible to first determine whether the computing efficiency of the first subtask on the second operator is higher than the computing efficiency on the first operator to decide whether to convert the first subtask from the first operator to the second operator for execution.
[0064] Specifically, when the computing efficiency of the first subtask on the second operator is higher than that on the first operator, the first subtask is executed on the second operator, and after the first computing task is completed on the first operator, the second subtask is executed on the first operator based on the computing result of the first subtask obtained from the shared memory, and when the computing efficiency of the first subtask on the second operator is less than or equal to that on the first operator, the second subtask of the first subtask in the second computing task is executed on the first operator.
[0065] In some embodiments of the present disclosure, the first operator and / or the second operator include any one of a Conv (Convolution) operator, an Add (vector addition) operator, a Matmul (matrix multiplication) operator, a Sigmoid (neuronal nonlinearity) operator, a Tanh (hyperbolic tangent) operator, a Softmax (normalized exponential) operator, and a BatchNorm (batch normalization, used to accelerate deep neural network training) operator.
[0066] Among them, the Conv operator and the Matmul operator can be converted to each other, operators such as Sigmoid / Tanh / Softmax can be converted to each other, and the BatchNorm operator can be converted into a combination of multiplication and addition during the compilation stage, that is, it can be converted into a Matmul operator and an Add operator. The Matmul operator and the Add operator are usually operated on the same hardware unit. The second computing task can be converted from the first operator to the second operator through conversion between operators, and then executed on the hardware unit corresponding to the second operator.
[0067] For example, the first operator can be any one of the Sigmoid, Tanh, and Softmax operators that is different from the second operator. Correspondingly, the second operator can be any one of the Sigmoid, Tanh, and Softmax operators that is different from the first operator. When the first operator is the BatchNorm operator, the second operator can be the Matmul operator and the Add operator, etc. Conversely, when the first operator is the Matmul operator and the Add operator, the second operator can be the BatchNorm operator, etc.
[0068] It should be noted that in the present disclosure, the first operator and the second operator can be converted to each other, and there are many examples of conversion between operators. The first operator and the second operator include but are not limited to the above types. On the other hand, depending on the hardware design, the operator to be converted (the first operator) and the target operator (the second operator) may also change. This scheme aims to propose that through software processing and in combination with the characteristics of the computing network, operators of the same type that were originally concentrated on a certain hardware unit can be converted to another hardware unit to perform calculations; what is ultimately expected to be achieved is that it can simultaneously achieve the effect of improving hardware utilization and reducing the overall running time of the computing network, thereby improving operating efficiency.
[0069] In summary, according to the multi-task processing method provided by the present invention, the computing efficiency parameters of the second computing task on the first operator on the first operator and the second operator are obtained. When the preset conditions are met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, the second computing task is executed on the second operator, which improves hardware utilization and reduces the overall running time of the computing network, thereby improving running efficiency.
[0070] based on Figure 1 or Figure 2 The embodiment shown, Figure 3 The following further illustrates a flow chart of a multi-tasking method proposed by the present disclosure.
[0071] exist Figure 3 In the embodiment shown, the first operator is used to perform convolution calculations, and the second operator is used to perform vector operations. Figure 1 or Figure 2 The embodiment shown is further defined as Figure 3 As shown, the method may include the following steps.
[0072] Figure 4 A simplified heterogeneous computing hardware structure is provided to illustrate the implementation process of this embodiment.
[0073] Specifically, the hardware structure includes two units for performing calculations, wherein the AI Core is used to calculate convolution operations, i.e., to allocate the first operator, and the Vector Core is used to calculate vector operations (such as vector addition Add, matrix multiplication Matmul, etc.), i.e., to allocate the second operator; the input data of the two calculation units exist in the same memory, and the calculated result data is also stored in the memory.
[0074] It should be noted that the AI Core and Vector Core in this embodiment are only used for the convenience of example and can be replaced by any hardware unit that can complete certain specific computing tasks in actual scenarios.
[0075] Figure 5 A computing network is provided to illustrate the problem existing in current technology, that is, during the computing process, there will be a situation where computing tasks are concentratedly assigned to one of the hardware units for a period of time, while other computing units are idle.
[0076] Specifically, Figure 5 As shown in the figure, the input data will go through several Conv operations in sequence, and finally get the final result output through an Add operation, corresponding to Figure 4 The hardware structure of AI Core completes Conv calculation and Vector Core completes Add calculation. So in terms of time, the whole calculation process is as follows: Figure 6 As shown in the figure. From the perspective of time, the computing tasks in the AI core are executed in the order in the network. Since the AI Core completes the calculation of each Conv operator in turn, the Conv operator labeled 4 can only start calculating after the previous three are completed and the AI Core is free. After the Conv operator labeled 4 is calculated, the two input data of the Add operator are ready, so the Vector Core starts working. Before that, the Vector Core has been idle, resulting in a waste of hardware resources.
[0077] Step 301: Obtain computing efficiency parameters of a second computing task on a first operator on the first operator and a second operator.
[0078] The specific implementation of the above step 301 can refer to, Figure 1 Step 101 of the embodiment shown, or Figure 2 Step 201, step 202 and step 203 of the illustrated embodiment are not described in detail in this disclosure.
[0079] When the preset conditions are met and the operation efficiency of the second computing task on the second operator is higher than the operation efficiency on the first operator, the method disclosed herein includes the following steps.
[0080] The preset condition includes at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
[0081] It should be understood that the explanation of related concepts (such as the first operator, the second computing task, etc.) in the embodiments of the present disclosure can refer to Figure 1 or Figure 2 The embodiments shown are not described in detail in this disclosure.
[0082] Step 302: Obtain input data and convolution parameters of the second computing task.
[0083] Step 303: convert the convolution parameters into a preset format.
[0084] Step 304: Execute a second computing task on a second operator based on the input data and the convolution parameters in a preset format.
[0085] The output data of the second computing task executed on the second operator is the same as the output data of the second computing task executed on the first operator.
[0086] In a specific embodiment of the present disclosure, Figure 5 As shown in the figure, the Conv operator labeled 4 is the first operator. When the operator labeled 1 starts calculating, the input data has been prepared and the calculation can theoretically begin. Since the operator labeled 4 is a Conv calculation and Vector Core can only perform vector operations, the Conv calculation needs to be transformed first.
[0087] In some embodiments of the present disclosure, the vector operation is a matrix multiplication operation, and the preset format is a sparse matrix.
[0088] Specifically, Figure 7 As shown, the Conv calculation is converted into a matrix multiplication calculation Matmul. The Conv calculation is a convolution of 4x4 input data (Input) and a 3x3 convolution kernel (Kernel). The transformation method is to expand the Kernel data in the Conv calculation and turn it into a sparse matrix C; and the Input data is arranged in order into a 16x1 column vector; the transformed sparse matrix C and the column vector are matrix multiplied, and the result is a 4x1 column vector, whose content is consistent with the result of the original Conv calculation. Since the calculation network is prepared in advance, the conversion of the Kernel data into a sparse matrix C can be prepared in advance; in the actual calculation process, the input data does not need to be rearranged, only the data needs to be taken at the corresponding position and transferred to the Vector Core to realize the above matrix multiplication calculation.
[0089] In the specific embodiment of the present disclosure, the second computing task is a Conv operator labeled 4 as an example. Figure 8 As shown, the Conv operator labeled 4 is converted into a Matmul operator and calculated on the Vector Core corresponding to the Matmul operator.
[0090] contrast Figure 6 and Figure 8 It can be seen that before using this solution, the Vector Core will remain idle until all four Conv operators are completed before starting to run; and the Conv operator labeled 4 cannot wait for the hardware resources of the AI Core, so even if the data is ready, it needs to wait until the previous three Conv operators are completed before starting to calculate; after using this solution, the operator labeled 4 can calculate directly on the Vector Core without waiting for the hardware resources of the AI Core. This not only saves the waiting time on the AI Core, but also uses the idle Vector Core, improves hardware utilization, and reduces the overall running time.
[0091] In some embodiments of the present disclosure, when a preset condition is not met, or the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method further includes: waiting for the first computing task to be completed on the first operator, and then executing the second computing task on the first operator.
[0092] For example, Figure 5 As shown, when the preset conditions are not met, or the Conv operator labeled 4 waits for the first computing task, i.e., the Conv operators labeled 1, 2, and 3 to complete, the calculation is performed on the AI Core corresponding to the Conv operator.
[0093] In some embodiments of the present disclosure, there are multiple second computing tasks, and the second computing task includes a first subtask and a second subtask, wherein when the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method also includes: determining whether the computing efficiency of the first subtask on the second operator is higher than the computing efficiency on the first operator; if so, executing the first subtask on the second operator, and after the first computing task is completed on the first operator, executing the second subtask on the first operator based on the computing result of the first subtask obtained from the shared memory.
[0094] In some embodiments of the present disclosure, the method further includes: if the computing efficiency of the first subtask on the second operator is lower than or equal to the computing efficiency on the first operator, wait for the first computing task to be completed on the first operator, and then execute the second computing task on the first operator.
[0095] For example, Figure 5 As shown, if the second computing task includes the Conv operator labeled 3 and the Conv operator labeled 4, the computing efficiency of the Conv operator labeled 3 and the Conv operator labeled 4 on the Vector Core is lower than or equal to the computing efficiency on the AI Core, the Conv operator labeled 3 is the first subtask, and the Conv operator labeled 4 is the second subtask. When the computing efficiency of the Conv operator labeled 3 on the Vector Core is higher than that on the AI Core, it is executed on the Vector Core, and after the Conv operators labeled 1 and 2 are executed, the computing result of the Conv operator labeled 3 is obtained from the shared memory, which is used to continue executing the Conv operator labeled 4 on the AI Core.
[0096] In some embodiments of the present disclosure, the preset condition also includes: the output of the second computing task is the input of the third computing task executed on the second operator, and the method also includes: based on the output of the second computing task, executing the third computing task on the second operator.
[0097] For example, Figure 8 As shown, the operator labeled 4 is the second computing task, and its output is the input of the Add operator executed on the Vector Core. Based on the output of the operator labeled 4, the Add calculation is executed on the Vector Core.
[0098] In the embodiments of the present disclosure, taking the first operator being used to perform convolution calculations and the second operator being used to perform vector operations as an example, when the first operator and the second operator are other operators, such as Sigmoid operator, Add operator, Tanh operator, Softmax operator, BatchNorm operator, etc., the specific implementation method is similar to the embodiments of the present disclosure and will not be elaborated in this disclosure.
[0099] It should be noted that the above examples do not limit the scope of implementation of the embodiments of the present disclosure.
[0100] In summary, the multi-task processing method disclosed in the present invention obtains the computing efficiency parameters of the second computing task on the first operator on the first operator and the second operator, obtains the input data and convolution parameters of the second computing task; converts the convolution parameters into a preset format; based on the input data and the convolution parameters in the preset format, executes the second computing task on the second operator, wherein the output data of the second computing task executed on the second operator is the same as the output data of the second computing task executed on the first operator, thereby improving hardware utilization and reducing the overall running time of the computing network, thereby improving running efficiency.
[0101] Fig. 9FIG. 4 is a schematic diagram of a multi-task processing device 400 provided in an embodiment of the present disclosure. Fig. 9 As shown, the multi-task processing device includes:
[0102] An acquisition module 410 is used to acquire a computing efficiency parameter of a second computing task on a first operator on the first operator and a second operator;
[0103] The processing module 420 is used to execute the second computing task on the second operator when a preset condition is met and the computing efficiency of the second computing task on the second operator is higher than that on the first operator, wherein the preset condition includes at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
[0104] In some embodiments, there are multiple second computing tasks, and the acquisition module 410 is specifically used to: acquire the overall computing efficiency parameter of the second computing task on the first operator and the second operator.
[0105] In some embodiments, the preset condition also includes: the output of the second computing task is the input of the third computing task executed on the second operator, and the processing module 420 is further used to: execute the third computing task on the second operator based on the output of the second computing task.
[0106] In some embodiments, there are multiple second computing tasks, and the second computing task includes a first subtask and a second subtask, wherein, when the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the processing module 420 is also used to: determine whether the computing efficiency of the first subtask on the second operator is higher than the computing efficiency on the first operator; if so, execute the first subtask on the second operator, and after the first computing task is completed on the first operator, execute the second subtask on the first operator based on the computing result of the first subtask obtained from the shared memory.
[0107] In some embodiments, the processing module 420 is further used to: if the computing efficiency of the first subtask on the second operator is lower than or equal to the computing efficiency on the first operator, wait for the first computing task to be completed on the first operator and then execute the second computing task on the first operator.
[0108] In some embodiments, the first operator is used to perform convolution operations, and the second operator is used to perform vector operations. The processing module 420 is specifically used to: obtain input data and convolution parameters of a second computing task; convert the convolution parameters into a preset format; and perform a second computing task on the second operator based on the input data and the convolution parameters in the preset format, wherein the output data of the second computing task performed on the second operator is the same as the output data of the second computing task performed on the first operator.
[0109] In some embodiments, the vector operation is a matrix multiplication operation, and the preset format is a sparse matrix.
[0110] In some embodiments, the first operator and / or the second operator includes any one of a Conv operator, an Add operator, a Matmul operator, a Sigmoid operator, a Tanh operator, a Softmax operator, and a BatchNorm operator.
[0111] In some embodiments, when the preset condition is not met, or the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the processing module 420 is also used to: wait for the first computing task to be completed on the first operator, and then execute the second computing task on the first operator.
[0112] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.
[0113] In summary, through the multi-task processing device, the computing efficiency parameters of the second computing task on the first operator on the first operator and the second operator, when the preset conditions are met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, the second computing task is executed on the second operator, wherein the preset conditions include at least one of the following: the first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state, and the hardware resources in idle state are utilized, and operators of the same type are partially converted and allocated to different hardware modules for calculation, so as to achieve the purpose of improving overall efficiency.
[0114] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.
[0115] Fig.10 1 is a block diagram of an electronic device 500 for implementing the multitasking method according to an exemplary embodiment. For example, the electronic device 500 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0116] Reference Fig.10, the electronic device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .
[0117] The processing component 502 generally controls the overall operation of the electronic device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.
[0118] The memory 504 is configured to store various types of data to support operations on the electronic device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0119] The power supply component 506 provides power to the various components of the electronic device 500. The power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 500.
[0120] The multimedia component 508 includes a screen that provides an output interface between the electronic device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the electronic device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0121] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the electronic device 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.
[0122] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0123] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 can detect the open / closed state of the electronic device 500, the relative positioning of components, such as the display and keypad of the electronic device 500, and the sensor assembly 514 can also detect the position change of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, the orientation or acceleration / deceleration of the electronic device 500, and the temperature change of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0124] The communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0125] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.
[0126] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by the processor 720 of the electronic device 500 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0127] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the multi-tasking method described in the above embodiments of the present disclosure.
[0128] The embodiments of the present disclosure further provide a computer program product, including a computer program, which executes the multi-task processing method described in the above embodiments of the present disclosure when a processor is used.
[0129] An embodiment of the present disclosure further proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive code instructions and send code instructions to the processor, and the processor is used to run the code instructions to execute the multi-tasking processing method described in the above embodiments of the present disclosure.
[0130] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0131] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0132] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.
[0133] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.
[0134] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0135] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0137] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A multitasking method, characterized in that: The method comprises: Obtaining a computing efficiency parameter of a second computing task on a first operator on the first operator and a second operator; When a preset condition is met and the operation efficiency of the second computing task on the second operator is higher than the operation efficiency on the first operator, the second computing task is executed on the second operator. The preset condition includes at least one of the following: a first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
2. The method according to claim 1, characterized in that The preset condition also includes: the output of the second computing task is the input of the third computing task executed on the second operator, and the method also includes: based on the output of the second computing task, executing the third computing task on the second operator.
3. The method according to claim 1, characterized in that There are multiple second computing tasks, and the second computing tasks include a first subtask and a second subtask, wherein when the computing efficiency of the second computing task on the second operator is lower than or equal to the computing efficiency on the first operator, the method further includes: determining whether a computing efficiency of the first subtask on the second operator is higher than a computing efficiency of the first subtask on the first operator; If yes, the first subtask is executed on the second operator, and after the first computing task is completed on the first operator, the second subtask is executed on the first operator based on the computing result of the first subtask obtained from the shared memory.
4. The method according to claim 3, characterized in that The method further comprises: If the computing efficiency of the first subtask on the second operator is lower than or equal to the computing efficiency on the first operator, wait until the first computing task is completed on the first operator, and then execute the second computing task on the first operator.
5. The method according to claim 1, characterized in that The first operator is used to perform a convolution operation, the second operator is used to perform a vector operation, and performing the second computing task on the second operator includes: Obtaining input data and convolution parameters of the second computing task; Converting the convolution parameters into a preset format; Based on the input data and the convolution parameters in a preset format, the second computing task is performed on the second operator, wherein output data of the second computing task performed on the second operator is the same as output data of the second computing task performed on the first operator.
6. The method according to claim 1, characterized in that When the preset condition is not met, or the operation efficiency of the second computing task on the second operator is lower than or equal to the operation efficiency on the first operator, the method further includes: After waiting for the first computing task to be completed on the first operator, the second computing task is executed on the first operator.
7. A multi-tasking processing device, characterized in that: The device comprises: An acquisition module, used to acquire a computing efficiency parameter of a second computing task on a first operator on the first operator and a second operator; a processing module, configured to execute the second computing task on the second operator when a preset condition is met and the computing efficiency of the second computing task on the second operator is higher than the computing efficiency on the first operator, The preset condition includes at least one of the following: a first computing task is being executed on the first operator, the execution order of the second computing task on the first operator is after the first computing task, and the second operator is in an idle state.
8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.
10. A chip, characterized in that: The method comprises one or more interface circuits and one or more processors; the interface circuit is used to receive code instructions and send the code instructions to the processor, and the processor is used to run the code instructions to execute the method according to any one of claims 1 to 6.
