A method, apparatus, electronic device, and storage medium for processing an algorithm model.

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for processing algorithm models. Each branchless algorithm model is parsed into a corresponding operator stream, and the priority of each operator stream is determined. In response to an execution instruction, a hardware acceleration module processes the hardware operators in a first operator stream and obtains a first processing state of the hardware acceleration module. A central processing unit (CPU) processes the software operators in a second operator stream and obtains a second processing state of the CPU. Based on the first and second processing states, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module, and another software operator is selected as the software operator to be processed by the CPU. The algorithm model processing method provided by this invention effectively shortens the computation time of the algorithm model, meets the real-time requirements of the algorithm model, and improves the resource utilization of artificial intelligence chips.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of big data processing technology, and in particular to a processing method, apparatus, electronic device and storage medium for an algorithm model. Background Technology

[0002] With the continuous development of artificial intelligence technology, more and more artificial intelligence technologies are being applied to various industries. The massive amount of data is one of the main characteristics of artificial intelligence technology. To achieve a highly real-time artificial intelligence algorithm model, it is necessary to solve the problem of massive data computation.

[0003] In related technologies, artificial intelligence chips assist the central processing unit (CPU) by setting up hardware acceleration modules to solve the problem of computing massive amounts of data for artificial intelligence algorithm models.

[0004] However, as the number of operators in AI chips increases, the number of operators that the hardware acceleration module cannot recognize and process also gradually increases. Moreover, the hardware acceleration module operates serially with the CPU, resulting in long computation times for AI algorithm models, which cannot meet the real-time requirements of AI algorithm models and wastes the resources of AI chips. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a processing method for an algorithm model that overcomes or at least partially solves the above problems.

[0006] In a first aspect, embodiments of this application disclose a method for processing an algorithm model, applied to an artificial intelligence chip, wherein the artificial intelligence chip includes a hardware acceleration module and a central processing unit, and the method includes:

[0007] Each branchless algorithm model is parsed into a corresponding operator stream, and the priority of each operator stream is determined; the operator stream includes hardware operators and software operators;

[0008] In response to the execution instruction, the hardware operators in the first operator stream are processed by the hardware acceleration module and a first processing state is obtained; and the software operators in the second operator stream are processed by the central processing unit and a second processing state is obtained.

[0009] If the first processing state is determined to be the hardware acceleration module processing completion state, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module according to the priority of the remaining operator streams other than the first operator stream and the second processing state.

[0010] If the second processing state is determined to be the state where the central processing unit has completed processing, another software operator is selected as the software operator to be processed by the central processing unit based on the priority of each operator stream and the first processing state.

[0011] Secondly, embodiments of this application disclose a processing device for an algorithm model, applied to an artificial intelligence chip, wherein the artificial intelligence chip includes a hardware acceleration module and a central processing unit, and the device includes:

[0012] The priority setting module is used to parse each of the sub-branchless algorithm models into a corresponding operator stream and determine the priority of each operator stream; the operator stream includes hardware operators and software operators;

[0013] The processing status acquisition module is used to respond to the execution instruction by processing the hardware operators in the first operator stream through the hardware acceleration module and acquiring the first processing status, and by processing the software operators in the second operator stream through the central processing unit and acquiring the second processing status.

[0014] The hardware operator processing module is used to select another hardware operator as the hardware operator to be processed by the hardware acceleration module when the first processing state is determined to be the processing completion state of the hardware acceleration module.

[0015] The software operator processing module is used to select another software operator as the software operator to be processed by the central processing unit when the second processing state is determined to be the central processing unit's processing completion state, based on the priority of each operator stream and the first processing state.

[0016] Thirdly, embodiments of this application also disclose an electronic device, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps of the processing method of the algorithm model as described in the first aspect.

[0017] Fourthly, embodiments of this application also disclose a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of the processing method of the algorithm model as described in the first aspect.

[0018] In this embodiment, each of the branchless algorithm models is parsed into corresponding operator streams, and the priority of each operator stream is determined. In response to an execution instruction, the hardware acceleration module processes the hardware operators in the first operator stream and obtains a first processing state. The central processing unit (CPU) processes the software operators in the second operator stream and obtains a second processing state. If the first processing state indicates that the hardware acceleration module has completed processing, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams (excluding the first operator stream) and the second processing state. If the second processing state indicates that the CPU has completed processing, another software operator is selected as the software operator to be processed by the CPU based on the priority of each operator stream and the first processing state. This solution effectively shortens the computation time of the algorithm model by scheduling the operator streams and performing parallel computation of hardware and software operators by the hardware acceleration module and the CPU, meeting the real-time requirements of the algorithm model and improving the resource utilization of the artificial intelligence chip. Attached Figure Description

[0019] Figure 1 This is a flowchart of the processing steps of an algorithm model provided in an embodiment of the present invention;

[0020] Figure 2 This is a structural diagram of an artificial intelligence chip provided in an embodiment of the present invention;

[0021] Figure 3 This is a deployment structure diagram of an algorithm model provided in an embodiment of the present invention;

[0022] Figure 4 This is a computational block diagram of a fully connected operator provided in an embodiment of the present invention;

[0023] Figure 5 This is a computational flowchart of a convolution operator provided in an embodiment of the present invention;

[0024] Figure 6 This is a flowchart of the calculation of a convolution operator after the transformation of a fully connected operator, provided in an embodiment of the present invention;

[0025] Figure 7 This is a block diagram of a processing device for an algorithm model provided in an embodiment of the present invention;

[0026] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present invention;

[0027] Figure 9 This is a block diagram of another electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0029] refer to Figure 1 This document illustrates a flowchart of the processing steps of an algorithm model provided in an embodiment of this application. The method is applied to an artificial intelligence chip, which includes a hardware acceleration module and a central processing unit. The method includes:

[0030] Step 101: Parse each branchless algorithm model of the algorithm model into a corresponding operator stream, and determine the priority of each operator stream.

[0031] In an embodiment of the present invention, Figure 2 A structural diagram of an artificial intelligence chip is shown. Figure 3 This is a deployment structure diagram of an algorithm model provided in an embodiment of the present invention. (Reference) Figure 2 and Figure 3 As shown, the artificial intelligence chip includes a hardware acceleration module 11 and a central processing unit 12, which are set up independently.

[0032] The hardware acceleration module 11 can handle computationally intensive tasks, thereby reducing the workload of the central processing unit. The hardware acceleration module 11 is also known as a hardware acceleration intellectual property core, or hardware acceleration IP (Intellectual Property) core. An IP core refers to an integrated circuit that is gradually separated from the chip design process, has been repeatedly verified, possesses specific functions, is reusable, and contains specific core elements. Hardware acceleration IP, on the other hand, is an integrated circuit that can implement hardware acceleration.

[0033] Furthermore, the Central Processing Unit (CPU) is one of the core components of a computer system, often referred to as the computer's brain, responsible for executing computer program instructions, performing data processing, and controlling operations.

[0034] In this embodiment of the invention, when the algorithm model is determined to be a branched structure algorithm model, a branching operation is performed on the branched structure algorithm model to obtain multiple branchless algorithm models.

[0035] Specifically, if the algorithm model is determined to be a branched algorithm model, continuous branching operations can be performed on the branched algorithm model by judging whether the condition is met, to obtain at least two sub-branch algorithm models. However, if the condition also exists in the sub-branch algorithm model, it is necessary to continue the branching operation on the non-branch algorithm model until a non-branch algorithm model is obtained.

[0036] In this embodiment of the invention, reference is made to Figure 2 and Figure 3 As shown, when the branchless algorithm model is deployed in an artificial intelligence chip, each branchless algorithm model needs to be parsed into a corresponding operator stream. The artificial intelligence chip then processes the operator stream by calling the hardware acceleration module 11 and the CPU 12. The operator stream includes hardware operators and software operators. Hardware operators are those computed using the hardware acceleration module 11, while software operators are those computed using the CPU 12. Therefore, hardware operators include, for example, convolution operators and pooling operators, while software operators include, for example, sampling operators and global uniform pooling operators.

[0037] Operators are the basic operational units in an algorithm model, and can be obtained by decomposing complex algorithm models.

[0038] Furthermore, after parsing each of the aforementioned branchless algorithm models into corresponding operator streams, the priority of each operator stream needs to be set according to the number of remaining operators in each operator stream. The operator stream with a larger number of remaining operators has a higher priority.

[0039] Optionally, step 101 specifically includes:

[0040] Sub-step 1011: Determine the hardware operator and the software operator.

[0041] In this embodiment of the invention, at least two consecutive operators that can be processed by the hardware acceleration module are considered as one hardware operator, non-consecutive operators that can be processed by the hardware acceleration module are considered as one hardware operator, and operators that can be processed by the central processing unit are considered as one software operator.

[0042] For example, there exists an operator stream in which the operators are, in order, operators that can be processed by the hardware acceleration module, operators that can be processed by the central processing unit (CPU), and operators that can be processed by the hardware acceleration module. Because in this operator stream, the first two operators are two consecutive operators that can be processed by the hardware acceleration module, forming one hardware operator; the CPU-processable operator is formed one software operator; and non-consecutive operators that can be processed by the hardware acceleration module are formed one hardware operator. Therefore, the hardware operators and software operators in this operator stream are, in order: hardware operator, software operator, and hardware operator.

[0043] Sub-step 1012: Parse each of the sub-branchless algorithm models into a corresponding operator stream, and obtain the remaining number of hardware operators and software operators in the operator stream.

[0044] In this embodiment of the invention, after each of the sub-branching algorithm models is parsed into a corresponding operator stream, the remaining number of operators in the operator stream is obtained by calculating the hardware operators and software operators in the operator stream.

[0045] For example, the operators in the first operator stream are ordered as hardware operators, software operators, software operators, and hardware operators in that order; the operators in the second operator stream are ordered as software operators, hardware operators, and software operators in that order; and the operators in the third operator stream are ordered as hardware operators and software operators in that order. Therefore, the number of remaining operators in the first operator stream is 4, the number of remaining operators in the second operator stream is 3, and the number of remaining operators in the third operator stream is 2.

[0046] Sub-step 1013: Set the priority of each operator stream according to the remaining number of operators.

[0047] In this embodiment of the invention, the priority of each operator stream is set according to the remaining operator streams of each operator stream. The operator stream with the largest number of remaining operators has the highest priority, and the operator stream with the smallest number of remaining operators has the lowest priority. Furthermore, since the priority of each operator stream is set based on the number of remaining operators, the priority of each operator stream changes when the number of remaining operators changes. When two operator streams have the same priority, their priorities are set according to the previously set priority. The priority of each operator stream is relative to the operator currently ranked first. When applying the priority of each operator stream, hardware operators are selected and invoked based on the priority of the operator stream whose currently ranked first operator is a hardware operator. Similarly, software operators are selected and invoked based on the priority of the operator stream whose currently ranked first operator is a software operator.

[0048] For example, the remaining operators in the first operator stream are ordered as hardware operators, software operators, software operators, and hardware operators; the remaining operators in the second operator stream are ordered as software operators, hardware operators, and software operators; and the remaining operators in the third operator stream are ordered as hardware operators and software operators. The first operator stream has 4 remaining operators, the second has 3, and the third has 2. Therefore, the priority of the first operator stream is 4, meaning the priority of the first hardware operator in the first operator stream is 4. The priority of the second operator stream is 3, meaning the priority of the first software operator in the first operator stream is 3. The priority of the third operator stream is 2, meaning the priority of the first hardware operator in the third operator stream is 2. Therefore, the initial priority of hardware operators called from highest to lowest is the first operator stream and then the third operator stream; initially, the first software operator in the second operator stream is called first.

[0049] Step 102: In response to the execution instruction, the hardware operators in the first operator stream are processed by the hardware acceleration module to obtain a first processing state, and the software operators in the second operator stream are processed by the central processing unit to obtain a second processing state.

[0050] In this embodiment of the invention, in response to an execution instruction, the hardware operators in the first operator stream are processed by a hardware acceleration module, and a first processing state of the hardware acceleration module is obtained. The first processing state includes a hardware acceleration module processing completion state and a hardware acceleration module processing incomplete state.

[0051] Furthermore, the software operators in the second operator stream are processed by the central processing unit (CPU), and a second processing state of the CPU is obtained. The second processing state includes a CPU processing completion state and a CPU processing incomplete state.

[0052] Step 103: If the first processing state is determined to be the hardware acceleration module processing completion state, then according to the priority of the remaining operator streams other than the first operator stream and the second processing state, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module.

[0053] In this embodiment of the invention, reference is made to Figure 3 As shown, when the first processing state is determined to be the hardware acceleration module's processing completion state, that is, when the hardware acceleration module completes the calculation of the current hardware operator, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams other than the first operator stream and the second processing state. The second processing state includes the central processing unit's processing completion state and the central processing unit's processing incomplete state.

[0054] Optionally, step 103 specifically includes:

[0055] Sub-step 1031: When the first processing state is the hardware acceleration module processing completion state and the second processing state is the central processing unit processing completion state, the other hardware operator is selected as the hardware operator to be processed by the hardware acceleration module according to the priority of the remaining operator streams other than the first operator stream.

[0056] In this embodiment of the invention, the computation time of the central processing unit (CPU) for software operators is much longer than the computation time of the hardware acceleration module for hardware operators. Furthermore, each sub-branchless algorithm model is parsed into a corresponding operator stream, which is computed serially. That is, the output of the operator ranked earlier in the operator stream becomes the input of the operator ranked later. While the operator ranked earlier has not been computed, the operator ranked later waits to be computed and is not allowed to be called or processed prematurely. Therefore, in the first processing state (when the hardware acceleration module has completed processing, i.e., when the hardware acceleration module has completed the computation of the current hardware operator) and in the second processing state (when the CPU has completed processing, i.e., when the CPU has completed the computation of the current software operator), since the next operator in the first operator stream must be a software operator, the other hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams other than the first operator stream.

[0057] For example, the remaining operators in the first operator stream are ordered as hardware operators, software operators, software operators, and hardware operators, respectively; the remaining operators in the second operator stream are ordered as software operators, hardware operators, and software operators, respectively; and the remaining operators in the third operator stream are ordered as hardware operators and software operators, respectively. Based on priority, the hardware acceleration module first calls the top-ranked hardware operator in the first operator stream for processing, and the central processing unit first calls the top-ranked software operator in the second operator stream for processing.

[0058] Furthermore, if the software operator is processed in advance, and it is also processed when the current hardware operator finishes processing, the remaining operators in the first operator stream are ordered as software operators, software operators, and hardware operators, respectively; the remaining operators in the second operator stream are ordered as hardware operators and software operators, respectively; and the remaining operators in the third operator stream are ordered as hardware operators and software operators, respectively. Therefore, the hardware acceleration module selects the hardware operators from the remaining operator streams (excluding the first operator stream, i.e., the second and third operator streams) based on their priorities, choosing the hardware operators from the second operator stream (which has the higher priority) as the hardware operators to be processed by the hardware acceleration module.

[0059] Sub-step 1032: If the first processing state is the hardware acceleration module processing completed state, and the second processing state is the central processing unit not processing completed state, then select the other hardware operator as the hardware operator to be processed by the hardware acceleration module according to the priority of the remaining operator streams other than the first operator stream and the second operator stream.

[0060] In this embodiment of the invention, when the first processing state is determined to be the hardware acceleration module's processing completion state, i.e., when the hardware acceleration module has completed the calculation of the current hardware operator, and when the second processing state is the central processing unit's processing incomplete state, i.e., when the central processing unit has not completed the calculation of the current software operator, the other hardware operator is selected as the hardware operator to be processed by the hardware acceleration module according to the priority of the remaining operator streams other than the first operator stream and the second operator stream.

[0061] For example, the remaining operators in the first operator stream are ordered as hardware operators, software operators, software operators, and hardware operators, respectively; the remaining operators in the second operator stream are ordered as software operators, hardware operators, and software operators, respectively; and the remaining operators in the third operator stream are ordered as hardware operators and software operators, respectively. Based on priority, the hardware acceleration module first calls the top-ranked hardware operator in the first operator stream for processing, and the central processing unit first calls the top-ranked software operator in the second operator stream for processing.

[0062] Furthermore, when the hardware acceleration module completes the calculation of the current hardware operator, because the next operator in the operator stream containing the current hardware operator must be a software operator, if the central processing unit (CPU) has not completed the calculation of the current software operator, because the operator stream containing the current software operator is a serial calculation, it cannot call the next sorted operator in the operator stream containing the current software operator. That is, it can only call the first sorted hardware operator in the remaining operator stream (the third operator stream, excluding the first and second operator streams) as the hardware operator to be processed by the hardware acceleration module.

[0063] Step 104: If the second processing state is determined to be the state where the central processing unit has completed processing, another software operator is selected as the software operator to be processed by the central processing unit according to the priority of each operator stream and the first processing state.

[0064] In this embodiment of the invention, when the second processing state is determined to be the central processing unit's (CPU) processing completion state, that is, when the CPU completes the processing of the current software operator, another software operator is selected as the CPU's pending software operator based on the priority of each operator stream and the first processing state. The first processing state includes the hardware acceleration module's processing completion state and the hardware acceleration module's processing incomplete state.

[0065] Optionally, step 104 specifically includes:

[0066] Sub-step 1041: If the second processing state is determined to be the state where the central processing unit has completed processing, and if the first processing state is determined to be the state where the hardware acceleration module has completed processing, then another software operator is selected as the software operator to be processed by the central processing unit according to the priority of each operator stream.

[0067] In this embodiment of the invention, the computation time of the central processing unit (CPU) for software operators is much longer than the computation time of the hardware acceleration module for hardware operators. Furthermore, each of the sub-branchless algorithm models is parsed into corresponding operator streams, which are computed sequentially. That is, the output of the operator ranked earlier in the operator stream becomes the input of the operator ranked later. While the operator ranked earlier has not been computed, the operator ranked later waits to be computed and is not allowed to be called or processed prematurely. Therefore, in the second processing state (CPU completion state), i.e., when the CPU has completed the computation of the current software operator, and in the first processing state (hardware acceleration module completion state), i.e., when the hardware acceleration module has completed the computation of the current software operator, the other software operator is selected as the software operator to be processed by the CPU based on the priority of each operator stream.

[0068] For example, the remaining operators in the first operator stream are ordered as hardware operators, software operators, software operators, and hardware operators, respectively; the remaining operators in the second operator stream are ordered as software operators, hardware operators, and software operators, respectively; and the remaining operators in the third operator stream are ordered as hardware operators and software operators, respectively. Based on priority, the hardware acceleration module first calls the top-ranked hardware operator in the first operator stream for processing, and the central processing unit first calls the top-ranked software operator in the second operator stream for processing.

[0069] Furthermore, if the software operator is processed in advance, and it is also processed when the current hardware operator finishes processing, the remaining operators in the first operator stream are ordered as software operator, software operator, and hardware operator, respectively; the remaining operators in the second operator stream are ordered as hardware operator and software operator, respectively; and the remaining operators in the third operator stream are ordered as hardware operator and software operator, respectively. Therefore, the CPU selects a software operator from the first operator stream as another operator, and treats this other operator as the software operator to be processed by the CPU.

[0070] Sub-step 1042: If the second processing state is determined to be the state where the central processing unit has completed processing, and if the first processing state is determined to be the state where the hardware acceleration module has not completed processing, then another software operator is selected as the software operator to be processed by the central processing unit based on the priority of the remaining operator streams other than the first operator stream.

[0071] In this embodiment of the invention, when the second processing state is the state where the central processing unit (CPU) has completed processing, i.e., when the CPU has completed the calculation of the current software operator, and when the first processing state is the state where the hardware acceleration module has not completed processing, i.e., when the hardware acceleration module has not completed the calculation of the current software operator, since the next operator in the second operator stream may be either a software operator or a hardware operator, the second operator stream also needs to be added to the remaining operator stream for selecting software operators. Therefore, based on the priority of the remaining operator streams other than the first operator stream, the other software operator is selected as the software operator to be processed by the CPU.

[0072] For example, the remaining operators in the first operator stream are ordered as hardware operators, software operators, software operators, and hardware operators, respectively; the remaining operators in the second operator stream are ordered as software operators, hardware operators, and software operators, respectively; and the remaining operators in the third operator stream are ordered as software operators and hardware operators, respectively. Based on priority, the hardware acceleration module first calls the top-ranked hardware operator in the first operator stream for processing, and the central processing unit first calls the top-ranked software operator in the second operator stream for processing.

[0073] Furthermore, if the software operator finishes processing, and the current hardware operator has not finished processing, the remaining operators in the first operator stream are ordered as software operator, software operator, and hardware operator, respectively; the remaining operators in the second operator stream are ordered as hardware operator and software operator, respectively; and the remaining operators in the third operator stream are ordered as software operator and hardware operator, respectively. Therefore, the CPU selects a software operator from the remaining operator stream (excluding the first operator stream, i.e., the third operator stream) as another operator, and treats this other operator as the software operator to be processed by the CPU.

[0074] Optionally, after step 103, the method further includes:

[0075] Step 105: Take the operator stream containing the other hardware operator as the first operator stream, and proceed to the step of processing the hardware operators in the first operator stream through the hardware acceleration module, until the hardware acceleration module has completed the calculation of all hardware operators.

[0076] In this embodiment of the invention, when the first processing state is determined to be the hardware acceleration module's processing completion state, i.e., when the hardware acceleration module completes the calculation of the current hardware operator, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams other than the first operator stream and the second processing state. The operator stream containing the other hardware operator is taken as the first operator stream and processed in step 102 until the hardware acceleration module completes the calculation of all hardware operators.

[0077] Optionally, after step 104, the method further includes:

[0078] Step 106: The operator stream containing the other software operator is taken as the second operator stream, and the process proceeds to the step of processing the software operators in the second operator stream by the central processing unit until the central processing unit has completed the calculation of all software operators.

[0079] In this embodiment of the invention, when the second processing state is determined to be the state where the central processing unit (CPU) has completed processing, another software operator is selected as the software operator to be processed by the CPU based on the priority of each operator stream and the first processing state. The operator stream containing the other software operator is designated as the second operator stream, and the process proceeds to step 102 until the CPU has completed the calculations for all software operators.

[0080] Optionally, prior to step 101, the method further includes:

[0081] Step 107: Convert the operators that are not recognizable by the hardware acceleration module in the algorithm model to obtain the operators that are recognizable by the hardware acceleration module.

[0082] In this embodiment of the invention, operators that are not recognizable by the hardware acceleration module in the algorithm model are converted to obtain operators that are recognizable by the hardware acceleration module.

[0083] Specifically, because the algorithm model contains a variety of operators that are not recognized by the hardware acceleration module, if the trained algorithm model is to be deployed on a self-developed artificial intelligence chip to obtain acceleration performance, the operators that are not recognized by the hardware acceleration module in the algorithm model must first be converted in order to obtain the operators that are recognized by the hardware acceleration module, and then the calculations can be performed in the hardware acceleration module to obtain acceleration performance.

[0084] Optionally, step 107 specifically includes:

[0085] Sub-step 1071: Transform the fully connected operator in the algorithm model to obtain the convolution operator.

[0086] In this embodiment of the invention, the fully connected operator in the algorithm model is transformed to obtain the convolution operator with a preset convolution kernel.

[0087] In an embodiment of the present invention, Figure 4 This is a computational flowchart of a fully connected operator provided in an embodiment of the present invention, referred to as follows. Figure 4 As shown, the calculation formula for the fully connected operator is as follows:

[0088]

[0089] Where y[c] is the c-th output of the fully connected operator, x i Here, n is the number of input parameters, and w is the number of input parameters. i[c] Here, is the weight, and bias[c] is the bias parameter.

[0090] Figure 5 This is a computational flowchart of a convolution operator provided in an embodiment of the present invention, referred to as follows. Figure 5 As shown, when the convolution operator's calculation mode is pad=0, meaning only valid window positions are used, and the valid mode does not perform padding, the calculation formula for the convolution operator is as follows:

[0091]

[0092] Where output is the output of the convolution operator, x i Here, n is the number of input parameters, and w is the number of input parameters. i Here, is the weight, and bias[c] is the bias parameter.

[0093] Figure 6 This is a flowchart illustrating the computation of a convolution operator after transformation by a fully connected operator, as provided in an embodiment of the present invention. Figure 6 As shown, the fully connected operator is transformed into a 3x3 convolution kernel convolution operator supported by the hardware acceleration module of the self-developed artificial intelligence chip. The calculation formula is as follows:

[0094]

[0095] Where w_(im[n]) represents the weight of the i-th input parameter among the n input parameters in the m-th channel, m = n / 9 rounded up, m represents the number of input parameter channels, output[n] is the output, bias[n] is the bias parameter, and n is the number of input parameters. And refer to... Figure 6 As shown, and referring to Figure 6 As shown, when the number of input parameters corresponding to the 3*3 convolution kernel is less than 9, the parameter positions corresponding to the convolution kernel can be filled with the number of zeros, where the number of zeros is 9*mn. It can be seen that the calculation results of the fully connected operator and the convolution operator are the same. Therefore, the convolution operator can be obtained by transforming the fully connected operator in the algorithm model, and the weights can be used as parameters of the 3*3 convolution kernel to apply the 3*3 convolution kernel to the input parameters.

[0096] For example, if n is 3, meaning there are 3 input parameters, then m = 1, and the number of zeros to fill in is 9 * mn = 6. This corresponds to a 3x3 convolution kernel. In a 3x3 queue, the three input parameters are placed in the first row, and then the second and third rows are filled with 6 zeros. The convolution kernel is then applied to process the 3x3 queue.

[0097] Optionally, before step 101, the method further includes:

[0098] Step 108: If the algorithm model is determined to be a branched structure algorithm model, perform a branching operation on the branched structure algorithm model to obtain multiple branchless algorithm models.

[0099] In this embodiment of the invention, the presence or absence of a judgment condition in the algorithm model can be used to determine whether the algorithm model is a branched or non-branched algorithm model. If a judgment condition exists in the algorithm model, it can be determined that the algorithm model is a branched algorithm model; if no judgment condition exists in the algorithm model, it can be determined that the algorithm model is a non-branched algorithm model.

[0100] Specifically, when the algorithm model is determined to be a branched algorithm model, by judging whether a condition is true or false, the branched algorithm model can be continuously branched to obtain multiple childless branched algorithm models. For example, the judgment condition is whether the input parameter n of this expression is greater than m. When the input parameter n is greater than m, this judgment condition is true, and the sub-branched algorithm model corresponding to the true judgment condition can be obtained. When the input parameter n is less than or equal to m, this judgment condition is false, and the sub-branched algorithm model whose judgment condition is false can be obtained. At this time, the branched algorithm model can be branched into two sub-branched algorithm models according to the judgment condition. However, when the judgment condition also exists in these two sub-branched algorithm models, it is necessary to continue the branching operation on the childless branched algorithm model until a childless branched algorithm model is obtained.

[0101] Step 109: If the algorithm model is determined to be a branchless structure algorithm model, the branchless structure algorithm model is parsed into a branchless operator stream, and the hardware operators and software operators in the branchless operator stream are determined according to the computational complexity of the operators.

[0102] If the algorithm model is determined to be a branchless structure algorithm model, the branchless structure algorithm model is parsed into a branchless operator stream, and the hardware operators and software operators in the branchless operator stream are determined according to the computational complexity of the operators.

[0103] In this embodiment of the invention, the presence or absence of a judgment condition in the algorithm model is used to determine whether the algorithm model is a branched or non-branched algorithm model. If no judgment condition exists in the algorithm model, it can be determined that the algorithm model is a non-branched algorithm model. If the algorithm model is determined to be a non-branched algorithm model, it is parsed into a non-branched operator stream, and the hardware and software operators in the non-branched operator stream are determined based on the computational complexity of the operators.

[0104] Here, the complexity of an operator refers to the time and memory resources required for its execution. A preset complexity can be used; when the computational complexity of an operator is greater than the preset complexity, it is considered a hardware operator; when the computational complexity is less than or equal to the preset complexity, it is considered a software operator. Multiple consecutive hardware operators can also be considered as a single hardware operator, or multiple consecutive hardware operators can be considered as multiple hardware operators. Because the branchless operator stream is computed serially, there are no restrictions on the combination of consecutive hardware operators.

[0105] Step 110: Based on the operator order in the branchless operator stream, call the hardware acceleration module to process the hardware operators in the branchless operator stream or call the central processing unit to process the software operators in the branchless operator stream.

[0106] In this embodiment of the invention, the hardware acceleration module is invoked to process the hardware operators in the branchless operator stream according to the operator order in the branchless operator stream, or the central processing unit is invoked to process the software operators in the branchless operator stream.

[0107] In this embodiment of the invention, when the operator in the branchless operator stream is a hardware operator, the hardware acceleration module is invoked to process the hardware operator; when the operator in the branchless operator stream is a software operator, the central processing unit (CPU) is invoked to process the software operator. Because the operators in the branchless operator stream are computed serially, when the hardware acceleration module is invoked to process a hardware operator or the CPU is invoked to process a software operator, other operators wait for the output result of the current hardware or software operator before they can be processed. That is, in the serial computation method, the next operator is processed only after the current operator has finished processing, and the processing result of the current operator is the input parameter of the next operator.

[0108] Optionally, after step 103, the method further includes:

[0109] Step 111: Call the hardware acceleration module to process the current hardware operator and obtain the processing result of the current hardware operator.

[0110] In this embodiment of the invention, the current hardware operator is processed by calling the hardware acceleration module, the processing result of the current hardware operator is obtained, and the processing of the current hardware operator is cached in the allocated buffer.

[0111] Step 112: Use the processing result of the current hardware operator as the input of the next software operator in the same operator stream until all hardware operators have been processed, obtain the processing result of the hardware acceleration module and cache it.

[0112] In this embodiment of the invention, when the hardware acceleration module processes the current hardware operator, obtains the processing result of the current hardware operator, and caches it, when the next software operator in the same operator stream is processed by the hardware acceleration module, the cached processing result of the current hardware operator is used as the input of the next software operator in the same operator stream. This process continues until all hardware operators have been processed, at which point the processing result of the hardware acceleration module is obtained and cached.

[0113] Optionally, after step 104, the method includes:

[0114] Step 113: Call the software acceleration module to process the current software operator and obtain the processing result of the current software operator.

[0115] In this embodiment of the invention, the current software operator is processed by calling the central processing unit to obtain the processing result of the current software operator, and the processing of the current software operator is cached in an allocated buffer.

[0116] Step 114: Use the processing result of the current software operator as the input of the next operator in the same operator stream until all the software operators have been processed, and obtain the processing result of the central processing unit and cache it.

[0117] In this embodiment of the invention, when the central processing unit (CPU) processes the current software operator, obtains the processing result of the current software operator, and caches it, when the next operator in the same operator stream is processed by the CPU, the cached processing result of the current software operator is used as the input of the next operator in the same operator stream. This process continues until all the software operators have been processed, at which point the CPU's processing result is obtained and cached. Alternatively, the CPU may, according to the needs of the artificial intelligence chip, call the cached processing result of the software acceleration module for processing.

[0118] In summary, in this embodiment, each branchless algorithm model is parsed into a corresponding operator stream, and the priority of each operator stream is determined. In response to an execution instruction, the hardware acceleration module processes the hardware operators in the first operator stream and obtains its first processing state; the central processing unit (CPU) processes the software operators in the second operator stream and obtains its second processing state. If the first processing state indicates that the hardware acceleration module has completed processing, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams (excluding the first operator stream) and the second processing state. If the second processing state indicates that the CPU has completed processing, another software operator is selected as the software operator to be processed by the CPU based on the priority of each operator stream and the first processing state. This solution effectively shortens the computation time of the algorithm model by scheduling the operator streams and performing parallel computation of hardware and software operators by the hardware acceleration module and the CPU, meeting the real-time requirements of the algorithm model and improving the resource utilization of the artificial intelligence chip.

[0119] refer to Figure 7This diagram illustrates a block diagram of a processing device for an algorithm model provided in an embodiment of this application, applied to an artificial intelligence chip, wherein the artificial intelligence chip includes a hardware acceleration module and a central processing unit, and the device 400 includes:

[0120] The priority setting module 401 is used to parse each branchless algorithm model of the algorithm model into a corresponding operator stream and determine the priority of each operator stream; the operator stream includes hardware operators and software operators;

[0121] The processing status acquisition module 402 is used to respond to the execution instruction by processing the hardware operators in the first operator stream through the hardware acceleration module and acquiring the first processing status, and by processing the software operators in the second operator stream through the central processing unit and acquiring the second processing status.

[0122] The hardware operator processing module 403 is used to select another hardware operator as the hardware operator to be processed by the hardware acceleration module when the first processing state is determined to be the processing completion state of the hardware acceleration module.

[0123] The software operator processing module 404 is used to select another software operator as the software operator to be processed by the central processing unit when the second processing state is determined to be the central processing unit's processing completion state, based on the priority of each operator stream and the first processing state.

[0124] Optionally, the priority setting module 401 includes:

[0125] The operator determination submodule is used to determine the hardware operator and the software operator;

[0126] The operator count acquisition submodule is used to parse each of the sub-branchless algorithm models into a corresponding operator stream, and obtain the remaining operator count of the hardware operators and the software operators in the operator stream;

[0127] The priority setting submodule is used to set the priority of each operator stream based on the remaining number of operators.

[0128] Optionally, the hardware operator processing module 403 includes:

[0129] The first hardware operator selection submodule is used to select another hardware operator as the hardware operator to be processed by the hardware acceleration module when the first processing state is the hardware acceleration module processing completion state and the second processing state is the central processing unit processing completion state, according to the priority of the remaining operator streams other than the first operator stream.

[0130] The second hardware operator selection submodule is used to select another hardware operator as the hardware operator to be processed by the hardware acceleration module when the first processing state is the hardware acceleration module processing completed state and the second processing state is the central processing unit not processing completed state, based on the priority of the remaining operator streams other than the first operator stream and the second operator stream.

[0131] Optionally, the software operator processing module 404 includes:

[0132] The first software operator selection submodule is used to select another software operator as the software operator to be processed by the central processing unit when the second processing state is determined to be the central processing unit processing completed state and the first processing state is determined to be the hardware acceleration module processing completed state, according to the priority of each operator stream.

[0133] The second software operator selection submodule is used to select another software operator as the software operator to be processed by the central processing unit when the second processing state is the state where the central processing unit has completed processing and when the first processing state is the state where the hardware acceleration module has not completed processing, based on the priority of the remaining operator streams other than the first operator stream.

[0134] Optionally, the device 400 further includes:

[0135] The first loop module is used to take the operator stream containing the other hardware operator as the first operator stream and enter the step of processing the hardware operators in the first operator stream by the hardware acceleration module until the hardware acceleration module has completed the calculation of all hardware operators.

[0136] Optionally, the device 400 further includes:

[0137] The second loop module is used to take the operator stream containing the other software operator as the second operator stream and enter the step of processing the software operators in the second operator stream by the central processing unit until the central processing unit has completed the calculation of all software operators.

[0138] Optionally, the device 400 further includes:

[0139] The operator conversion module is used to convert operators that are not recognizable by the hardware acceleration module in the algorithm model to obtain operators that are recognizable by the hardware acceleration module.

[0140] Optionally, the operator transformation module further includes:

[0141] The fully connected operator transformation submodule is used to transform the fully connected operators in the algorithm model to obtain the convolution operator.

[0142] Optionally, the device 400 further includes:

[0143] The branching operation module is used to perform branching operations on the algorithm model when the algorithm model is determined to be a branched structure algorithm model, so as to obtain multiple branchless algorithm models.

[0144] Optionally, the device 400 further includes:

[0145] The branchless operator stream acquisition module is used to parse the branchless structure algorithm model into a branchless operator stream when it is determined that the algorithm model is a branchless structure algorithm model, and to determine the hardware operators and software operators in the branchless operator stream according to the computational complexity of the operators.

[0146] The branchless operator stream processing module is used to call the hardware acceleration module to process the hardware operators in the branchless operator stream or call the central processing unit to process the software operators in the branchless operator stream according to the operator order in the branchless operator stream.

[0147] Optionally, the device 400 further includes:

[0148] The first processing result acquisition module is used to call the hardware acceleration module to process the current hardware operator and obtain the processing result of the current hardware operator;

[0149] The second processing result acquisition module is used to take the processing result of the current hardware operator as the input of the next software operator in the same operator stream, until all the hardware operators are processed, and to acquire and cache the processing result of the hardware acceleration module.

[0150] Optionally, the device 400 further includes:

[0151] The third processing result acquisition module is used to call the software acceleration module to process the current software operator and obtain the processing result of the current software operator;

[0152] The fourth processing result acquisition module is used to take the processing result of the current software operator as the input of the next operator in the same operator stream until all the software operators are processed, and to acquire and cache the processing result of the central processing unit.

[0153] In summary, in this embodiment, each branchless algorithm model is parsed into a corresponding operator stream, and the priority of each operator stream is determined. In response to an execution instruction, the hardware acceleration module processes the hardware operators in the first operator stream and obtains its first processing state; the central processing unit (CPU) processes the software operators in the second operator stream and obtains its second processing state. If the first processing state indicates that the hardware acceleration module has completed processing, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams (excluding the first operator stream) and the second processing state. If the second processing state indicates that the CPU has completed processing, another software operator is selected as the software operator to be processed by the CPU based on the priority of each operator stream and the first processing state. This solution effectively shortens the computation time of the algorithm model by scheduling the operator streams and performing parallel computation of hardware and software operators by the hardware acceleration module and the CPU, meeting the real-time requirements of the algorithm model and improving the resource utilization of the artificial intelligence chip. Furthermore, by converting operators that the hardware acceleration module in the algorithm model cannot recognize and process, the computation time of the algorithm model is effectively improved, giving full play to the performance of the artificial intelligence chip.

[0154] Figure 8 A block diagram of an electronic device 600 is shown according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0155] Reference Figure 8 The sub-device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input / output (I / O) interface 612, sensor component 614, and communication component 616.

[0156] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0157] Memory 604 is used to store various types of data to support the operation of electronic device 600. Examples of such data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0158] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0159] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a multimedia mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0160] Audio component 610 is used to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) used to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0161] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0162] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0163] Communication component 616 facilitates wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0164] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement the processing method of the algorithm model provided in the embodiments of this application.

[0165] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0166] Figure 9A block diagram of an electronic device 700 is shown according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. (Refer to...) Figure 9 The electronic device 700 includes a processing component 722, which further includes one or more processors, and memory resources represented by a memory 732 for storing instructions, such as application programs, that can be executed by the processing component 722. The application programs stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 722 is configured to execute instructions to perform a processing method of an algorithmic model provided in embodiments of this application.

[0167] Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 758. Electronic device 700 may operate on an operating system stored in memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0168] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0169] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for processing an algorithm model, applied to an artificial intelligence chip, wherein the artificial intelligence chip includes a hardware acceleration module and a central processing unit, characterized in that, The method includes: Each branchless algorithm model is parsed into a corresponding operator stream, and the priority of each operator stream is determined; the operator stream includes hardware operators and software operators; In response to the execution instruction, the hardware operators in the first operator stream are processed by the hardware acceleration module and a first processing state is obtained; and the software operators in the second operator stream are processed by the central processing unit and a second processing state is obtained. If the first processing state is determined to be the hardware acceleration module processing completion state, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module according to the priority of the remaining operator streams other than the first operator stream and the second processing state. If the second processing state is determined to be the state where the central processing unit has completed processing, another software operator is selected as the software operator to be processed by the central processing unit based on the priority of each operator stream and the first processing state.

2. The method according to claim 1, characterized in that, When the first processing state is determined to be the hardware acceleration module's processing completion state, after selecting another hardware operator as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams other than the first operator stream and the second processing state, the method further includes: The operator stream containing the other hardware operator is taken as the first operator stream, and the process proceeds to the step of processing the hardware operators in the first operator stream by the hardware acceleration module until the hardware acceleration module has completed the calculation of all the hardware operators. When the second processing state is determined to be the central processing unit's processing completion state, after selecting another software operator as the software operator to be processed by the central processing unit based on the priority of each operator stream and the first processing state, the method further includes: The operator stream containing the other software operator is taken as the second operator stream, and the process proceeds to the step of processing the software operators in the second operator stream by the central processing unit until the central processing unit has completed the calculation of all the software operators.

3. The method according to claim 1, characterized in that, Before parsing each branchless algorithm model into a corresponding operator stream and determining the priority of each operator stream, the method further includes: The operators that are not recognizable by the hardware acceleration module in the algorithm model are converted to obtain the operators that are recognizable by the hardware acceleration module. The operators that the hardware acceleration module cannot recognize include fully connected operators, and the operators that the hardware acceleration module can recognize include convolution operators. The step of converting the hardware acceleration module-unrecognizable operators in the algorithm model to obtain the hardware acceleration module-recognizable operators includes: The fully connected operator in the algorithm model is transformed to obtain the convolution operator.

4. The method according to claim 1, characterized in that, Before parsing each branchless algorithm model into a corresponding operator stream and determining the priority of each operator stream, the method further includes: If the algorithm model is determined to be a branched structure algorithm model, a branching operation is performed on the branched structure algorithm model to obtain multiple childless branchless algorithm models. If the algorithm model is determined to be a branchless structure algorithm model, the branchless structure algorithm model is parsed into a branchless operator stream, and the hardware operators and software operators in the branchless operator stream are determined according to the computational complexity of the operators. Based on the operator order in the branchless operator stream, the hardware acceleration module is invoked to process the hardware operators in the branchless operator stream, or the central processing unit is invoked to process the software operators in the branchless operator stream.

5. The method according to claim 1, characterized in that, The step of parsing each branchless algorithm model into a corresponding operator stream and determining the priority of each operator stream includes: Determine the hardware operator and the software operator; Each of the sub-branching algorithm models is parsed into a corresponding operator stream, and the remaining number of hardware operators and software operators in the operator stream is obtained; The priority of each operator stream is set based on the remaining number of operators.

6. The method according to claim 1, characterized in that, The second processing state includes a central processing unit (CPU) processing completion state and a CPU processing incomplete state. When the first processing state is determined to be the hardware acceleration module processing completion state, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams other than the first operator stream and the second processing state, including: When the first processing state is determined to be the hardware acceleration module processing completion state, and the second processing state is the central processing unit processing completion state, the other hardware operator is selected as the hardware operator to be processed by the hardware acceleration module according to the priority of the remaining operator streams other than the first operator stream. If the first processing state is determined to be the hardware acceleration module's processing completed state, and the second processing state is the central processing unit's processing not completed state, then according to the priority of the remaining operator streams other than the first operator stream and the second operator stream, the other hardware operator is selected as the hardware operator to be processed by the hardware acceleration module.

7. The method according to claim 1, characterized in that, The first processing state includes a hardware acceleration module processing completion state and a hardware acceleration module processing incomplete state. When the second processing state is determined to be the central processing unit processing completion state, selecting another software operator as the software operator to be processed by the central processing unit based on the priority of each operator stream and the first processing state includes: If the second processing state is determined to be the state where the central processing unit has completed processing, and if the first processing state is determined to be the state where the hardware acceleration module has completed processing, then another software operator is selected as the software operator to be processed by the central processing unit according to the priority of each operator stream. If the second processing state is determined to be the state where the central processing unit has completed processing, and if the first processing state is determined to be the state where the hardware acceleration module has not completed processing, then another software operator is selected as the software operator to be processed by the central processing unit based on the priority of the remaining operator streams other than the first operator stream.

8. The method according to claim 1, characterized in that, When the first processing state is determined to be the hardware acceleration module's processing completion state, another hardware operator is selected as the hardware operator to be processed by the hardware acceleration module based on the priority of the remaining operator streams other than the first operator stream and the second processing state, including: The hardware acceleration module is invoked to process the current hardware operator, and the processing result of the current hardware operator is obtained; The processing result of the current hardware operator is used as the input of the next software operator in the same operator stream until all hardware operators are processed. The processing result of the hardware acceleration module is then obtained and cached. When the second processing state is determined to be the central processing unit's processing completion state, another software operator is selected as the software operator to be processed by the central processing unit based on the priority of each operator stream and the first processing state, including: The software acceleration module is invoked to process the current software operator, and the processing result of the current software operator is obtained; The processing result of the current software operator is used as the input of the next operator in the same operator stream until all the software operators have been processed. The processing result of the central processing unit is then obtained and cached.

9. A processing device for an algorithm model, applied to an artificial intelligence chip, wherein the artificial intelligence chip includes a hardware acceleration module and a central processing unit, characterized in that, The device includes: The priority setting module is used to parse each branchless algorithm model into a corresponding operator stream and determine the priority of each operator stream; the operator stream includes hardware operators and software operators; The processing status acquisition module is used to respond to the execution instruction by processing the hardware operators in the first operator stream through the hardware acceleration module and acquiring the first processing status, and by processing the software operators in the second operator stream through the central processing unit and acquiring the second processing status. The hardware operator processing module is used to select another hardware operator as the hardware operator to be processed by the hardware acceleration module when the first processing state is determined to be the processing completion state of the hardware acceleration module. The software operator processing module is used to select another software operator as the software operator to be processed by the central processing unit when the second processing state is determined to be the central processing unit's processing completion state, based on the priority of each operator stream and the first processing state.

10. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps of the processing method of the algorithm model as described in any one of claims 1 to 8.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the processing method of the algorithm model as described in any one of claims 1 to 8.

Images