Model operation method and apparatus, and computing device

Abstract

Embodiments of the present application provide a model operation method and device and a computing device. The method is applied to a computing device, the computing device comprising at least one processing core set and a shared memory corresponding to each processing core set, the processing core set comprising at least two processing cores, and the processing cores in the processing core set sharing the corresponding shared memory. The method comprises: determining a plurality of processing operators in a target model and an operator priority corresponding to each processing core; determining, for any one multiply-accumulate (MAC) operation, core-out input data corresponding to the processing core set, and transmitting the core-out input data to the shared memory corresponding to the processing core set; and for any one processing core, controlling the processing core to perform operator operation according to data in the shared memory corresponding to the processing core set, a dependency relationship between the plurality of processing operators, and the operator priority corresponding to the processing core, to obtain an intermediate operation result and store the intermediate operation result to the shared memory. The efficiency of model operation is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model operation method, apparatus and computing device. Background Technology

[0002] The output of a neural network model can be obtained by running the processing operators corresponding to multiple algorithms in the neural network model through the processing core in a computing device (e.g., an artificial intelligence processor).

[0003] In related technologies, model computation can be performed as follows: The computing device includes at least one processing core, each processing core corresponding to an on-core memory, and each processing core runs the corresponding processing operator in the neural network model. The input data of the neural network model is stored in Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM). The processing operator and processing core for processing the input data are determined, thereby determining the on-core memory for storing the input data. Input data is retrieved from DDR via Direct Memory Access (DMA) and stored in the corresponding on-core memory. The processing core reads the input data from the corresponding on-core memory and runs the corresponding processing operator to perform multiply-accumulate (MAC) calculations on the input data, obtaining intermediate computation results, and storing the intermediate computation results in the corresponding on-core memory.

[0004] In the above process, each processing core, when running a processing operator to perform MAC calculations, can only read input data or store intermediate calculation results in its corresponding core memory. If a processing core needs intermediate calculation results from other processing cores for MAC calculations, it needs to read the intermediate calculation results from the core memory of those other processing cores and store them in its own core memory before performing the MAC calculation. During this process, the computing device needs to perform multiple read and store operations, resulting in low efficiency in model computation. Summary of the Invention

[0005] This application provides a model calculation method, apparatus, and computing device to solve the problem of low efficiency in model calculation.

[0006] In a first aspect, embodiments of this application provide a model computation method applied in a computing device, the computing device including at least one set of processing cores and shared memory corresponding to each set of processing cores, the set of processing cores including at least two processing cores, the processing cores in the set of processing cores sharing the corresponding shared memory, the method including:

[0007] Determine multiple processing operators in the target model, and the operator priority corresponding to each processing core, wherein the operator priority includes the priority of each processing operator;

[0008] For any multiply-accumulate MAC calculation, determine the off-core input data corresponding to the processing core set, and transfer the off-core input data to the shared memory corresponding to the processing core set;

[0009] For any processing core in any set of processing cores, based on the data in the shared memory corresponding to the set of processing cores, the dependencies between the multiple processing operators, and the operator priority corresponding to the processing core, the processing core is controlled to perform operator operations to obtain intermediate operation results, and the intermediate operation results are stored in the shared memory.

[0010] In one possible implementation, based on the data in the shared memory corresponding to the processing core set, the dependencies between the multiple processing operators, and the operator priorities corresponding to the processing cores, the processing cores are controlled to perform operator operations to obtain intermediate operation results, including:

[0011] Determine the data type of each piece of data in the shared memory, wherein the data type is an off-core input type or an intermediate operation result type;

[0012] Based on the data type of each data in the shared memory, the dependency relationship, and the operator priority corresponding to the processing core, a target processing operator is determined from the plurality of processing operators, and the operation data corresponding to the target processing operator is determined in the shared memory;

[0013] Based on the target processing operator, the processing kernel performs calculations on the computational data to obtain the intermediate computational result.

[0014] In one possible implementation, based on the data types of the data in the shared memory, the dependencies, and the operator priorities corresponding to the processing core, a target processing operator is determined from the plurality of processing operators, and the computational data corresponding to the target processing operator is determined in the shared memory, including:

[0015] Based on the operator priority corresponding to the processing kernel, determine the highest priority operator corresponding to the processing kernel;

[0016] Determine whether the highest priority operator has a dependent operator based on the dependency relationship;

[0017] If so, the target processing operator and the computational data corresponding to the target processing operator are determined based on the data type of each data in the shared memory, the dependency relationship, and the operator priority corresponding to the processing core;

[0018] If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

[0019] In one possible implementation, the target processing operator and the corresponding computational data are determined based on the data types of the data in the shared memory, the dependencies, and the operator priority corresponding to the processing core, including:

[0020] Determine whether intermediate operation result type intermediate operation data exists in the shared memory;

[0021] If so, then determine the processing operator corresponding to each intermediate operation data, and determine the target processing operator and the operation data corresponding to the target processing operator based on the processing operator corresponding to each intermediate operation data, the dependency relationship, and the operator priority corresponding to the processing kernel. The processing operator corresponding to the intermediate operation data is the operator that calculates the intermediate operation data.

[0022] If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

[0023] In one possible implementation, the target processing operator and the corresponding computational data are determined based on the processing operators corresponding to each intermediate computational data, the dependencies, and the operator priority corresponding to the processing kernel, including:

[0024] The number of intermediate operation data corresponding to each processing operator is determined based on the processing operator corresponding to each intermediate operation data.

[0025] Determine the kernel side length N of the target model convolution kernel, where N is an integer greater than or equal to 1;

[0026] The processing operators are sorted in descending order of priority to obtain the processing order.

[0027] According to the processing order, based on the dependency relationship, the number of intermediate operation data corresponding to each processing operator, and the size of the convolution kernel, it is determined in turn whether the processing operator is the target processing operator, until the target processing operator is determined among the multiple processing operators;

[0028] The computational data corresponding to the target processing operator is determined from the data in the shared memory.

[0029] In one possible implementation, based on the dependency relationship, the amount of intermediate computation data corresponding to each processing operator, and the convolution kernel size, it is sequentially determined whether the processing operator is the target processing operator, until the target processing operator is determined among the plurality of processing operators, including:

[0030] Based on the dependency relationship, determine the dependent operator corresponding to the i-th processing operator. If the number of intermediate operation data corresponding to the dependent operator is greater than or equal to N, then the processing operator is determined as the target processing operator.

[0031] Wherein, i takes the values ​​1, 2, ..., until the target processing operator is determined, i is an integer less than or equal to M, and M is the number of the plurality of processing operators.

[0032] In one possible implementation, the method further includes, prior to performing any MAC calculation:

[0033] For any given processing core, multiple sub-memories corresponding to the processing core are determined. The memory capacity of the multiple sub-memories is the same, and the sum of the capacities of the multiple sub-memories is the memory capacity of the shared memory.

[0034] Determine the computational cost of each processing operator executed by the processing core;

[0035] The number of sub-memory corresponding to the processing core is determined based on the computational amount of each processing operator run by the processing core;

[0036] Based on the number of sub-memory corresponding to the processing core, memory is allocated for the processing core in the shared memory corresponding to the processing core.

[0037] In one possible implementation, the method further includes:

[0038] After calculating the target operation result corresponding to the target model, the target operation result is output to a preset memory.

[0039] Secondly, embodiments of this application provide a model computation apparatus, the apparatus comprising:

[0040] The first determining module is used to determine multiple processing operators in the target model, and the operator priority corresponding to each processing core, wherein the operator priority includes the priority of each processing operator;

[0041] The second determining module is used to determine the input data corresponding to the processing core set for any multiply-accumulate MAC calculation, and to transmit the input data to the shared memory corresponding to the processing core set.

[0042] The control module is used to determine the intermediate operation results existing in the shared memory and the operator priority corresponding to each processing core, control the processing core to perform operator operations, obtain the latest intermediate operation results, and store the latest intermediate operation results in the shared memory.

[0043] In one possible implementation, the control module is specifically used for:

[0044] Determine the data type of each piece of data in the shared memory, wherein the data type is an off-core input type or an intermediate operation result type;

[0045] Based on the data type of each data in the shared memory, the dependency relationship, and the operator priority corresponding to the processing core, a target processing operator is determined from the plurality of processing operators, and the operation data corresponding to the target processing operator is determined in the shared memory;

[0046] Based on the target processing operator, the processing kernel performs calculations on the computational data to obtain the intermediate computational result.

[0047] In one possible implementation, the control module is specifically used for:

[0048] Based on the operator priority corresponding to the processing kernel, determine the highest priority operator corresponding to the processing kernel;

[0049] Determine whether the highest priority operator has a dependent operator based on the dependency relationship;

[0050] If so, the target processing operator and the computational data corresponding to the target processing operator are determined based on the data type of each data in the shared memory, the dependency relationship, and the operator priority corresponding to the processing core;

[0051] If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

[0052] In one possible implementation, the control module is specifically used for:

[0053] Determine whether intermediate operation result type intermediate operation data exists in the shared memory;

[0054] If so, then determine the processing operator corresponding to each intermediate operation data, and determine the target processing operator and the operation data corresponding to the target processing operator based on the processing operator corresponding to each intermediate operation data, the dependency relationship, and the operator priority corresponding to the processing kernel. The processing operator corresponding to the intermediate operation data is the operator that calculates the intermediate operation data.

[0055] If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

[0056] In one possible implementation, the control module is specifically used for:

[0057] The number of intermediate operation data corresponding to each processing operator is determined based on the processing operator corresponding to each intermediate operation data.

[0058] Determine the kernel side length N of the target model convolution kernel, where N is an integer greater than or equal to 1;

[0059] The processing operators are sorted in descending order of priority to obtain the processing order.

[0060] According to the processing order, based on the dependency relationship, the number of intermediate operation data corresponding to each processing operator, and the size of the convolution kernel, it is determined in turn whether the processing operator is the target processing operator, until the target processing operator is determined among the multiple processing operators;

[0061] The computational data corresponding to the target processing operator is determined from the data in the shared memory.

[0062] In one possible implementation, the control module is specifically used for:

[0063] Based on the dependency relationship, determine the dependent operator corresponding to the i-th processing operator. If the number of intermediate operation data corresponding to the dependent operator is greater than or equal to N, then the processing operator is determined as the target processing operator.

[0064] Wherein, i takes the values ​​1, 2, ..., until the target processing operator is determined, i is an integer less than or equal to M, and M is the number of the plurality of processing operators.

[0065] In one possible implementation, the device further includes a distribution module and an output module.

[0066] The allocation module is used for:

[0067] For any given processing core, multiple sub-memories corresponding to the processing core are determined. The memory capacity of the multiple sub-memories is the same, and the sum of the capacities of the multiple sub-memories is the memory capacity of the shared memory.

[0068] Determine the computational cost of each processing operator executed by the processing core;

[0069] The number of sub-memory corresponding to the processing core is determined based on the computational amount of each processing operator run by the processing core;

[0070] Based on the number of sub-memory corresponding to the processing core, memory is allocated for the processing core in the shared memory corresponding to the processing core.

[0071] The output module is used for:

[0072] After calculating the target operation result corresponding to the target model, the target operation result is output to a preset memory.

[0073] Thirdly, this application provides a chip on which a computer program is stored, and when the computer program is executed by the chip, it implements the method as described in any of the first aspects.

[0074] Fourthly, this application provides a chip module on which a computer program is stored, and when the computer program is executed by the chip module, it implements the method described in any of the first aspects.

[0075] Fifthly, embodiments of this application provide a computing device, including:

[0076] At least one processor; and

[0077] A memory communicatively connected to the at least one processor; wherein,

[0078] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in any of the first aspects.

[0079] In a sixth aspect, embodiments of this application provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the first aspects.

[0080] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any one of the first aspects.

[0081] The model operation method, apparatus, and computing device provided in this application divide the processing cores in the computing device into at least one set of processing cores, with each set of processing cores corresponding to a shared memory. Each set of processing cores includes at least two processing cores, and the processing cores in the set share the corresponding shared memory. Multiple processing operators in the target model and the operator priority corresponding to each processing core are determined. For any processing core in any set of processing cores, based on the data in the shared memory corresponding to the set of processing cores, the dependencies between multiple processing operators, and the operator priority corresponding to the processing core, the processing core is controlled to perform operator operations to obtain intermediate operation results, which are then stored in the shared memory. In the above process, after controlling the processing core to perform operator operations and obtaining the latest intermediate operation result, the latest intermediate operation result is stored in the shared memory. This allows the processing core to directly obtain the intermediate operation result from the shared memory when performing the next MAC calculation, avoiding reading the intermediate operation result from the core memory corresponding to other processing cores, reducing the number of read and storage operations of the computing device, and improving the efficiency of model operation. Attached Figure Description

[0082] Figure 1 A schematic diagram illustrating the application scenarios provided in the embodiments of this application;

[0083] Figure 2 A flowchart illustrating a model calculation method provided in an embodiment of this application;

[0084] Figure 3 This is a schematic diagram illustrating the process of numbering processing operators provided in an embodiment of this application;

[0085] Figure 4 A flowchart illustrating another model calculation method provided in an embodiment of this application;

[0086] Figure 5 This is a schematic diagram illustrating the process of transferring external input data to the shared memory corresponding to the processing core set, as provided in an embodiment of this application.

[0087] Figure 6 This is a schematic diagram illustrating the use of shared memory during model computation, provided in an embodiment of this application.

[0088] Figure 7 A schematic diagram illustrating the model calculation process provided in the embodiments of this application;

[0089] Figure 8 This is a schematic diagram of the external input and output data during the model operation process provided in the embodiments of this application;

[0090] Figure 9 This is a schematic diagram of the structure of a model computing device provided in an embodiment of this application;

[0091] Figure 10 This is a schematic diagram of another model computing device provided in an embodiment of this application;

[0092] Figure 11 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0093] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0094] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0096] To facilitate understanding, the following will be combined with... Figure 1 The application scenarios applicable to the embodiments of this application will be described.

[0097] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application. Please refer to [link / reference]. Figure 1The system includes a computing device 101. The computing device 101 can be a terminal device or a server, and it contains multiple processing cores (processing core 1, processing core 2, ..., processing core X). Each processing core has its corresponding internal memory. When a processing core runs multiple processing operators of the target model, it performs prefetching of input data, pre-storage of input data, computation of input data, and prefetching of output data. Each process has different internal memory depending on the processing core. Input data can be retrieved from the corresponding internal memory. The processing operators perform MAC calculations on the input data to obtain output data, which is then stored in the internal memory corresponding to the processing core, and finally output from the internal memory to DDR. The output data is either an intermediate computation result or the target computation result of the target model.

[0098] In related technologies, model computation can be performed as follows: Input data of the neural network model is stored in DDR. The core memory for storing the input data is determined. Input data is retrieved from DDR via DMA and stored in the corresponding core memory. The processing core reads the input data from its corresponding core memory and runs the corresponding processing operator to perform MAC calculations on the input data, obtaining intermediate computation results, which are then stored in the corresponding core memory. In this process, each processing core can only read input data or store intermediate computation results in its corresponding core memory when running a processing operator for MAC calculations. If a processing core needs intermediate computation results from other processing cores for MAC calculations, it needs to read the intermediate computation results from the core memory of those other processing cores and store them in its own core memory before performing the MAC calculation. Furthermore, the intermediate computation results obtained from each MAC calculation must be output to DDR. When performing multiple MAC calculations, the computing device needs to perform multiple read-and-store operations, resulting in low efficiency in model computation.

[0099] In this embodiment, the processing cores in the computing device are divided into at least one set of processing cores, and each set of processing cores corresponds to a shared memory. Each set of processing cores includes at least two processing cores, and the processing cores in the set share the corresponding shared memory. Multiple processing operators in the target model and the operator priority corresponding to each processing core are determined. For any processing core in any set of processing cores, based on the data in the shared memory corresponding to the set, the dependencies between multiple processing operators, and the operator priority corresponding to the processing core, the processing core is controlled to perform operator operations to obtain intermediate operation results, which are then stored in the shared memory. In the above process, after controlling the processing core to perform operator operations and obtaining the latest intermediate operation result, the latest intermediate operation result is stored in the shared memory. This allows the processing core to directly obtain the intermediate operation result from the shared memory when performing the next MAC calculation, avoiding reading the intermediate operation result from the core memory corresponding to other processing cores, reducing the number of read and storage operations of the computing device, and improving the efficiency of model computation.

[0100] The method described in this application will now be illustrated through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; identical or similar content will not be repeated in different embodiments.

[0101] Figure 2 This is a flowchart illustrating a model calculation method provided in an embodiment of this application. Please refer to [link / reference]. Figure 2 The method may include:

[0102] S201. Determine the multiple processing operators in the target model, and the operator priority corresponding to each processing core.

[0103] The execution entity in this application embodiment can be a computing device, or a chip, chip module, or model computing device disposed in the computing device. The model computing device can be implemented by software or by a combination of software and hardware. The computing device can be a terminal device or a server, and the terminal device can be a computer.

[0104] Operator priorities include the priority of each processing operator.

[0105] The processing cores can be randomly assigned numbers based on the number of processing cores included in the computing device. Once the processing core numbers are determined, they will not be changed.

[0106] The priority of each processing operator can be determined based on its order of operation. Each processing operator is then numbered according to its order of operation. The processing operators are executed sequentially from smallest to largest number. The operation types for processing operators can be sequential, parallel, or a combination of serial and parallel operations. In practical applications, parallel operations are possible, so two processing operators performing parallel operations can be numbered sequentially from left to right.

[0107] Below, in conjunction with Figure 3 The process of numbering the processing operators is explained. Figure 3 This is a schematic diagram illustrating the process of numbering processing operators as provided in an embodiment of this application. Please refer to [link / reference]. Figure 3 This includes processing operators 301 to 304. Processing operators 301 to 304 are multiple processing operators corresponding to the target model. There are three possible operation orders for processing operators 301 to 304.

[0108] Please refer to Method 1. The operation type of processing operators 301 and 302 is parallel operation. Processing operator 303 is operated after processing operator 301, and processing operator 304 is operated after processing operator 302. It can be determined that processing operators 301 and 302 are numbered first, sequentially from left to right. Processing operators 303 and 304 are numbered sequentially after processing operators 301 and 302. Optionally, the identifier value of the processing operators can be set to determine whether the operation type between the two processing operators is sequential or parallel. If the operation type between the two processing operators is parallel operation, the identifier value of the processing operator that performs the operation first is less than the identifier value of the processing operator that performs the operation later. If the operation type between the two processing operators is sequential operation, the identifier values ​​of the two processing operators are the same. Therefore, the specific numbering and corresponding identifier data of processing operators 301 to 304 in Method 1 are shown in Table 1:

[0109] Table 1

[0110]

[0111]

[0112] Please refer to Method 2, where the operation type of processing operators 301 to 304 is sequential operation. In this case, the processing operators are numbered according to the order of sequential operations. Simultaneously, a corresponding identifier value is determined for each processing operator. The specific numbering and corresponding identifier data of processing operators 301 to 304 in Method 2 are shown in Table 2:

[0113] Table 2

[0114] Processing Operators serial number Identification value Processing operator 301 F1 1 Processing operator 302 F2 2 Processing operator 303 F3 3 Processing operator 304 F4 4

[0115] Please refer to Method 3, where the operation type of processing operators 301 to 304 is mixed operation. In this case, the processing operators for parallel operations are determined to be processing operators 301 and 302. The operators for sequential operations are processing operators 301, 303, and 304, respectively. Based on the above method, the specific numbers and corresponding identification data of processing operators 301 to 304 in Method 3 are shown in Table 3:

[0116] Table 3

[0117] Processing Operators serial number Identification value Processing operator 301 F1 1 Processing operator 302 F2 1 Processing operator 303 F3 2 Processing operator 304 F4 3

[0118] When determining the operator priority for each processing core, first determine the highest-level operator for each processing core. Then, based on the operator number, determine the operator priority for each processing core sequentially.

[0119] The operator priority corresponding to each processing core can be determined sequentially based on the number of each processing operator as follows: the number of the processing operator with the highest priority is determined as the target number; the operator priority is determined by starting from the target number, from largest to smallest to the first number, and then from the largest number from largest to smallest, until the target number.

[0120] For example, based on the processing operators 301 to 304 shown in the example above, their corresponding numbers are determined to be F1 to F4. Assume the computing device includes four processing cores, each with corresponding numbers C1, C2, C3, and C4. For processing operator 301, first determine that the highest-level operator corresponding to processing core C1 is processing operator 301. The number F1 of processing operator 301 is determined as the target number. When determining the operator priorities of processing operators 302 to 304, since the target number F1 is the first number, the operator priorities are determined from the largest number down to the target number. The operator priorities corresponding to processing core C1 are thus obtained as F1, F4, F3, and F2.

[0121] When determining the operator priority for each processing core, in addition to the highest priority operator for each core, the operators that process input data or obtain initial model output data are given the highest priority. This ensures that intermediate computation results are obtained while processing them in shared memory to obtain output results as quickly as possible, improving the utilization of MAC computation and maximizing the use of shared memory.

[0122] S202. For any multiply-accumulate MAC calculation, determine the input data corresponding to the processing core set, and transfer the input data to the shared memory corresponding to the processing core set.

[0123] The input data corresponding to the set of processing kernels can be determined based on the size of the convolution kernel corresponding to the target model, the stride of the convolution kernel, the number of intermediate operation results stored in shared memory, and the number of at least one processing kernel corresponding to shared memory.

[0124] Input data may include one or more input values.

[0125] For example, as illustrated above, the processing operators are determined to be F1 to F4, and the operation type of processing operators F1 to F4 is sequential operation. Assume the convolution kernel size is 3×3, the stride is 1, and the number of at least one processing core corresponding to shared memory is 4. Since the convolution kernel size is 3×3, the input data needs to satisfy 3 processing cores for MAC calculation. For the first MAC calculation, since it is the first MAC calculation, there are no intermediate calculation results in shared memory. At this time, each processing core runs processing operator F1 to perform MAC calculation, and each processing core needs 3 input data to perform the calculation. Since the stride of the convolution kernel is 1, the specific input data required by each processing core can be shown in Table 4:

[0126] Table 4

[0127]

[0128]

[0129] As shown in Table 4, the first MAC calculation determines that the input data corresponding to the processing core set includes input data 1 to input data 6. Therefore, input data 1 to input data 6 are obtained from DDR and transferred to the shared memory corresponding to the processing core set via DMA.

[0130] For non-initial MAC calculations, intermediate computation results exist in the shared memory corresponding to the processing kernel set. If the number of intermediate computation results is the same as the side length of the convolution kernel, then it is determined that the intermediate computation results can be used as input to a certain processing kernel for MAC calculation.

[0131] S203. For any processing core in any set of processing cores, based on the data in the shared memory corresponding to the set of processing cores, the dependencies between multiple processing operators, and the operator priority corresponding to the processing core, control the processing core to perform operator operations, obtain intermediate operation results, and store the intermediate operation results in the shared memory.

[0132] Intermediate operation results can be obtained as follows: Determine the data type of each data in shared memory, which can be either an external input type or an intermediate operation result type; Based on the data type, dependencies, and operator priority of each data in shared memory, determine the target processing operator among multiple processing operators, and determine the operation data corresponding to the target processing operator in shared memory; Based on the target processing operator, perform operations on the operation data through the processing core to obtain the intermediate operation results.

[0133] The model computation method provided in this application determines multiple processing operators in the target model and the operator priority corresponding to each processing core. For any multiply-accumulate MAC calculation, the off-core input data corresponding to the processing core set is determined and transferred to the shared memory corresponding to the processing core set. For any processing core in any processing core set, based on the data in the shared memory corresponding to the processing core set, the dependencies between multiple processing operators, and the operator priority corresponding to the processing core, the processing core is controlled to perform operator operations to obtain intermediate computation results, which are then stored in the shared memory. In the above process, after controlling the processing core to perform operator operations and obtaining the latest intermediate computation result, the latest intermediate computation result is stored in the shared memory. This allows the processing core to directly obtain the intermediate computation result from the shared memory when performing the next MAC calculation, avoiding reading the intermediate computation result from the on-core memory corresponding to other processing cores, reducing the number of read and storage operations of the computing device, and improving the efficiency of model computation.

[0134] Based on any of the above embodiments, the following, in conjunction with Figure 4 The detailed process of model calculation is explained.

[0135] Figure 4 This is a flowchart illustrating another model calculation method provided in an embodiment of this application. Please refer to... Figure 4 The method includes:

[0136] S401. Determine multiple processing operators in the target model and the operator priority corresponding to each processing core. The operator priority includes the priority of each processing operator.

[0137] It should be noted that this application does not limit the type of processing operator. Multiple processing operators in the target model can be of the same type or different types. It also supports the fusion of multiple types of processing operators. For example, convolutional operators can be fused with pooling, accumulation, and depthwise separable convolutional operators. MAC calculations within each kernel can be combined with vector and other computational units from other kernels.

[0138] S402. For any multiply-accumulate MAC calculation, determine the off-core input data corresponding to the processing core set, and transfer the off-core input data to the shared memory corresponding to the processing core set.

[0139] Below, in conjunction with Figure 5 The process of transferring external input data to the shared memory corresponding to the processing core set is explained. Figure 5 This is a schematic diagram illustrating the process of transferring off-core input data to the shared memory corresponding to the processing core set, as provided in an embodiment of this application. Please refer to... Figure 5 The system includes DDR 501, shared memory 502, and shared memory 503. DDR 501, shared memory 502, and shared memory 503 are located in the computing device. Shared memory 502 corresponds to processing core set 1, which includes processing core 1, processing core 2, processing core 3, and processing core 4. Multiple processing cores in processing core set 1 can retrieve data from or store data in shared memory 502. Shared memory 503 corresponds to processing core set 2, which includes processing core 5, processing core 6, processing core 7, and processing core 8. Multiple processing cores in processing core set 2 can retrieve data from or store data in shared memory 503. Multiple input data of the target model are stored in DDR 501. For any multiply-accumulate MAC calculation, the external input data corresponding to the processing core set is determined, and the corresponding external input data is retrieved from DDR 501. External input data is transferred to shared memory 502 and / or shared memory 503 corresponding to the processing core set via DMA.

[0140] S403. For any processing core in any set of processing cores, determine the data type of each data in the shared memory.

[0141] The data type is either an external input type or an intermediate operation result type.

[0142] The off-core input type indicates that data is retrieved from the computing device's DDR and stored in shared memory via DMA. The intermediate operation result type indicates that the data is obtained through operator operations.

[0143] S404. Determine the highest priority operator corresponding to the processing kernel based on the operator priority corresponding to the processing kernel.

[0144] For example, the target model includes four processing operators: F1, F2, F3, and F4. The computing device includes four processing cores, and the operator priorities corresponding to each processing core are shown in Table 5.

[0145] Table 5

[0146] Processing kernel Operator priority C1 F1, F4, F3, F2 C2 F2, F1, F4, F3 C3 F3, F2, F1, F4 C4 F4, F3, F2, F1

[0147] Based on the operator priorities corresponding to each processing core shown in Table 5, the highest priority operator corresponding to processing core C1 is F1, the highest priority operator corresponding to processing core C2 is F2, the highest priority operator corresponding to processing core C3 is F3, and the highest priority operator corresponding to processing core C4 is F4.

[0148] S405. Determine whether the highest priority operator has a dependent operator based on the dependency relationship.

[0149] If so, execute S406.

[0150] If not, execute S413.

[0151] If the first processing operator performs MAC calculation to obtain an intermediate result, and the second processing operator performs MAC calculation on the intermediate result to obtain the final result, then the first processing operator is determined to be the dependent operator.

[0152] For example, based on the multiple processing operators shown in Table 5 above, the order of operations for these operators is determined to be F1, F2, F3, and F4. It can be determined that processing operator F1 processes the input data to obtain intermediate operation results, thus confirming that processing operator F1 has no dependent operators. Processing operators F2, F3, and F4 require intermediate operation data input from the previous processing operator to perform MAC calculations, therefore confirming that processing operators F2, F3, and F4 have dependent operators. The specific dependent operators corresponding to processing operators F2, F3, and F4 are shown in Table 6.

[0153] Table 6

[0154]

[0155]

[0156] S406. Determine whether intermediate operation data of intermediate operation result type exists in the shared memory.

[0157] If so, execute S407.

[0158] If not, execute S413.

[0159] S407. Determine the processing operator corresponding to each intermediate operation data.

[0160] The processing operator corresponding to each intermediate operation data can be determined based on the dependencies between processing operators.

[0161] For example, based on the multiple processing operators shown in the examples above, the specific operational data obtained by each processing operator can be determined as shown in Table 7:

[0162] Table 7

[0163] Processing Operators Intermediate processing data F1 Calculation data 1 F2 Calculation data 2 F3 Calculation data 3 F4 Calculation data 4

[0164] If the order of operations for multiple processing operators is determined to be F1, F2, F3, and F4, then the data types of operation data 1, operation data 2, and operation data 3 shown in Table 7 are all intermediate operation result types. Therefore, the processing operators corresponding to each intermediate operation data can be determined as shown in Table 7.

[0165] S408. Determine the number of intermediate operation data corresponding to each processing operator based on the processing operator corresponding to each intermediate operation data.

[0166] The processing operator corresponding to the intermediate operation data is the operator that obtains the intermediate operation data through operation.

[0167] S409. Determine the kernel side length N of the target model convolution kernel.

[0168] N is an integer greater than or equal to 1.

[0169] It should be noted that the convolution kernel of the target model in this application is a square convolution kernel, that is, the four sides of the convolution kernel are the same length.

[0170] S410. Sort the multiple processing operators according to their priority from largest to smallest to obtain the processing order.

[0171] For example, based on the operator priority corresponding to each processing core shown in Table 5 above, the processing order of processing core C2 can be determined as F2, F1, F4, F3.

[0172] S411. According to the processing order, based on the dependency relationship, the number of intermediate operation data corresponding to each processing operator, and the size of the convolution kernel, determine whether the processing operator is the target processing operator in turn, until the target processing operator is determined among multiple processing operators.

[0173] The following method can be used to determine whether a processing operator is a target processing operator based on the dependency relationship, the number of intermediate operation data corresponding to each processing operator, and the convolution kernel size: Based on the dependency relationship, determine the dependent operator corresponding to the i-th processing operator. If the number of intermediate operation data corresponding to the dependent operator is greater than or equal to N, then the processing operator is determined as the target processing operator; where i takes the values ​​1, 2, ..., until the target processing operator is determined, i is an integer less than or equal to M, and M is the number of multiple processing operators.

[0174] For example, assuming the convolution kernel size is 3×3, based on the dependencies and as shown in the example above, the dependent operator corresponding to the second processing operator F2 is determined to be F1. If the number of intermediate operation data corresponding to the dependent operator F1 is determined to be 4, then the number of intermediate operation data corresponding to the dependent operator is greater than 3. In this case, processing operator F2 is determined as the target processing operator.

[0175] S412. Determine the operation data corresponding to the target processing operator in the shared memory data.

[0176] When determining the computational data corresponding to the target processing operator, if the number of intermediate computational data corresponding to the dependent operator is equal to N, then the N intermediate computational data are determined as the computational data corresponding to the target processing operator. If the number of intermediate computational data corresponding to the dependent operator is greater than N, then the first N computational data are determined as the computational data corresponding to the target processing operator according to the order in which the intermediate computational data is obtained.

[0177] For example, as illustrated above, the target processing operator is determined to be F2. In this case, the operation data corresponding to the target processing operator identified in the shared memory includes operation data A1, operation data A2, operation data A3, and operation data A4. If the number of intermediate operation data corresponding to the dependent operator is greater than 3, then operation data A1, operation data A2, and operation data A3 are identified as the operation data corresponding to the target processing operator from among the multiple intermediate operation data according to the order in which they are obtained.

[0178] After S412, execute S414.

[0179] S413. Determine the highest priority operator as the target processing operator, and determine the operation data corresponding to the target processing operator in the data of the off-core data type in shared memory.

[0180] When the highest priority operator has no dependent operators, or when there is no intermediate operation data of intermediate operation result type in the shared memory, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined from the data of the off-core data type in the shared memory.

[0181] When determining the computational data corresponding to the target processing operator, the range of computational data is determined based on the size of the convolution kernel. The initial computational data to be acquired is determined based on the stride of the convolution kernel.

[0182] For example, as illustrated above, the highest priority operator for processing core C1 is determined to be F1. Since the highest priority operator F1 has no dependent operators, it is designated as the target processing operator. The corresponding computational data for the target processing operator is then determined from the data of the off-core data type in shared memory.

[0183] S414. Based on the target processing operator, perform calculations on the data to obtain intermediate calculation results.

[0184] Before performing any MAC calculation, the shared memory capacity can be divided into multiple equal sub-memories, with each processing core corresponding to at least one sub-memory.

[0185] Memory can be allocated to the processing core in the following way: determine the computational cost of each processing operator running on the processing core; determine the amount of sub-memory corresponding to the processing core based on the computational cost of each processing operator running on the processing core; allocate memory for the processing core in the shared memory corresponding to the processing core based on the amount of sub-memory corresponding to the processing core.

[0186] The number of sub-memory corresponding to the processing core can be determined using the following formula 1:

[0187] bank=(K h -pad)+(stride1+stride2+…+stride N )×2+3×

[0188] m((n1-1))×stride1+((n2-1))×stride2+((n N -1))×stride N )

[0189] Where bank represents the number of sub-memory units; K h is the weight corresponding to the height of the convolution kernel; pad is the padding step of the convolution kernel; stride is the stride of the convolution kernel corresponding to the processing kernel; m is the number of processing operators; n is the ratio of the MAC computation of any processing operator corresponding to each processing kernel to the total computation of multiple processing operators corresponding to that processing kernel; N is the number of processing kernels.

[0190] It should be noted that if n is not an integer, then n is rounded up, and the resulting integer value is updated to n.

[0191] S415. Store the intermediate calculation results in shared memory.

[0192] Once it is determined that the intermediate operation results stored in shared memory are no longer used as input data for the processing operator, the corresponding sub-memory is released to improve the utilization of shared memory.

[0193] In actual use, an additional user cache is added outside of shared memory to store overlapping data. The memory capacity of the cache is smaller than that of the shared memory.

[0194] After calculating the target operation result corresponding to the target model, the target operation result is output to the preset memory.

[0195] The default memory can be DDR.

[0196] Below, in conjunction with Figure 6 The process of using shared memory during model computation is explained. Figure 6 This diagram illustrates the use of shared memory during model computation, as provided in this embodiment of the application. Please refer to... Figure 6 The system includes a computing device 601. The computing device 101 can be a terminal device or a server. The computing device 101 has a processing core set, which includes multiple processing cores (processing core 1, processing core 2, ..., processing core S). The processing core set corresponds to a shared memory 1, which includes multiple sub-memories. Model computation includes four processes: prefetching, pre-storage, MAC computation, and output. Each process corresponds to a shared memory. For any multiply-accumulate MAC computation, the external input data corresponding to the processing core set is determined. This external input data is retrieved from the DDR and stored in the shared memory 2 corresponding to the data prefetching process. The external input data is transferred from shared memory 2 to the shared memory 1 corresponding to the processing core set. The shared memory 1 corresponding to the processing core set stores the external input data, intermediate computation data, and target computation result used for MAC computation. After the target computation result is determined, the target computation result is transferred from shared memory 1 to the shared memory 3 corresponding to the MAC computation output temporary storage, and then transferred to the DDR via DAM. In this embodiment, since intermediate computation data can be processed by the core in shared memory 1 to obtain the target computation result through MAC calculation, the data of the first processing operator of the target model is pre-stored in shared memory 2, and the MAC calculation output of the last processing operator of the target model is temporarily stored in shared memory 3. This reduces the number of times the computing device 601 reads and stores data during model computation, thus improving the efficiency of model computation.

[0197] The model operation method provided in this application determines multiple processing operators in the target model and the operator priority corresponding to each processing core. For any multiply-accumulate MAC calculation, the off-core input data corresponding to the processing core set is determined and transferred to the shared memory corresponding to the processing core set. For any processing core in any processing core set, the data type of each data in the shared memory is determined. Based on the data type, dependency relationship, and operator priority corresponding to the processing core in the shared memory, the target processing operator is determined among the multiple processing operators, and the operation data corresponding to the target processing operator is determined in the shared memory. According to the target processing operator, the operation data is processed by the processing core to obtain intermediate operation results. The intermediate operation results are then stored in the shared memory. In the above process, the processing core is controlled to perform operator operations, and after obtaining the latest intermediate operation results, the latest intermediate operation results are stored in the shared memory. This allows the processing core to directly obtain the intermediate operation results from the shared memory when performing the next MAC calculation, avoiding reading the intermediate operation results from the core memory corresponding to other processing cores, reducing the number of read and storage operations of the computing device, and improving the efficiency of model operation.

[0198] Based on any of the above embodiments, the following, in conjunction with Figures 7-8 The process of model calculation is illustrated with an example.

[0199] Figure 7 This is a schematic diagram illustrating the model calculation process provided in an embodiment of this application. Please refer to [link / reference]. Figure 7 The system includes 16 external input data points for the target model and multiple MAC calculations. Each external input data point includes at least one or more data values. The target model is determined to contain four processing operators: F1, F2, F3, and F4. The order of operation for these operators is F1, F2, F3, and F4. The computing device includes a set of four processing cores: C1, C2, C3, and C4. This set of cores corresponds to a shared memory location. Based on the operator priority for each processing core, the highest priority operator for each core is determined to be F1, F2 for C2, F3 for C3, and F4 for C4. The convolution kernel size of the target model is assumed to be 3×3 with a stride of 1. Based on the priority of the processing operators corresponding to each processing core and the data types of each data in shared memory, the execution order of multiple calculations can be obtained as shown in Table 8:

[0200] Table 8

[0201]

[0202]

[0203] Before each MAC calculation, an execution order table as shown in Table 8 can be generated based on the operation order and the external input data. MAC calculations can be performed according to the execution order table, and the execution order table can be updated after each MAC calculation until the target operation result is obtained. If the operation order of multiple processing operators involves parallel operations and mixed operations, the identifier value of each processing operator during the operation process can be determined, and the identifier value can be updated after each MAC calculation.

[0204] Table 8 shows rows from the first to the last, with each row indicating one MAC calculation. For example, the first row indicates the first MAC operation. For any array in the first four columns of Table 8, any value in the array indicates the operation status of the corresponding processing operator. A value of 1 indicates that the corresponding processing operator is performing an operation, while a value of 0 or "-" indicates that the corresponding processing operator is not performing an operation. For any array in the last column of Table 8, any value in the array indicates the number of data points in the input data for the corresponding processing operator. When the number of input data points is the same as the side length N of the convolution kernel, the corresponding processing operator can perform operations on the input data to obtain the result.

[0205] Please see Figure 7 In the first MAC operation shown, the computing device stores 16 off-kernel input data (input data 1 to input data 16) of the target model into DDR. Since the first MAC operation has not yet obtained intermediate results, all processing kernels in the processing kernel set run the processing operator F1. The convolution kernel size of the target model is determined to be 3×3 with a stride of 1. Therefore, each processing kernel in the processing kernel set needs to input 3 input data, and the stride between kernels is 1. Therefore, the required input data is determined to be input data 1 to input data 6. Corresponding to the first row of Table 8 above, processing kernels 1 to 4 have not yet performed operations. After the computing device retrieves input data 1 to input data 6 from DDR and transfers input data 1 to input data 6 to the shared memory corresponding to the processing kernel set, each processing kernel runs the processing operator F1, obtaining 4 intermediate operation results. Therefore, only the 6 input data corresponding to the processing operator F1 are in the shared memory, corresponding to the number of input data shown in Table 8 above as (6, 0, 0, 0).

[0206] Please see Figure 7The second MAC operation shown identifies four intermediate operation data (intermediate data 1-1 to intermediate data 1-4) corresponding to operator F1 in shared memory. These four intermediate operation data can only be processed by operator F2. Since the step size is 1, two processing cores can process these four intermediate operation data. Within the processing core set, processing cores 2 and 3 are identified as having the highest priority for operator F2. Therefore, processing cores 2 and 3 process the four intermediate operation data to obtain the intermediate operation data (intermediate data 2-1 and intermediate data 2-2) corresponding to operator F2. Processing cores 1 and 2 continue to acquire external input data for MAC operations to obtain more intermediate operation data. Corresponding to the second row of data in Table 8 above, the second MAC operation requires four external input data and four intermediate operation data. Therefore, the executable data for the operator is (4, 4, 0, 0). This process continues until the target operation result is obtained. The target operation result is then transmitted to the DDR of the computing device.

[0207] Figure 8 This is a schematic diagram illustrating the input and output data outside the kernel during the model calculation process provided in this application embodiment. Please refer to... Figure 8 This includes the off-core input and output data for each of the six MAC calculations. The six MAC calculations correspond to... Figure 7 The diagram shows six MAC calculations. For any given operation, the data corresponding to the dotted horizontal dashed box indicates the end of the MAC calculation for that data. The data corresponding to the dashed box indicates the data obtained in this MAC calculation. For the first MAC calculation, the external input data is determined to be input data 1 to input data 6. After the first MAC calculation, four intermediate operation data (intermediate data 1-1 to intermediate data 1-4) corresponding to the processing operator F1 are obtained. For the second MAC calculation, the external input data is determined to be input data 1 to input data 8. Since input data 1 to input data 4 will not be used again after this MAC calculation, they are represented by dotted horizontal dashed boxes. The output data obtained in this MAC calculation is determined to be two intermediate operation data (intermediate data 2-1 and intermediate data 2-2) corresponding to the processing operator F2, and two intermediate operation data (intermediate data 1-5 and intermediate data 1-6) corresponding to the processing operator F1. This process continues until the output data is obtained. Figure 6 The six MAC calculations shown correspond to the off-core input and output data.

[0208] The model operation method provided in this application determines multiple processing operators in the target model and the operator priority corresponding to each processing core. For any multiply-accumulate MAC calculation, the off-core input data corresponding to the processing core set is determined and transferred to the shared memory corresponding to the processing core set. For any processing core in any processing core set, the data type of each data in the shared memory is determined. Based on the data type, dependency relationship, and operator priority corresponding to the processing core in the shared memory, the target processing operator is determined among the multiple processing operators, and the operation data corresponding to the target processing operator is determined in the shared memory. According to the target processing operator, the operation data is processed by the processing core to obtain intermediate operation results. The intermediate operation results are then stored in the shared memory. In the above process, the processing core is controlled to perform operator operations, and after obtaining the latest intermediate operation results, the latest intermediate operation results are stored in the shared memory. This allows the processing core to directly obtain the intermediate operation results from the shared memory when performing the next MAC calculation, avoiding reading the intermediate operation results from the core memory corresponding to other processing cores, reducing the number of read and storage operations of the computing device, and improving the efficiency of model operation.

[0209] Figure 9 This is a schematic diagram of a model computing device provided in an embodiment of this application. Please refer to... Figure 9 The model computing device 10 may include:

[0210] The first determining module 11 is used to determine multiple processing operators in the target model and the operator priority corresponding to each processing core, wherein the operator priority includes the priority of each processing operator;

[0211] The second determining module 12 is used to determine the input data corresponding to the processing core set for any multiply-accumulate MAC calculation, and to transmit the input data to the shared memory corresponding to the processing core set.

[0212] Control module 13 is used to determine the intermediate operation results existing in the shared memory and the operator priority corresponding to each processing core, control the processing core to perform operator operations, obtain the latest intermediate operation results, and store the latest intermediate operation results in the shared memory.

[0213] In one possible implementation, the control module 13 is specifically used for:

[0214] Determine the data type of each piece of data in the shared memory, wherein the data type is an off-core input type or an intermediate operation result type;

[0215] Based on the data type of each data in the shared memory, the dependency relationship, and the operator priority corresponding to the processing core, a target processing operator is determined from the plurality of processing operators, and the operation data corresponding to the target processing operator is determined in the shared memory;

[0216] Based on the target processing operator, the processing kernel performs calculations on the computational data to obtain the intermediate computational result.

[0217] In one possible implementation, the control module 13 is specifically used for:

[0218] Based on the operator priority corresponding to the processing kernel, determine the highest priority operator corresponding to the processing kernel;

[0219] Determine whether the highest priority operator has a dependent operator based on the dependency relationship;

[0220] If so, the target processing operator and the computational data corresponding to the target processing operator are determined based on the data type of each data in the shared memory, the dependency relationship, and the operator priority corresponding to the processing core;

[0221] If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

[0222] In one possible implementation, the control module 13 is specifically used for:

[0223] Determine whether intermediate operation result type intermediate operation data exists in the shared memory;

[0224] If so, then determine the processing operator corresponding to each intermediate operation data, and determine the target processing operator and the operation data corresponding to the target processing operator based on the processing operator corresponding to each intermediate operation data, the dependency relationship, and the operator priority corresponding to the processing kernel. The processing operator corresponding to the intermediate operation data is the operator that calculates the intermediate operation data.

[0225] If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

[0226] In one possible implementation, the control module 13 is specifically used for:

[0227] The number of intermediate operation data corresponding to each processing operator is determined based on the processing operator corresponding to each intermediate operation data.

[0228] Determine the kernel side length N of the target model convolution kernel, where N is an integer greater than or equal to 1;

[0229] The processing operators are sorted in descending order of priority to obtain the processing order.

[0230] According to the processing order, based on the dependency relationship, the number of intermediate operation data corresponding to each processing operator, and the size of the convolution kernel, it is determined in turn whether the processing operator is the target processing operator, until the target processing operator is determined among the multiple processing operators;

[0231] The computational data corresponding to the target processing operator is determined from the data in the shared memory.

[0232] In one possible implementation, the control module 13 is specifically used for:

[0233] Based on the dependency relationship, determine the dependent operator corresponding to the i-th processing operator. If the number of intermediate operation data corresponding to the dependent operator is greater than or equal to N, then the processing operator is determined as the target processing operator.

[0234] Wherein, i takes the values ​​1, 2, ..., until the target processing operator is determined, i is an integer less than or equal to M, and M is the number of the plurality of processing operators.

[0235] The model computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0236] Figure 10 This is a schematic diagram of another model computing device provided in an embodiment of this application. Figure 9 Based on the illustrated embodiments, please refer to Figure 10 The model computing device 10 also includes a distribution module 14 and an output module 15.

[0237] The allocation module 14 is used for:

[0238] For any given processing core, multiple sub-memories corresponding to the processing core are determined. The memory capacity of the multiple sub-memories is the same, and the sum of the capacities of the multiple sub-memories is the memory capacity of the shared memory.

[0239] Determine the computational cost of each processing operator executed by the processing core;

[0240] The number of sub-memory corresponding to the processing core is determined based on the computational amount of each processing operator run by the processing core;

[0241] Based on the number of sub-memory corresponding to the processing core, memory is allocated for the processing core in the shared memory corresponding to the processing core.

[0242] The output module 15 is used for:

[0243] After calculating the target operation result corresponding to the target model, the target operation result is output to a preset memory.

[0244] The model computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0245] Figure 11 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Please refer to... Figure 11 The computing device 20 may include a memory 21 and a processor 22. Exemplarily, the memory 21 and the processor 22 are interconnected via a bus 23.

[0246] Memory 21 is used to store program instructions;

[0247] The processor 22 is used to execute the program instructions stored in the memory, so that the computing device 20 performs the method shown in the above method embodiment.

[0248] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0249] This application provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above-described method when executed by a processor.

[0250] This application embodiment may also provide a computer program product, including a computer program that, when executed by a processor, can implement the above-described method.

[0251] All or part of the steps in the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-described method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), random access memory (RAM), flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0252] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0253] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0254] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0255] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

[0256] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

Claims

1. A model calculation method, characterized in that, Applied in a computing device, the computing device includes at least one set of processing cores and shared memory corresponding to each set of processing cores, wherein the set of processing cores includes at least two processing cores, and the processing cores in the set of processing cores share the corresponding shared memory, the method includes: Determine multiple processing operators in the target model, and the operator priority corresponding to each processing core, wherein the operator priority includes the priority of each processing operator; For any multiply-accumulate MAC calculation, determine the off-core input data corresponding to the processing core set, and transfer the off-core input data to the shared memory corresponding to the processing core set; The data type of each data in the shared memory is determined, wherein the data type is an external input type or an intermediate operation result type; for any processing core in any set of processing cores, the highest priority operator corresponding to the processing core is determined according to the operator priority corresponding to the processing core; the dependency relationship between multiple processing operators is used to determine whether the highest priority operator has a dependent operator; if so, the target processing operator and the operation data corresponding to the target processing operator are determined according to the data type in the shared memory corresponding to the set of processing cores, the dependency relationship between the multiple processing operators, and the operator priority corresponding to the processing core; if not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined from the data of the external data type in the shared memory, the intermediate operation result is obtained according to the operation data, and the intermediate operation result is stored in the shared memory.

2. The method according to claim 1, characterized in that, The step of obtaining intermediate calculation results based on the calculation data includes: Based on the target processing operator, the processing kernel performs calculations on the computational data to obtain the intermediate computational result.

3. The method according to claim 1, characterized in that, Based on the data types of each data in the shared memory, the dependencies, and the operator priority corresponding to the processing core, the target processing operator and the corresponding computational data are determined, including: Determine whether intermediate operation result type intermediate operation data exists in the shared memory; If so, then determine the processing operator corresponding to each intermediate operation data, and determine the target processing operator and the operation data corresponding to the target processing operator based on the processing operator corresponding to each intermediate operation data, the dependency relationship, and the operator priority corresponding to the processing kernel. The processing operator corresponding to the intermediate operation data is the operator that calculates the intermediate operation data. If not, the highest priority operator is determined as the target processing operator, and the operation data corresponding to the target processing operator is determined in the data of the off-core data type of the shared memory.

4. The method according to claim 3, characterized in that, Based on the processing operators corresponding to each intermediate computational data, the dependencies, and the operator priorities corresponding to the processing kernel, the target processing operator and the computational data corresponding to the target processing operator are determined, including: The number of intermediate operation data corresponding to each processing operator is determined based on the processing operator corresponding to each intermediate operation data. Determine the kernel side length N of the target model convolution kernel, where N is an integer greater than or equal to 1; The processing operators are sorted in descending order of priority to obtain the processing order. According to the processing order, based on the dependency relationship, the number of intermediate operation data corresponding to each processing operator, and the size of the convolution kernel, it is determined in turn whether the processing operator is the target processing operator, until the target processing operator is determined among the multiple processing operators; The computational data corresponding to the target processing operator is determined from the data in the shared memory.

5. The method according to claim 4, characterized in that, Based on the dependencies, the amount of intermediate computation data corresponding to each processing operator, and the convolution kernel size, the processing operator is sequentially determined to be the target processing operator until the target processing operator is determined among the plurality of processing operators, including: Based on the dependency relationship, determine the dependent operator corresponding to the i-th processing operator. If the number of intermediate operation data corresponding to the dependent operator is greater than or equal to N, then the processing operator is determined as the target processing operator. Wherein, i takes the values ​​1, 2, ..., until the target processing operator is determined, i is an integer less than or equal to M, and M is the number of the plurality of processing operators.

6. The method according to any one of claims 1-4, characterized in that, Before performing any MAC calculation, the method further includes: For any given processing core, multiple sub-memories corresponding to the processing core are determined. The memory capacity of the multiple sub-memories is the same, and the sum of the capacities of the multiple sub-memories is the memory capacity of the shared memory. Determine the computational cost of each processing operator executed by the processing core; The number of sub-memory corresponding to the processing core is determined based on the computational amount of each processing operator run by the processing core; Based on the number of sub-memory corresponding to the processing core, memory is allocated for the processing core in the shared memory corresponding to the processing core.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: After calculating the target operation result corresponding to the target model, the target operation result is output to a preset memory.

8. A model computing device, characterized in that, The device includes: The first determining module is used to determine multiple processing operators in the target model, and the operator priority corresponding to each processing core, wherein the operator priority includes the priority of each processing operator; The second determining module is used to determine the input data corresponding to the processing core set for any multiply-accumulate MAC calculation, and to transmit the input data to the shared memory corresponding to the processing core set. The control module is used to determine the data type of each data in the shared memory, wherein the data type is an external input type or an intermediate operation result type; for any processing core in any set of processing cores, determine the highest priority operator corresponding to the processing core according to the operator priority corresponding to the processing core; determine whether the highest priority operator has dependent operators according to the dependency relationship between multiple processing operators; if so, determine the target processing operator and the operation data corresponding to the target processing operator according to the data type in the shared memory corresponding to the set of processing cores, the dependency relationship between the multiple processing operators, and the operator priority corresponding to the processing core; if not, determine the highest priority operator as the target processing operator, determine the operation data corresponding to the target processing operator in the data of the external data type in the shared memory, obtain the intermediate operation result according to the operation data, and store the intermediate operation result in the shared memory.

9. A computing device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images