Model operation optimization method, device, storage medium and electronic device

Abstract

This specification discloses a model operation optimization method, device, storage medium and electronic device. In the model operation optimization method provided in this specification, a target model is obtained, and the operation function contained in the target model is determined; according to the operation function, the operator required to execute the operation function is determined; at least part of the operators in each operator is split; according to the input, output and operation type of each operator, each operator is fused with the split operators to obtain several fused operators; the fused operator is used to execute the operation function contained in the target model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model operation optimization method, device, storage medium and electronic device. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning, as a powerful machine learning method, has been widely used in image recognition, speech recognition, natural language processing and other fields. In these fields, Convolutional Neural Networks (CNN) is a common and efficient neural network architecture.

[0003] In CNN, there are some deep neural networks (DNN) with a large number of layers, such as ResNet50 and other models. In the operation processing of this type of model, the traditional method usually executes each layer of operators one by one. However, this method has the problem of frequently writing intermediate results back to the main memory, which greatly increases the number of off-chip memory accesses and leads to an increase in the memory access bandwidth requirement. At the same time, since each layer of operators is executed one by one, the parallel processing capabilities of the hardware cannot be fully utilized, and the overall processing efficiency is low. These problems make it difficult for traditional methods to meet the needs of efficient processing when the application scale and complexity of deep neural network models are increasing.

[0004] Therefore, how to speed up the efficiency of convolutional neural networks in computing and processing is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a model operation optimization method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a model operation optimization method, including:

[0008] Acquire a target model and determine the computing functions included in the target model;

[0009] According to the computing function, determining an operator required to perform the computing function;

[0010] Splitting at least some of the operators;

[0011] According to the input, output and operation type of each operator, each operator is fused with the split operators to obtain a plurality of fused operators;

[0012] The fusion operator is used to execute the calculation function included in the target model.

[0013] Optionally, the target model is a ResNet50 convolutional neural network model.

[0014] Optionally, the operators include at least a Reshape operator, a CONV operator, a Norm operator, a Relu operator, a MaxPooling operator, an ADD operator, an Avg Pooling operator, a FC operator, and a Softmax operator.

[0015] Optionally, splitting at least part of the operators includes:

[0016] The Norm operator is split to obtain a Norm_a operator and a Norm_b operator.

[0017] Optionally, each operator is fused with the split operator to obtain several fused operators, specifically including:

[0018] The Reshape operator, the CONV operator, and the Norm_a operator are fused to obtain a first fused operator;

[0019] The Norm_b operator, the Relu operator, the Max Pooling operator, the CONV operator, and the Norm_a operator are fused to obtain a second fused operator;

[0020] The Norm_b operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a third fused operator;

[0021] The CONV operator and the Norm_a operator are fused to obtain a fourth fused operator;

[0022] The Norm_b operator, the ADD operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a fifth fused operator, wherein the Norm_b operator is used twice;

[0023] The Norm_b operator, the ADD operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a sixth fused operator;

[0024] The Norm_b operator, the ADD operator, the Relu operator, the Avg Pooling operator, the FC operator, and the Softmax operator are fused to obtain a seventh fusion operator.

[0025] Optionally, the operation type includes at least a convolution operation type and a vector operation type.

[0026] Optionally, several fusion operators are obtained, including:

[0027] Several fusion operators for forward propagation and several fusion operators for backward propagation are obtained.

[0028] This specification provides a model operation optimization device, the device comprising:

[0029] An acquisition module, used to acquire a target model and determine the operation functions included in the target model;

[0030] A determination module, used to determine an operator required to perform the operation function according to the operation function;

[0031] A splitting module, used for splitting at least part of the operators in each operator;

[0032] A fusion module, used for fusing the operators with the split operators according to the input, output and operation type of the operators to obtain a plurality of fused operators;

[0033] An execution module is used to use the fusion operator to execute the calculation function included in the target model.

[0034] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned model operation optimization method is implemented.

[0035] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned model operation optimization method when executing the program.

[0036] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0037] In the model operation optimization method provided in this specification, a target model is obtained, and the operation functions contained in the target model are determined; based on the operation functions, the operators required to execute the operation functions are determined; at least part of the operators are split; based on the input, output and operation type of each operator, the operators are fused with the split operators to obtain several fused operators; and the fused operators are used to execute the operation functions contained in the target model.

[0038] When the model operation optimization method provided in this specification is used to optimize the operation of the target model, the operator can be split and fused according to the characteristics of the operator when processing data, and the fused operator can be used to execute the operation function in the target model. This method can greatly reduce the data access process that the target model needs to perform during the operation, and effectively improve the operation efficiency of the target model. Specifically, this method splits the normalization operator so that the model completes part of the calculation before performing the full reduction operation, so as to reduce the possible blockage in the data transmission and calculation process and improve the overall calculation efficiency. Furthermore, the splitting of the operator enables the calculation of some operators to be carried out in parallel with the DMA transmission. Since the DMA transmission time is much longer than the calculation time, this design can effectively utilize the transmission waiting time to perform the calculation task. At the same time, through this splitting and parallel processing, the calculations of different dimensions are split into each part of the operator, which can effectively reduce the memory space required to cache the entire batch size data on the chip, thereby significantly reducing the memory requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0040] Figure 1 A schematic diagram of a flow chart of a model operation optimization method in this specification;

[0041] Figure 2a A schematic diagram of an operator structure of the initial part of a target model including a fusion operator provided in this specification;

[0042] Figure 2b A schematic diagram of an operator structure of the middle part of a target model including a fusion operator provided in this specification;

[0043] Figure 2c A schematic diagram of an operator structure at the end of a target model including a fusion operator provided in this specification;

[0044] Figure 3 A schematic diagram of the calculation steps of the fusion operator during forward propagation in the target model provided in this specification;

[0045] Figure 4 A schematic diagram of the calculation steps of the fusion operator during back propagation in the target model provided in this specification;

[0046] Figure 5 A schematic diagram of the calculation steps of a fusion operator used to calculate parameter weights in a target model during back propagation in the target model provided in this specification;

[0047] Figure 6 A schematic diagram of a model calculation optimization device provided in this specification;

[0048] Figure 7 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0051] Figure 1 The following is a flow chart of a model operation optimization method in this specification, which specifically includes the following steps:

[0052] S100: Acquire a target model, and determine the computing functions included in the target model.

[0053] All steps in the model operation optimization method provided in this specification can be implemented by any electronic device with computing functions, such as terminals, servers and other devices.

[0054] This method is mainly used to optimize the operations involved in the neural network model when using a CNN model with a DNN structure, such as ResNet50, to improve the model's operational efficiency. Based on this, the target model can be first obtained in this step, and the operational functions included in the target model can be determined.

[0055] For the convenience of explanation and illustration, this specification will mainly use the ResNet50 model as an example for subsequent explanation, that is, unless otherwise specified, the target model is the ResNet50 convolutional neural network model.

[0056] At the same time, in this method, the target model can be built on high-performance processors such as general-purpose computing on graphics processing units (GPGPU). At present, GPGPU generally has the following architecture: main memory (Global Memory, GM), on-chip storage unit (Scratchpad Memory, SPM) and register. Different types of computing units share on-chip storage units (first-level storage) for specific operator calculations: 1. Matrix multiplication and convolution computing unit, 2. Vector computing unit, 3. Special function computing unit (such as softmax and sigmoid).

[0057] After determining the various computing functions included in the target model, it can be used as a basis to expand and implement the subsequent optimization steps in this method.

[0058] S102: According to the operation function, determine an operator required to execute the operation function.

[0059] Generally speaking, in a neural network model, all computing functions are implemented by operators. Each computing function contains several operators, which may be different or the same. In simple terms, the computing function can be regarded as a goal, and the several operators contained in the computing function are the steps to achieve this goal. In other words, the process of realizing a computing function is actually the process of executing several operators under the computing function.

[0060] In the field of deep learning, operators are usually defined as basic computing units. Simply put, an operator can be regarded as a mathematical operation. The process of realizing a certain operation function can also be regarded as the process of performing multi-step mathematical operations. At present, the operators contained in each network layer of the relatively mature neural network model are relatively fixed. Specifically, for the CNN model of the DNN structure involved in this specification, taking the ResNet50 model as an example, the operators required to realize the operation functions contained therein include at least Reshape operator, CONV operator, Norm operator, Relu operator, Max Pooling operator, ADD operator, Avg Pooling operator, FC operator, Softmax operator, etc.

[0061] Among them, the Reshape operator is an operator that reshapes the data. It can change the shape (i.e., dimension and size) of the data, but keep the total number of elements unchanged. In deep learning, data is usually represented in the form of multidimensional arrays (such as tensors). The Reshape operator can reconstruct these multidimensional arrays to meet the input requirements of different models. The CONV operator mainly slides on the input data through a matrix called a convolution kernel (or filter) and performs element-level multiplication and summation at each position to generate an output feature map. The Norm operator can perform a variety of reduction operations on the data, such as summation, maximum value, etc., so it can be used for normalization. The Relu operator is one of the most commonly used activation functions in the field of deep learning. For all input values ​​x, its output y is: y=max(0,x). That is, if the input x is greater than 0, the output y is equal to x; if the input x is less than or equal to 0, the output y is 0. The Max Pooling operator is a commonly used pooling operation in convolutional neural networks. Its main function is to downsample the input feature map, reduce the amount of calculation by reducing the size of the feature map, and retain important features to improve the generalization ability of the model. The Max Pooling operation divides the input feature map into several non-overlapping rectangular areas (called pooling windows or pooling kernels), and then takes the maximum value in each area as the output of the area. In deep learning technology, the ADD operator is usually used to perform element-by-element addition operations on two or more tensors. The Avg Pooling operator, that is, the average pooling operation, divides the input feature map into several non-overlapping rectangular areas, and then calculates the average value of all elements in each area, and uses the average value as the output of the area. The FC operator is called the Fully Connected operator. Its main function is to linearly transform the input feature space into another feature space. Mathematically, it is usually implemented through matrix multiplication and a bias term. The main function of the Softmax operator is to convert a K-dimensional vector z containing any real number into another K-dimensional real vector σ(z), in which the value of each element is between (0, 1) and the sum of all elements is 1, satisfying the properties of probability distribution.

[0062] It is worth mentioning that during the operation of the target model, the same operator may appear more than once.

[0063] S106: Split at least part of the operators.

[0064] Among the operators used in traditional methods, some operators do not process data in a coherent manner, which may cause delays or interruptions in calculations. Therefore, in this method, some operators with such characteristics can be split to improve the overall calculation efficiency.

[0065] Taking the Norm operator, a normalization operator used in the ResNet50 model, as an example, the Norm operator combines the two steps of reduction and broadcast. In the Reduce phase, the data of all nodes are aggregated into one result; in the Broadcast phase, this result is sent to all nodes to ensure that all nodes have the same copy of the data. For the Norm operator, its execution process will interrupt data transmission and calculation. In other words, when a part of the data is normalized, the data cannot be transmitted or other calculated. Therefore, in this method, the Norm operator can be specifically split to obtain the Norm_a operator and the Norm_b operator.

[0066] The Norm operator is split into two parts, the Norm_a operator and the Norm_b operator, wherein the Norm_a operator and the Norm_b operator are responsible for normalizing a part of the data respectively. Thus, when the normalization processing of one part of the data is performed, the transmission and calculation of the other part of the data will not be affected.

[0067] Furthermore, the Norm_a operator and the Norm_b operator can be further split at the data dimension level. The following takes the Norm_a operator as an example for explanation.

[0068] For element-wise operators, since they are dimension-independent, they can be split flexibly. However, operations like reduce require data to be unified between different cores, which will block other calculations and data transmission. Therefore, the splitting basis is mainly focused on the reduce operation.

[0069] Specifically, the Norm_a operator is split into norm_a_part1 and norm_a_part2. Among them, norm_a_part1 is used to calculate the sum of the E and F dimensions, while norm_a_part2 is responsible for the all-reduce operation, unifying the N dimensions in different cores. The purpose of this design is to complete some calculations before performing the all-reduce operation to reduce possible blockages in data transmission and calculation processes and improve overall computing efficiency.

[0070] Among them, the specific forms of norm_a_part1 and norm_a_part2 are as follows:

[0071] norm_a_part1:

[0072]

[0073] norm_a_part2:

[0074]

[0075] Through the above split design, the calculation of norm_a_part1 is carried out in parallel with the next round of Direct Memory Access (DMA) transfer, and the norm_b operator is carried out in parallel with the DMA transfer of the result writeback. Since the DMA transfer time is much longer than the calculation time, this design can effectively utilize the transmission waiting time to perform the calculation task. Under this split and parallel processing, the second round of calculation only needs to load M-dimensional data instead of N * E * F * M-dimensional data, which can effectively reduce the memory space required to cache the entire batch size data on-chip, thereby significantly reducing memory requirements.

[0076] This step can be used to split some operators for use in subsequent steps.

[0077] S106: According to the input, output and operation type of each operator, the operators are fused with the split operators to obtain a plurality of fused operators.

[0078] After the splitting of some operators is completed in step S104, in this step, the operators that have not been split and the operators split in step S104 may be merged.

[0079] When using traditional methods to perform operations in neural network models, every time an operator needs to be executed, the computing unit that executes the operator needs to retrieve the data required to execute the operator from the main memory; similarly, after completing the calculation of an operator, the calculation result of the operator needs to be stored back in the main memory. Even if the calculation result of the previous operator can be directly used as the input of the next operator, the above process still needs to be executed, that is, the previous operator stores the calculation result in the main memory, and the next operator takes out the calculation result and uses it as input. This feature causes a large number of unnecessary data access processes in the operation of the neural network model.

[0080] To solve the above problem, in this step, the method fuses each operator according to its input, output and operation type to reduce the data access process of the neural network model during the operation.

[0081] On the one hand, fusion is performed based on the input and output of the operator. Specifically, for any two operators, when the output of the adjacent and relatively forward operator in the neural network model can be used as the input of the relatively backward operator, the two operators can be fused. The above characteristics can be summarized as the transferability of the two operators in the use of data. It is not difficult to understand that when two adjacent operators are transferable in the use of data, the two operators can be fused into one operator to avoid the data access steps generated between the processes of executing the two operators. Furthermore, when this transferability exists between any two adjacent operators in multiple consecutive operators, the multiple consecutive operators can be fused into one operator.

[0082] On the other hand, the fusion can be performed according to the operation type of the operator. When the operation type of several consecutive operators is the same operation type, these operators can be fused into one operator. Among them, the operation type can at least include a convolution operation type and a vector operation type. In the operation of the neural network model, convolution operation and vector operation are operations on two different types of data, respectively, and the two cannot usually be executed crosswise. Therefore, when fusing operators, it is necessary to pay attention to fusing operators of convolution operation type and vector operation type separately.

[0083] In this method, generally, only when several operators satisfy the above two conditions at the same time will these operators be merged into one operator.

[0084] Still taking the ResNet50 model as an example, when this step is used to fuse the operators used in the model, when the Norm operator is split into the Norm_a operator and the Norm_b operator, the following seven fusion operators can be obtained during the forward propagation of the model, as follows:

[0085] Type1: (Reshape + CONV + Norm_a), this fusion operator integrates shape adjustment, convolution, and batch normalization (Norm_a) of the first part to reduce data transmission and improve execution efficiency.

[0086] Type2: (Norm_b + Relu + Max Pooling + CONV + Norm_a), this fusion operator integrates batch normalization (Norm_b), Relu activation, maximum pooling, convolution and Norm_a in the second part to optimize data flow and reduce latency.

[0087] Type3: (Norm_b + Relu + CONV + Norm_a), this fusion operator simplifies the data processing process and improves the processing speed by integrating Norm_b, Relu activation and convolution operations.

[0088] Type4: (CONV + Norm_a), this fusion operator is specifically for the batch normalization operation performed immediately after the convolution operation to reduce data movement between operations.

[0089] Type5: (Norm_b + Norm_b + ADD + Relu + CONV + Norm_a), this fusion operator integrates Norm_b, element-level addition, Relu activation, and convolution operations, providing a more efficient path for data flow.

[0090] Type6: (Norm_b + ADD + Relu + CONV + Norm_a), this fusion operator is specially designed to process the Norm_b operation in a parallel path, followed by element-wise addition, Relu activation, and convolution.

[0091] Type7: (Norm_b + ADD + Relu + Avg Pooling + FC + Softmax), this fusion operator combines Norm_b, element-level addition, Relu activation, average pooling, fully connected layer and Softmax operation, providing an efficient processing solution for the tail of the network.

[0092] In this manual, Figure 2a , Figure 2b , Figure 2c Together they form Figure 2. Figure 2 shows an operator structure of a target model when performing a forward operation when the target model is processed using the model operation optimization method provided in this specification. In this embodiment, the target model is still a ResNet50 model. Figure 2a This is a schematic diagram of the operator structure of the initial part of the target model. Figure 2b This is a schematic diagram of the operator structure in the middle part of the target model. Figure 2c It is a schematic diagram of the operator structure at the end of the target model. As shown in Figure 2, the small rectangular solid line box represents the original unfused operator, and the large rectangular dotted line box represents the operator fused by this method. It can be seen that all the operators originally existing in the target model are replaced by the fusion operator proposed in this method. As introduced in the background technology of this specification, there is a data access process between the execution processes of every two adjacent operators. If the traditional method is adopted, then under the target model framework shown in Figure 2, there is a data access process between every two small rectangular solid line boxes; and if the model operation optimization method provided in this specification is adopted, then only one data access process is required between every two large rectangular dotted line boxes. Obviously, this method greatly reduces the number of data accesses required.

[0093] At the same time, it is mentioned in the introduction of the Norm operator and the Norm_a operator and Norm_b operator obtained by splitting it that since the Norm operator will definitely cause interruptions in data transmission and calculation during execution, this method divides the data into two parts by splitting to perform normalization, so that the data can be transmitted and calculated alternately. On this basis, in this step, the design of the fusion operator starts with the Norm_b operator and ends with the Norm_a operator (except that the fusion operator 1 is the starting operator in the model, and the Norm_b operator is not required; the fusion operator 7 is the ending operator in the model, and the Norm_a operator is not required), so that the data access process and the normalization process are executed simultaneously. At the same time, the above design method integrates the Norm_b and Relu operations into a pipeline, which is executed in parallel with the DMA that writes the results back. Ensure the stability and accuracy of the data during the transmission process and simplify the complexity of the overall design and implementation.

[0094] The above-mentioned method of fusing operators is the method of fusing the target model during forward propagation. In addition, when the target model is in reverse propagation, the fusion of operators can also be performed according to similar logic. That is, when fusing each operator and the split operator, several fused operators for forward propagation and several fused operators for reverse propagation are obtained.

[0095] During the back propagation of the model, the following seven fusion operators can be obtained, as follows:

[0096] Type8: (Reshape + CONV + Norm_a), since Type8 is in the first layer of the network, there is no need for a reverse dx structure.

[0097] Type9: (Anti-Norm_b + CONV + ADD + Max Pooling + Anti-Relu + Anti-Norm_a)

[0098] Type10: (Anti-Norm_b + CONV + Anti-Relu + Anti-Norm_a)

[0099] Type11: (CONV + Anti-Norm_b)

[0100] Type12: (Anti-Norm_b + CONV + Anti-Relu + ADD + Anti-Norm_a | Anti-Norm_a)

[0101] Type13: (Anti-Norm_b + CONV + Anti-Relu + ADD + Anti-Norm_a)

[0102] Type14: (Anti-Norm_b + FC + Avg Pooling + Relu + ADD + Anti-Norm_a)

[0103] Among them, the operator with the Anti- prefix is ​​the reverse operator corresponding to the original operator. Corresponding to Norm_a and Norm_b obtained by splitting the Norm operator, in the process of back propagation, the Anti-Norm operator is also split into Anti-Norm_a operator and Anti-Norm_b operator, and the Anti-Norm_a operator can be further split into anti_norm_a_part1 operator and anti_norm_a_part2 operator, and the Anti-Norm_b operator can be further split into anti_norm_b_part1 operator and anti_norm_b_part2 operator.

[0104] anti_norm_a_part1 calculates sum(dy) and sum(dy * x_hat), where:

[0105] dy is the gradient of the input.

[0106] is the standardized input feature, and the calculation formula is:

[0107]

[0108] in, is the input feature, μ is the mean, σ is the standard deviation, It is the inverse of the standard deviation. The calculation formula is as follows:

[0109]

[0110] The corresponding two accumulation formulas are:

[0111]

[0112]

[0113]

[0114] anti_norm_a_part2 mainly performs global reduce operations, that is, globally accumulates and sums the results obtained from each core calculation, and finally calculates the mean and the square of the standard deviation. This can be expressed as:

[0115]

[0116] Variance calculation uses the Kahan summation algorithm to reduce floating point errors. The formula is:

[0117]

[0118] When using the Kahan summation algorithm, the variance calculation formula reduces floating point errors at each accumulation step.

[0119] From these formulas, we can see that anti_norm_a_part1 calculates the gradient and the related operations of the standardized input, while anti_norm_a_part2 performs the precise calculation of global synchronization and variance.

[0120] The Anti-Norm_b operator also consists of two parts: the anti_norm_b_part1 operator and the anti_norm_b_part2 operator. These two parts together complete the calculation of the input residual from the scaled gradient back to the scaled input residual during the back propagation process of Batch Normalization.

[0121] anti_norm_b_part1 calculates the intermediate variables a, b, and c in back propagation, which are used in the second part to calculate the data residual before scaling.

[0122] 1. The calculation of the intermediate variable a is:

[0123]

[0124] Among them, γ is the scaling factor of the channel, dγ is the gradient of the channel, σ is the standard deviation of the channel, and NormVal is a constant added for numerical stability.

[0125] 2. The calculation of the intermediate variable c is:

[0126]

[0127] This formula is used to restore the gradient to the unscaled data.

[0128] 3. The calculation of the intermediate variable b is:

[0129]

[0130] in, is the mean of the channel, and dβ is the gradient of the offset of the channel.

[0131] anti_norm_b_part2 uses the intermediate variables a, b, and c calculated in the first part, combined with the input data y and the back-propagated gradient dy, to calculate the data residual before scaling, that is, the final output gradient.

[0132] Among them, the output residual The calculation is:

[0133]

[0134] During the forward propagation process, the convolution operation of the current layer will generate an intermediate result Y. This result is usually normalized, and the normalized result Y' will be used as the input of the next layer to adjust the distribution of data and improve the training stability of the model. However, in this solution, the Norm operator is split into two parts, a and b, and distributed in two layers. The intermediate result Y will be used as the forward input of the next layer.

[0135] In back propagation, we need to calculate the gradient to update the model weights. The key point here is that due to the existence of the normalization operation, the input during back propagation is not the original Y of the current layer, but the normalized result of the next layer plus the results of other calculations. This normalized result will be passed to the previous layer in back propagation and used in the anti-normalization Anti-Norm_a step.

[0136] Because in the forward propagation, Y is processed by Norm_a before being passed to the next layer; in the backward propagation, we need to use the Y' stored in the next layer. However, since the output of the next layer has been normalized by norm_b, this leads to "misalignment" of the data. In other words, in the backward propagation, the current layer needs to obtain the Y' stored in the next layer in the forward propagation, not the original Y.

[0137] In order to deal with this misalignment, additional data storage is required in the model. Specifically, the model needs to save the results of each layer before and after normalization during forward propagation to ensure that these stored data can be correctly used for anti-normalization processing (Anti-Norm_a and Anti-Norm_b) during back propagation. This additional data storage is necessary to ensure the correctness of gradient calculation and weight update in back propagation.

[0138] S108: Using the fusion operator to execute the calculation function included in the target model.

[0139] After the fusion operation of the operators is completed in step S106, the obtained fusion operator can be finally used in this step to run the target model to complete the optimized operation.

[0140] When the model operation optimization method provided in this specification is used to optimize the operation of the target model, the operators can be split and fused according to the characteristics of the operators when processing data, and the fused operators are used to perform the operation functions in the target model. This method can greatly reduce the data access process that needs to be performed by the target model during the operation process, and effectively improve the operation efficiency of the target model.

[0141] Additionally, you may need to consider the storage requirements of the target model while the target model is running. In the ResNet50-V2 model, the storage requirements are mainly divided into weight storage and feature map storage. The following discussion is based on 32-bit single-precision floating-point numbers and takes ResNet50-V2 processing the ImageNet 2012 dataset (input size is 3224224) as an example for analysis.

[0142] The first thing to consider is the model's weight storage requirements. The weight layers in ResNet50-V2 mainly include convolutional layers, batch normalization layers, and fully connected layers. Since the batch normalization layer has relatively few weights, it is not the focus of discussion, but its weight number will be included in the total weight number.

[0143] The layer with the largest number of weights: such as the third layer, contains $K \times K \times M \times N$ parameters, where $K$ is the size of the convolution kernel, $M$ and $N$ are the number of input and output channels respectively, and requires $4 \times K \times K \times M \times N$ bytes of storage space.

[0144] The layer with the smallest number of weights: such as the Conv1 convolution layer, which contains $K_1 \times K_1 \times C \timesN_1$ parameters, where $K_1$ is the size of the convolution kernel, $C$ and $N_1$ are the number of input and output channels respectively, and requires $4 \times K_1 \times K_1 \times C \times N_1$ Bytes of storage space.

[0145] The total number of weights of the entire network: There are $P$ parameters in total, requiring a total of $4P$ Bytes of storage space.

[0146] Due to the large storage requirements for weights, it is recommended to transfer the weights of the convolutional layer to the chip in layers. For layers with a small number of weights, all of their weights can be transferred to the chip at once; for layers with a large number of weights, the weights need to be transferred to the chip in blocks.

[0147] In the weight storage requirement analysis, batch normalization parameters and bias parameters are not included. The details are as follows:

[0148] Bias parameter: The number of biases in each convolutional layer is equal to the number of convolution kernels, and the total number of biases is $Q$, or $4Q$Bytes.

[0149] Batch normalization parameters: The number of parameters of each batch normalization layer is twice the number of convolution kernels in this layer. The total batch normalization parameters are $2Q$, or $8Q$ Bytes.

[0150] Compared with weights, the number of batch normalization parameters and biases is small, so the batch normalization parameters and biases of each layer can be transferred to the chip.

[0151] At the same time, the storage requirements of the feature maps in the target model also need to be considered. The feature map with the largest storage requirement is the output feature map of the first layer, that is, the input feature map of the second layer, with a size of $N \times H_1 \times W_1 \times C_1$, requiring a total of $4 \times N \times H_1 \times W_1 \times C_1$ Bytes. The output feature maps of the 42nd, 43rd, 45th, 46th, 48th, and 49th layers, that is, the input feature maps of the 43rd, 44th, 46th, 47th, 49th, and 50th layers, with a size of $N \times H_2 \times W_2 \times C_2$, requiring a total of $4 \times N \times H_2\times W_2 \times C_2$ Bytes.

[0152] In a specific embodiment, the output feature map of each layer can be stored as the input feature map of the next layer. For a layer with a smaller feature map storage requirement, all feature maps of the layer can be stored on-chip; for a layer with a larger storage requirement, the feature map needs to be stored off-chip in blocks and transferred to the chip in blocks when the next layer is calculated.

[0153] The data flow between operators within a fusion operator only flows on the chip, while the data flow between adjacent fusion operators requires the data to be written from the chip to the outside first.

[0154] Furthermore, the calculation process of the fusion operator provided in this specification in the target model can be as follows.

[0155] Figure 3 This is a method for calculating the fusion operator during forward propagation provided in this manual, such as Figure 3 As shown, the calculation steps of the fusion operator during forward propagation are as follows:

[0156] Step 1: Input data reading and preprocessing:

[0157] 1. Off-chip read: Read input data x1 or x1 + x2 from main memory via DMA request.

[0158] 2. On-chip storage: Store the read data in the on-chip local data memory (SPM).

[0159] 3. On-chip processing: The vector computing unit reads the x1 and x2 data from the SPM and performs normb+relu operations to obtain the data to be convolved.

[0160] 4. On-chip writeback: Write the pre-processed data to be convolved back to SPM, and finally form conv_input data.

[0161] Step 2: Weight data management:

[0162] 1. Off-chip read: Read weight data from main memory via DMA request.

[0163] 2. On-chip storage: Store the read weight data into the on-chip SPM.

[0164] 3. On-chip multiplexing: After all weight data is stored on-chip, it can be reused between all NHW calculations without repeated reading.

[0165] Step 3: Convolution operation and normalization processing:

[0166] 1. On-chip reading: The matrix multiplication and convolution operation unit reads the conv_input data bNHW×CbNHW \times CbNHW×C and weight data C×bMC \times bMC×bM obtained after pre_conv processing from SPM.

[0167] 2. Convolution calculation: The matrix multiplication convolution operation unit performs convolution operation to obtain the convolution result y (shape is bNEF×bMbNEF \times bMbNEF×bM).

[0168] 3. Write out of chip: write the convolution result y back to the main memory.

[0169] 4. Normalization: The convolution result is processed by norm_a_part1 and then norm_a_part2 to obtain the final sigma and mean results.

[0170] Figure 4 This manual provides a method for calculating the fusion operator during back propagation, such as Figure 4 As shown, the calculation steps of the fusion operator during forward propagation are as follows:

[0171] Step 1: Read the residual of the upper layer network, the input feature map of the current layer network and the corresponding parameters to the on-chip storage unit:

[0172] 1. Off-chip reading: Read the residual data of the upper network, the input feature map data of the current network, and the corresponding parameters (such as weights and batch normalization parameters) from the main memory (GM) through DMA.

[0173] 2. On-chip storage: The above data is stored in the on-chip local data memory (SPM) for subsequent calculations.

[0174] Step 2: Perform reverse normalization (Anti_norm_b) calculation through the vector calculation unit:

[0175] 1. On-chip reading: The vector computation unit reads the input feature map and corresponding parameters from the on-chip SPM.

[0176] 2. Calculation: Perform reverse normalization (Anti_norm_b) calculation in the vector calculation unit to obtain the data to be convolved.

[0177] 3. On-chip storage: The calculated data to be convolved is stored back in the on-chip SPM.

[0178] 4. Off-chip write: Write the data to be convolved from SPM back to the main memory for subsequent dw calculations.

[0179] Step 3: The matrix multiplication and convolution operation unit directly obtains the data to be convolved from the chip and performs convolution calculations:

[0180] 1. On-chip reading: The matrix multiplication and convolution operation unit reads the data to be convolved from the on-chip SPM.

[0181] 2. Calculation: The matrix multiplication and convolution operation unit performs convolution calculation to obtain the convolution result.

[0182] 3. Off-chip write: write the convolution result back to the main memory.

[0183] Step 4: Addition and reverse activation (add_anti_relu) processing:

[0184] 1. On-chip read: read the convolution result and sec_da_i / a_i data from SPM.

[0185] 2. Calculation: Addition and inverse activation (add_anti_relu) are performed in the execution unit to obtain the dy_i result.

[0186] 3. On-chip storage: Store the dy_i result back to the on-chip SPM.

[0187] 4. Write out to chip: Write the dy_i result from SPM back to main memory and perform anti_norm_a_part1 processing.

[0188] Step 5: Process all dy_i data and perform anti_norm_a_part2 processing:

[0189] 1. On-chip read: read the dy_i data on the chip through a loop.

[0190] 2. Calculation: Perform anti_norm_a_part1 processing in the vector calculation unit.

[0191] 3. Write out to the chip: Write the processed dy_i data back to the main memory.

[0192] 4. Loop reading: After n and c loops, read and process all dy_i data (shape is [C / 32, N, H, W,32]).

[0193] 5. Calculation: Perform anti_norm_a_part2 processing to obtain the final d_gamma and d_beta results.

[0194] 6. Write out to chip: Write the d_gamma and d_beta results back to main memory.

[0195] Figure 5 This manual provides a method for calculating the weight fusion operator used to calculate the parameter weights in the target model during back propagation, such as Figure 5 As shown, the calculation steps of the fusion operator during forward propagation are as follows:

[0196] Step 1: Input data reading and on-chip storage:

[0197] 1. Off-chip read: read input data from main memory through DMA request.

[0198] 2. On-chip storage: Store the read dy data and x data into the local data memory on the chip.

[0199] Step 2: Convolution operation:

[0200] 1. On-chip read: read dy data and x data from SPM.

[0201] 2. Calculation: Perform convolution operation, the convolution result is dw.

[0202] Step 3: Reduce operation:

[0203] 1. On-chip Reduce: Perform reduce operations on the convolution result dw between slave cores in the same row to obtain the final dw.

[0204] Step 4: Update and write back:

[0205] 1. Update operation: Perform an update operation to prepare to write back the updated w_fp32 and w_fp16 data.

[0206] 2. Off-chip write: Write the updated w_fp32 and w_fp16 data back to the main memory.

[0207] The above is the model operation optimization method provided in this specification. Based on the same idea, this specification also provides a corresponding model operation optimization device, such as Figure 6 shown.

[0208] Figure 6 A schematic diagram of a model operation optimization device provided in this specification specifically includes:

[0209] An acquisition module 200 is used to acquire a target model and determine the operation functions included in the target model;

[0210] A determination module 202, configured to determine an operator required to perform the operation function according to the operation function;

[0211] A splitting module 204, configured to split at least part of the operators in each operator;

[0212] A fusion module 206, configured to fuse the operators with the split operators according to the input, output and operation type of the operators to obtain a plurality of fused operators;

[0213] The execution module 208 is used to use the fusion operator to execute the operation function included in the target model.

[0214] Optionally, the target model is a ResNet50 convolutional neural network model.

[0215] Optionally, the operators include at least a Reshape operator, a CONV operator, a Norm operator, a Relu operator, a MaxPooling operator, an ADD operator, an Avg Pooling operator, a FC operator, and a Softmax operator.

[0216] Optionally, the splitting module 204 is specifically configured to split the Norm operator to obtain a Norm_a operator and a Norm_b operator.

[0217] Optionally, the fusion module 206 is specifically used to fuse the Reshape operator, CONV operator, and Norm_a operator to obtain a first fusion operator; and to fuse the Norm_b operator, Relu operator, and Max operator. The Pooling operator, the CONV operator, and the Norm_a operator are fused to obtain a second fused operator; the Norm_b operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a third fused operator; the CONV operator and the Norm_a operator are fused to obtain a fourth fused operator; the Norm_b operator, the ADD operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a fifth fused operator, wherein the Norm_b operator is used twice; the Norm_b operator, the ADD operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a sixth fused operator; the Norm_b operator, the ADD operator, the Relu operator, the AvgPooling operator, the FC operator, and the Softmax operator are fused to obtain a seventh fused operator.

[0218] Optionally, the operation type includes at least a convolution operation type and a vector operation type.

[0219] Optionally, the fusion module 206 is specifically used to obtain a plurality of fusion operators for forward propagation and a plurality of fusion operators for backward propagation.

[0220] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides model operation optimization methods.

[0221] This manual also provides Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0222] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0223] The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91 matrix multiplication and convolution operation unit M, Microchip PIC18F26K20, and SiliconeLabs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0224] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0225] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0226] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0227] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0228] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0229] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0230] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0231] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0232] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0233] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0234] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0235] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0236] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0237] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A model operation optimization method, characterized in that: include: Acquire a target model and determine a computing function included in the target model, wherein the target model is built on a general purpose computing graphics processing unit (GPGPU); According to the computing function, determining an operator required to perform the computing function; Splitting at least some of the operators; According to the input, output and operation type of each operator, each operator is fused with the split operators to obtain a plurality of fused operators; Using the fusion operator to execute the computing function included in the target model; The operators include forward operators and reverse operators, wherein the forward operators include at least Reshape operator, CONV operator, Norm operator, Relu operator, Max Pooling operator, ADD operator, Avg Pooling operator, FC operator, Softmax operator, and the reverse operators include at least Anti-Norm operator and Anti-Relu operator; The Norm operator and the Anti-Norm operator are used to perform a Reduce operation function and a Broadcast operation function; At least part of the operators in each operator is split, specifically including: Splitting the Norm operator to obtain a Norm_a operator and a Norm_b operator, and splitting the Auti-Norm operator to obtain an Anti-Norm_a operator and an Anti-Norm_b operator; The operators are fused with the split operators to obtain several fused operators, including: The Reshape operator, the CONV operator, and the Norm_a operator are fused to obtain a first fused operator; The Norm_b operator, the Relu operator, the Max Pooling operator, the CONV operator, and the Norm_a operator are fused to obtain a second fused operator; The Norm_b operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a third fused operator; The CONV operator and the Norm_a operator are fused to obtain a fourth fused operator; The Norm_b operator, the ADD operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a fifth fused operator, wherein the Norm_b operator is used twice; The Norm_b operator, the ADD operator, the Relu operator, the CONV operator, and the Norm_a operator are fused to obtain a sixth fused operator; The Norm_b operator, the ADD operator, the Relu operator, the Avg Pooling operator, the FC operator, and the Softmax operator are fused to obtain a seventh fusion operator; The Reshape operator, the CONV operator, and the Norm_a operator are fused to obtain an eighth fusion operator; The Anti-Norm_b operator, the CONV operator, the ADD operator, the Max Pooling operator, the Anti-Relu operator, and the Anti-Norm_a operator are fused to obtain a ninth fusion operator; The Anti-Norm_b operator, the CONV operator, the Anti-Relu operator, and the Anti-Norm_a operator are fused to obtain a tenth fused operator; The CONV operator and the Anti-Norm_b operator are fused to obtain an eleventh fused operator; The Anti-Norm_b operator, the CONV operator, the Anti-Relu operator, the ADD operator, and the Anti-Norm_a | Anti-Norm_a operator are fused to obtain a twelfth fused operator; The Anti-Norm_b operator, the CONV operator, the Anti-Relu operator, the ADD operator, and the Anti-Norm_a operator are fused to obtain a thirteenth fused operator; The Anti-Norm_b operator, the FC operator, the Avg Pooling operator, the Relu operator, the ADD operator, and the Anti-Norm_a operator are fused to obtain a fourteenth fused operator.

2. The method according to claim 1, characterized in that The GPGPU at least includes a main memory GM, an on-chip storage unit SPM, a register, a direct memory access DMA, a matrix multiplication and convolution operation unit, a vector calculation unit, and a special function calculation unit.

3. The method according to claim 2, characterized in that Several fusion operators are obtained, including: Several fusion operators for forward propagation and several fusion operators for backward propagation are obtained.

4. The method according to claim 3, characterized in that The processing steps of the fusion operator for forward propagation include: The SPM obtains input data and weight data from the GM via the DMA; The vector calculation unit reads the input data stored in the SPM, preprocesses the input data to obtain forward data to be convolved, and returns the forward data to be convolved to the SPM; The matrix multiplication and convolution operation unit reads the forward data to be convolved and the weight data stored in the SPM, calculates the forward data to be convolved according to the weight data, obtains a forward convolution result, and stores the forward convolution result in the GM; The forward convolution result is normalized.

5. The method according to claim 3, characterized in that The processing steps of the fusion operator for back-propagation include: For each network layer in the target model, read the residual data of the upper network layer of the network layer and the input feature map data and parameters of the network layer from the GM through the DMA; Storing the residual data, the input feature map data and the parameters in the SPM; The vector calculation unit reads the input feature map data and the parameters from the SPM, performs inverse normalization processing on the input feature map data according to the parameters to obtain inverse convolution data, and returns the inverse convolution data to the SPM; The matrix multiplication and convolution operation unit reads the inverse data to be convolved from the SPM, performs operation on the inverse data to be convolved, obtains an inverse convolution result, and returns the inverse convolution result to the SPM; the special function calculation unit reads the inverse convolution result from the SPM, performs inverse activation processing on the inverse convolution result, obtains an inverse activation result, and returns the inverse activation result to the SPM; Each reverse activation result is processed to obtain a reverse propagation parameter, and the reverse propagation parameter is stored in the GM.

6. A model operation optimization device, characterized in that: include: An acquisition module, used for acquiring a target model and determining a computing function included in the target model, wherein the target model is built on a general purpose computing graphics processing unit GPGPU; A determination module, used to determine an operator required to perform the operation function according to the operation function; A splitting module, used for splitting at least part of the operators in each operator; A fusion module, used for fusing the operators with the split operators according to the input, output and operation type of the operators to obtain a plurality of fused operators; An execution module, used for executing the operation function included in the target model by using the fusion operator; The operators include forward operators and reverse operators, wherein the forward operators include at least Reshape operator, CONV operator, Norm operator, Relu operator, Max Pooling operator, ADD operator, Avg Pooling operator, FC operator, Softmax operator, and the reverse operators include at least Anti-Norm operator and Anti-Relu operator; The Norm operator and the Anti-Norm operator are used to perform a Reduce operation function and a Broadcast operation function; The splitting module is specifically used to split the Norm operator to obtain the Norm_a operator and the Norm_b operator, and split the Auti-Norm operator to obtain the Anti-Norm_a operator and the Anti-Norm_b operator; The fusion module is specifically used to fuse the Reshape operator, CONV operator, and Norm_a operator to obtain a first fusion operator; fuse the Norm_b operator, Relu operator, Max Pooling operator, CONV operator, and Norm_a operator to obtain a second fusion operator; fuse the Norm_b operator, Relu operator, CONV operator, and Norm_a operator to obtain a third fusion operator; fuse the CONV operator and Norm_a operator to obtain a fourth fusion operator; fuse the Norm_b operator, ADD operator, Relu operator, CONV operator, and Norm_a operator to obtain a fifth fusion operator, wherein the Norm_b operator is used twice; fuse the Norm_b operator, ADD operator, Relu operator, CONV operator, and Norm_a operator to obtain a sixth fusion operator; fuse the Norm_b operator, ADD operator, Relu operator, Avg The Pooling operator, the FC operator, and the Softmax operator are fused to obtain the seventh fused operator; the Reshape operator, the CONV operator, and the Norm_a operator are fused to obtain the eighth fused operator; the Anti-Norm_b operator, the CONV operator, the ADD operator, the Max Pooling operator, the Anti-Relu operator, and the Anti-Norm_a operator are fused to obtain the ninth fused operator; the Anti-Norm_b operator, the CONV operator, the Anti-Relu operator, and the Anti-Norm_a operator are fused to obtain the tenth fused operator; the CONV operator and the Anti-Norm_b operator are fused to obtain the eleventh fused operator; the Anti-Norm_b operator, the CONV operator, the Anti-Relu operator, the ADD operator, and the Anti-Norm_a operator are fused to obtain the eleventh fused operator. The Anti-Norm_a operator is fused to obtain a twelfth fused operator; the Anti-Norm_b operator, CONV operator, Anti-Relu operator, ADD operator, and Anti-Norm_a operator are fused to obtain a thirteenth fused operator; the Anti-Norm_b operator, FC operator, AvgPooling operator, Relu operator, ADD operator, and Anti-Norm_a operator are fused to obtain a fourteenth fused operator.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images