Processing method and device of neural network model, equipment and storage medium

Abstract

The application relates to a neural network model processing method and device, equipment and a storage medium. The method comprises the following steps: calculating input channel to-be-filled values and output channel to-be-filled values of a target operator of a to-be-processed neural network model; storing the input channel to-be-filled values and the output channel to-be-filled values of the target operator in an input channel mask array and an output channel mask array of the target operator in the form of masks; performing a parameter filling operation on the target operator according to the input channel mask array and the output channel mask array storing the to-be-filled values; inserting a filling operator and / or a clipping operator into the to-be-processed neural network model according to the target operator after the parameter filling operation is performed, to obtain a target neural network model. The application can avoid inserting unnecessary operators into the target neural network model, and improve the running efficiency of the target neural network model on an AI special chip.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for processing neural network models. Background Technology

[0002] With the rapid development of deep learning, neural network models have been widely applied in machine vision. The resulting emergence of dedicated AI chips has significantly accelerated these models, making them run much faster than models on general-purpose chips. However, this acceleration comes at the cost of versatility. Neural network models trained on general-purpose chips cannot be directly run on dedicated AI chips; software optimization and conversion are required to adapt them for these specialized devices.

[0003] Currently, existing technologies for converting neural network models simply involve padding or pruning the operators within the model without considering the global nature of the neural network. When there are many non-convolutional operators in the model, excessive data padding is inserted, resulting in too many unnecessary operators being inserted into the neural network model and reducing its operating efficiency on AI-dedicated chips. Summary of the Invention

[0004] In view of the above, this application provides a method, apparatus, device and storage medium for processing neural network models, the purpose of which is to solve the above-mentioned technical problems.

[0005] Firstly, this application provides a method for processing a neural network model, the method comprising:

[0006] Calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed;

[0007] The input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are stored in the input channel mask array and the output channel mask array of the target operator in the form of masks, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays.

[0008] Based on the input channel mask array and output channel mask array storing the values ​​to be filled, global information of the neural network model to be processed is obtained, and parameter filling operation is performed on the target operator based on the global information;

[0009] Based on the target operator after the parameter filling operation, a filling operator and / or a pruning operator are inserted into the neural network model to be processed to obtain the target neural network model.

[0010] Secondly, this application provides a processing apparatus for a neural network model, the processing apparatus comprising:

[0011] Calculation module: used to calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed;

[0012] Storage module: used to store the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator in the form of masks in the input channel mask array and the output channel mask array of the target operator, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays.

[0013] Filling module: used to obtain global information of the neural network model to be processed based on the input channel mask array and the output channel mask array which store the values ​​to be filled, and to perform parameter filling operation on the target operator based on the global information;

[0014] Insertion module: Used to insert filling operators and / or pruning operators into the neural network model to be processed based on the target operator after the execution parameter filling operation to obtain the target neural network model.

[0015] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0016] Memory, used to store computer programs;

[0017] When a processor executes a program stored in memory, it implements the steps of the processing method for the neural network model described in any embodiment of the first aspect.

[0018] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the processing method for a neural network model as described in any embodiment of the first aspect.

[0019] The technical solutions provided in this application have the following advantages compared with the prior art:

[0020] Since the masking mechanism can not only store the location information of valid channel data, but also perform mask value calculations according to the characteristics of operators, this application stores the input channel and output channel values ​​to be filled of the target operator in the form of masks in the input channel mask array and output channel mask array of the target operator, respectively. This allows the channel mask information to be passed down, meaning that after calculating the value to be filled for a certain target operator, the data to be filled for the next target operator can be obtained without actually filling that target operator, thus covering all operators of the neural network model. Having obtained the data to be filled for all target operators, parameter filling operations are performed on the target operators according to the input channel mask array and output channel mask array. Based on the target operators after parameter filling operations, filling operators and / or pruning operators are inserted into the neural network model to be processed. This fully utilizes the global information of the neural network model to obtain the target neural network model, avoiding the insertion of unnecessary operators into the target neural network model. Without changing the original structure of the neural network, it adapts to AI-specific chips and does not affect their acceleration effect, thereby improving the running efficiency of the target neural network model on AI-specific chips. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a preferred embodiment of the neural network model processing method of this application;

[0024] Figure 2 This is a schematic diagram illustrating the process of the neural network model processing method in Embodiment 1 of this application;

[0025] Figure 3 This is the computation graph structure of another neural network model to be processed in an embodiment of this application;

[0026] Figure 4 for Figure 3 A schematic diagram illustrating the representation of the channel mask array of operators in the neural network model to be processed;

[0027] Figure 5 This is a schematic diagram of a preferred embodiment of the processing device for the neural network model of this application;

[0028] Figure 6 This is a schematic diagram of a preferred embodiment of the electronic device of this application;

[0029] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0031] It should be noted that the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0032] This application provides a method for processing neural network models. (Refer to...) Figure 1 The diagram shown is a flowchart illustrating an embodiment of the neural network model processing method of this application. This method can be executed by an electronic device, which can be implemented in software and / or hardware. The neural network model processing method includes:

[0033] Step S10: Calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed;

[0034] Step S20: Store the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator in the form of masks in the input channel mask array and the output channel mask array of the target operator, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays.

[0035] Step S30: Obtain global information of the neural network model to be processed based on the input channel mask array and the output channel mask array that store the values ​​to be filled, and perform parameter filling operation on the target operator based on the global information;

[0036] Step S40: Based on the target operator after the parameter filling operation, insert the filling operator and / or pruning operator into the neural network model to be processed to obtain the target neural network model.

[0037] In this embodiment, the neural network model to be processed refers to a neural network model trained on a general-purpose chip that needs to be adapted and run on an AI-dedicated chip. The neural network model contains multiple operators, such as convolution operators, pooling operators, and splicing operators. If the neural network model trained on the general-purpose chip is adapted and run on the AI-dedicated chip, the input and output channels of the target operators in the neural network model need to be filled with data. The target operators refer to the operators in the neural network model that need to have their channels filled with data.

[0038] After obtaining the neural network model to be processed, the input channel and output channel values ​​to be filled for the target operator of the neural network model are calculated. Since most AI-specific chips require the neural network model's operator to have 2 input and 2 output channels... n (The value of n may vary depending on the specific AI chip used.) Assuming n is 6, the number of input and output channels of the neural network model's operators needs to be padded to a multiple of 64. Specifically, calculating the input channel and output channel padded values ​​of the target operator of the neural network model to be processed includes:

[0039] Obtain the target values ​​of the input channel and the target value of the output channel of the target operator of the neural network model to be processed;

[0040] Obtain the initial values ​​of the input channel and the initial values ​​of the output channel of the target operator;

[0041] Based on the topological order of the neural network model to be processed, the target values ​​of the input channel and output channel of the target operator, and the initial values ​​of the input channel and output channel of the target operator, the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are calculated.

[0042] The target values ​​for the input and output channels of the target operator refer to the number of input and output channels of the operator in the neural network model required by the AI-specific chip. For example, if the AI-specific chip requires the neural network model to have 2 input and 2 output channels... 5 Then the target value of both the input channel and the output channel of the target operator is 2. 5The initial values ​​of the input and output channels of the target operator refer to the input and output channel values ​​of the target operator in the neural network model after training on a general-purpose chip. For example, if the input and output channel values ​​of a certain target operator are 32 and 3 respectively after training on a general-purpose chip, then the initial values ​​of the input and output channels of the target operator are 32 and 3 respectively. Based on the topological order of the neural network model to be processed, the target values ​​of the input and output channels of the target operator, and the initial values ​​of the input and output channels of the target operator, the input channel filling value and output channel filling value of the target operator can be calculated. The input channel filling value refers to the data length that needs to be filled in the input channel of the target operator, and the output channel filling value refers to the data length that needs to be filled in the output channel of the target operator.

[0043] Further, the step of calculating the input channel to be filled value and the output channel to be filled value of the target operator based on the topological order of the neural network model to be processed, the target value of the input channel and the target value of the output channel of the target operator, and the initial value of the input channel and the initial value of the output channel of the target operator, includes:

[0044] According to the topological order of the neural network model to be processed, the difference between the target value of the input channel of the target operator and the initial value of the input channel of the target operator are calculated sequentially, and used as the value to be filled in the input channel of the target operator;

[0045] Based on the topological order of the neural network model to be processed, the difference between the target value of the output channel of the target operator and the initial value of the output channel of the target operator are calculated sequentially, and used as the value to be filled in the output channel of the target operator.

[0046] Since the neural network model to be processed typically has multiple target operators, the topological order of the model ensures that the input channel and output channel values ​​to be filled for each target operator are calculated. The difference between the target value of the input channel and its initial value is calculated sequentially according to the topological order, and this difference is used as the input channel value to be filled. Similarly, the difference between the target value of the output channel and its initial value is calculated sequentially according to the topological order, and this difference is used as the output channel value to be filled. For example, if the target value of the input channel is 64 and the initial value is 3, then the input channel value to be filled is 61. If the target value of the output channel is 64 and the initial value is 32, then the output channel value to be filled is 32.

[0047] The input and output channel values ​​to be filled for the target operator are stored as masks in the target operator's input channel mask array and output channel mask array, respectively. The masking mechanism uses 0s and 1s to represent whether the current data is valid. This mechanism not only stores the location information of valid channel data but also performs mask value calculations based on the characteristics of the operator, ensuring that the channel mask information is passed down and thus covering all operators in the neural network model. The target operator's input and output channel mask arrays are pre-initialized empty arrays; the input channel mask array is denoted as `in_channel_mask`, and the output channel mask array is denoted as `out_channel_mask`. Since a mask can indicate whether the current data is valid, storing the input channel and output channel values ​​to be filled using a mask can represent the valid and invalid positions of the input and output channels. After calculating the value to be filled for a certain target operator, the data to be filled for the next target operator can be obtained without actually filling that target operator. This yields the data to be filled for all target operators, i.e., the global information of the neural network model to be processed. Then, relevant filling operators and / or pruning operators can be inserted based on the global information.

[0048] Based on the input channel mask array and output channel mask array storing the values ​​to be filled, global information of the neural network model to be processed is obtained. Based on this global information, parameter padding is performed on the target operator. Taking a convolution operator as an example, the weights of the convolution operator are [F, C, H, W], where F is the initial value of the output channel, C is the initial value of the input channel, and H and W represent the width and height of the weights, respectively. If F and C do not satisfy a power of 2, they need to be padded with 0s to make them powers of 2. If the weights are [32, 3, 7, 7], and n is 6, then 32 and 3 need to be padded to 64, resulting in a padded weight of [64, 64, 7, 7]. The padding method is to append data to the end of the original data. For example, if an array represents data with 3 original input channels [1, 1, 1], and it is padded to 6 channels, the result after padding is [1, 1, 1, 0, 0, 0], where 0 represents the padded data.

[0049] In one embodiment, obtaining global information of the neural network model to be processed based on an input channel mask array and an output channel mask array storing the values ​​to be filled, and performing parameter filling operation on the target operator based on the global information, includes:

[0050] The weight data of the target operator is initialized as a zero matrix containing the target values ​​of the input channel and the target values ​​of the output channel;

[0051] By traversing the input channel target values ​​and output channel target values ​​in the zero matrix, and based on the input channel mask array and output channel mask array that store the values ​​to be filled, the global information of the neural network model to be processed is obtained;

[0052] The number of input channels and the number of output channels in the zero matrix are filled according to the global information.

[0053] The target operator's weight and bias are filled using the input channel mask array and output channel mask array that store the values ​​to be filled. Taking the weight data as an example, the target operator's weight data weight[F, C, H, W] (F represents the number of output channels, C represents the number of input channels, and H and W represent the length and width of the weight, respectively) is initialized as a zero matrix [F`, C`, H, W]. F` is the target value of the input channel, and C` is the target value of the output channel. That is, F` and C` are the sizes after filling. F` and C` are traversed, and the global information of the neural network model to be processed is obtained based on the input channel mask array and output channel mask array that store the values ​​to be filled. The global information is used to perform the filling operation on the number of input channels and the number of output channels in the zero matrix. If the mask array is 1, the original value is filled into the channels of the zero matrix; if the mask array is 0, it is skipped. For example, suppose the number of input channels in the zero matrix is ​​represented as [0, 0, 0, 0, 0, 0], that is, the target value of the input channels in the zero matrix is ​​6. If the mask array is [1, 1, 1, 0, 0, 0], then the result after filling is [1, 1, 1, 0, 0, 0].

[0054] Based on the target operator after the parameter filling operation, the filling operator and / or pruning operator are inserted into the neural network model to be processed to obtain the target neural network model. Since the number of input and output channels of the operators in the target neural network meets the operating requirements of the AI ​​professional chip, it can be adapted to run on the AI ​​professional chip without changing the original structure of the neural network model and without affecting its acceleration effect.

[0055] In one embodiment, inserting a padding operator and / or a pruning operator into the neural network model to be processed based on the target operator after the parameter padding operation includes:

[0056] If the input channel mask array of the target operator after the parameter filling operation is not empty, and the input node corresponding to the target operator is a non-target operator that does not require parameter filling operation, then a filling operator is inserted between the target operator and the non-target operator.

[0057] If the input channel mask array of the target operator after the parameter padding operation is not empty (i.e., the target operator has an in_channel_mask), and the input node corresponding to the target operator (i.e., the operator of the previous node of the target operator) is a non-target operator that does not need padding (i.e., an operator without an in_channel_mask array), then a padding operator (i.e., a Pad operator) is inserted between the two, and padding is performed only in the channel dimension, and the length of padding is the total number of 0s in the in_channel_mask array.

[0058] In one embodiment, inserting a padding operator and / or a pruning operator into the neural network model to be processed based on the target operator after the parameter padding operation includes:

[0059] If the output channel mask array of the target operator after the parameter filling operation is not empty, and the output node corresponding to the target operator is a non-target operator that does not require the parameter filling operation, then a pruning operator is inserted between the target operator and the non-target operator.

[0060] If the output channel mask array of the target operator after the parameter padding operation is not empty (i.e., the target operator has an out_channel_mask), and the output node corresponding to the target operator (i.e., the operator of the next node of the target operator) is a non-target operator that does not need padding, then the insertion will insert a cropping operator (i.e., a crop operator) between the two. The crop operation can traverse the out_channel_mask. If the out_channel_mask[i] value of the operator is 1, then the output channel data is retained; if it is 0, then the channel data is discarded.

[0061] The processing method for the aforementioned neural network model will be further explained using a network containing only convolution operators as an example. (Refer to...) Figure 2 The diagram shown illustrates the processing method for a neural network model containing only convolution operators in an embodiment of this application. Input represents the input data, and Conv2D represents the convolution operation. Figure 2 The left part is the computation graph structure of the network, and the middle part is the representation of the channel mask array of the operator in the network. [1]*32 means there are 32 ones, the plus sign means to merge the values ​​of two arrays, and [1]*32+[0]*32 means that there are 32 valid values ​​in the first 34 channels of a length of 64, and 32 padding values ​​in the last 34 channels. Figure 2 The right side of the image shows the target neural network model after the pad operator (i.e., the pad operator) has been inserted. Using masks to represent the input and output channels of each operator facilitates global analysis.

[0062] The processing method for the aforementioned neural network model is further illustrated using a network containing convolution, pooling, and concatenation operators. (Refer to...) Figure 3 The diagram shows the computational graph structure of the neural network model to be processed. `Conv2D` represents the convolution operation, and `MaxPool` is the pooling operator (maximum pooling operation), which does not change the channel values ​​of the input data. `Concat` is the concatenation operator (concatenating data according to a specified dimension), which may change the channel values ​​of the input data. For example, concatenating dimension 1 (channel dimension). In this case, the traditional approach is to insert a `Crop` operation into the input of `Concat`, otherwise the calculation result will be incorrect. However, using the processing method of this application, the `Crop` operation on the input of `Concat` can be subtracted, and the `Crop` operation is only performed on the final output. Although... Figure 3 The Concat operation in this paper only accepts two inputs, but it should be noted that the number of inputs for Concat is unlimited. Using the processing method of this application, the Crop operation of Concat can be consistently reduced to only one. Specifically, Figure 4 for Figure 3 The representation of the channel mask array of operators in the network is such that, since MaxPool does not change the input channel value, it only needs to inherit the value of the input out_channel_mask, and the values ​​of in_channel_mask and out_channel_mask are the same.

[0063] Concat performs data concatenation based on the specified dimensions. Since this case involves concatenation along the channel dimension, its in_channel_mask is obtained by concatenating the input out_channel_mask in the input order. Assuming the input order is from left to right, the in_channel_mask value of Concat is [1]*16+[0]*48+[1]*32+[0]*32. Because channel_mask can represent the positional relationship between valid data and padding data, it is only necessary to perform a Crop operation on the output of Concat, and then iterate through out_channel_mask. If the value of out_channel_mask[i] is 1, the output channel data of Concat is retained; if the value of out_channel_mask[i] is 0, its output channel data is discarded to obtain the correct output. This can reduce unnecessary Crop operators and improve the running efficiency of the target neural network model on AI-dedicated chips.

[0064] Reference Figure 5 The diagram shown is a functional block diagram of the processing device 100 for the neural network model of this application.

[0065] The neural network model processing device 100 described in this application can be installed in an electronic device. Depending on the functions implemented, the neural network model processing device 100 may include a calculation module 110, a storage module 120, a filling module 130, and an insertion module 140. The module described in this application can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0066] In this embodiment, the functions of each module / unit are as follows:

[0067] Calculation module 110: used to calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed;

[0068] Storage module 120: used to store the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator in the form of masks in the input channel mask array and the output channel mask array of the target operator, respectively, wherein the input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays;

[0069] Filling module 130: used to obtain global information of the neural network model to be processed based on the input channel mask array and the output channel mask array storing the values ​​to be filled, and to perform parameter filling operation on the target operator based on the global information;

[0070] Insertion module 140: used to insert a padding operator and / or a pruning operator into the neural network model to be processed according to the target operator after the execution parameter padding operation to obtain the target neural network model.

[0071] In one embodiment, calculating the input channel fill values ​​and output channel fill values ​​of the target operator of the neural network model to be processed includes:

[0072] Obtain the target values ​​of the input channel and the target value of the output channel of the target operator of the neural network model to be processed;

[0073] Obtain the initial values ​​of the input channel and the initial values ​​of the output channel of the target operator;

[0074] Based on the topological order of the neural network model to be processed, the target values ​​of the input channel and output channel of the target operator, and the initial values ​​of the input channel and output channel of the target operator, the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are calculated.

[0075] In one embodiment, calculating the input channel to be filled value and the output channel to be filled value of the target operator based on the topological order of the neural network model to be processed, the target values ​​of the input channel and the target values ​​of the output channel of the target operator, and the initial values ​​of the input channel and the initial values ​​of the output channel of the target operator includes:

[0076] According to the topological order of the neural network model to be processed, the difference between the target value of the input channel of the target operator and the initial value of the input channel of the target operator are calculated sequentially, and used as the value to be filled in the input channel of the target operator;

[0077] Based on the topological order of the neural network model to be processed, the difference between the target value of the output channel of the target operator and the initial value of the output channel of the target operator are calculated sequentially, and used as the value to be filled in the output channel of the target operator.

[0078] In one embodiment, obtaining global information of the neural network model to be processed based on an input channel mask array and an output channel mask array storing the values ​​to be filled, and performing parameter filling operation on the target operator based on the global information, includes:

[0079] The weight data of the target operator is initialized as a zero matrix containing the target values ​​of the input channel and the target values ​​of the output channel;

[0080] By traversing the input channel target values ​​and output channel target values ​​in the zero matrix, and based on the input channel mask array and output channel mask array that store the values ​​to be filled, the global information of the neural network model to be processed is obtained;

[0081] The number of input channels and the number of output channels in the zero matrix are filled according to the global information.

[0082] In one embodiment, inserting a padding operator and / or a pruning operator into the neural network model to be processed based on the target operator after the parameter padding operation includes:

[0083] If the input channel mask array of the target operator after the parameter filling operation is not empty, and the input node corresponding to the target operator is a non-target operator that does not require parameter filling operation, then a filling operator is inserted between the target operator and the non-target operator.

[0084] In one embodiment, inserting a padding operator and / or a pruning operator into the neural network model to be processed based on the target operator after the parameter padding operation includes:

[0085] If the output channel mask array of the target operator after the parameter filling operation is not empty, and the output node corresponding to the target operator is a non-target operator that does not require the parameter filling operation, then a pruning operator is inserted between the target operator and the non-target operator.

[0086] Reference Figure 6 The diagram shown is a schematic diagram of a preferred embodiment of the electronic device 1 of this application.

[0087] The electronic device 1 includes, but is not limited to, a memory 11, a processor 12, a display 13, and a communication interface 14. The electronic device 1 can connect to a network via the communication interface 14. The network can be an intranet, the Internet, a Global System for Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA) network, a 4G network, a 5G network, Bluetooth, Wi-Fi, a voice communication network, or other wireless or wired networks.

[0088] The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device 1, such as the hard disk or memory of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped with the electronic device 1. Of course, the memory 11 may include both the internal storage unit and its external storage device of the electronic device 1. In this embodiment, the memory 11 is typically used to store the operating system and various computer programs installed on the electronic device 1, such as the program code of the neural network model processing program 10. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or will be output.

[0089] In some embodiments, processor 12 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 12 is typically used to control the overall operation of the electronic device 1, such as performing data interaction or communication-related control and processing. In this embodiment, processor 12 is used to run program code stored in memory 11 or process data, such as running the program code of a neural network model processing program 10.

[0090] The display 13 may be referred to as a display screen or display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch liquid crystal display, or an organic light-emitting diode (OLED) touch screen, etc. The display 13 is used to display information processed in the electronic device 1 and to display a visual working interface.

[0091] The communication interface 14 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface), which is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0092] Figure 6 Only an electronic device 1 with components 11-14 and a neural network model is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0093] In the above embodiments, when the processor 12 executes the processing program 10 of the neural network model stored in the memory 11, it can perform the following steps:

[0094] Calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed;

[0095] The input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are stored in the input channel mask array and the output channel mask array of the target operator in the form of masks, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays.

[0096] Based on the input channel mask array and output channel mask array storing the values ​​to be filled, global information of the neural network model to be processed is obtained, and parameter filling operation is performed on the target operator based on the global information;

[0097] Based on the target operator after the parameter filling operation, a filling operator and / or a pruning operator are inserted into the neural network model to be processed to obtain the target neural network model.

[0098] The storage device can be the memory 11 of the electronic device 1, or it can be other storage devices that are communicatively connected to the electronic device 1.

[0099] For a detailed explanation of the above steps, please refer to the above. Figure 5 Functional block diagram of an embodiment of a neural network model processing device 100 and Figure 1 This document describes a flowchart illustrating an embodiment of a method for processing neural network models.

[0100] Furthermore, this application embodiment also proposes a computer-readable storage medium, which can be non-volatile or volatile. This computer-readable storage medium can be any one or any combination of several of the following: hard disk, multimedia card, SD card, flash memory card, SMC, read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area. The program storage area stores a processing program 10 for a neural network model. When the neural network model processing program 10 is executed by a processor, it performs the following operations:

[0101] Calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed;

[0102] The input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are stored in the input channel mask array and the output channel mask array of the target operator in the form of masks, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays.

[0103] Based on the input channel mask array and output channel mask array storing the values ​​to be filled, global information of the neural network model to be processed is obtained, and parameter filling operation is performed on the target operator based on the global information;

[0104] Based on the target operator after the parameter filling operation, a filling operator and / or a pruning operator are inserted into the neural network model to be processed to obtain the target neural network model.

[0105] The specific implementation of the computer-readable storage medium in this application is largely the same as the specific implementation of the processing method of the above-described neural network model, and will not be repeated here.

[0106] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware simulation platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, electronic device, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0108] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for processing a neural network model, characterized in that, The method includes: Calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed. The neural network model to be processed refers to the neural network model trained on a general-purpose chip that needs to be adapted and run on a dedicated AI chip. The target input channel and output channel values ​​of the target operator refer to the number of input channels and the number of output channels of the operator of the neural network model required by the dedicated AI chip. The input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are stored in the input channel mask array and the output channel mask array of the target operator in the form of masks, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays. Based on the input channel mask array and output channel mask array storing the values ​​to be filled, global information of the neural network model to be processed is obtained, and parameter filling operation is performed on the target operator based on the global information; Based on the target operator after the parameter filling operation, a filling operator and / or a pruning operator are inserted into the neural network model to be processed to obtain the target neural network model.

2. The method for processing neural network models as described in claim 1, characterized in that, The calculation of the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed includes: Obtain the target values ​​of the input channel and the target value of the output channel of the target operator of the neural network model to be processed; Obtain the initial values ​​of the input channel and the initial values ​​of the output channel of the target operator; Based on the topological order of the neural network model to be processed, the target values ​​of the input channel and output channel of the target operator, and the initial values ​​of the input channel and output channel of the target operator, the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are calculated.

3. The method for processing neural network models as described in claim 2, characterized in that, The step of calculating the input channel to be filled value and the output channel to be filled value of the target operator based on the topological order of the neural network model to be processed, the target value of the input channel and the target value of the output channel of the target operator, and the initial value of the input channel and the initial value of the output channel of the target operator includes: According to the topological order of the neural network model to be processed, the difference between the target value of the input channel of the target operator and the initial value of the input channel of the target operator are calculated sequentially, and used as the value to be filled in the input channel of the target operator; Based on the topological order of the neural network model to be processed, the difference between the target value of the output channel of the target operator and the initial value of the output channel of the target operator are calculated sequentially, and used as the value to be filled in the output channel of the target operator.

4. The method for processing the neural network model as described in claim 1, characterized in that, The step of obtaining global information of the neural network model to be processed based on the input channel mask array and the output channel mask array storing the values ​​to be filled, and performing parameter filling operation on the target operator based on the global information, includes: The weight data of the target operator is initialized as a zero matrix containing the target values ​​of the input channel and the target values ​​of the output channel; By traversing the input channel target values ​​and output channel target values ​​in the zero matrix, and based on the input channel mask array and output channel mask array that store the values ​​to be filled, the global information of the neural network model to be processed is obtained; The number of input channels and the number of output channels in the zero matrix are filled according to the global information.

5. The method for processing the neural network model as described in claim 1, characterized in that, The step of inserting a padding operator and / or a pruning operator into the neural network model to be processed based on the target operator after the parameter padding operation includes: If the input channel mask array of the target operator after the parameter filling operation is not empty, and the input node corresponding to the target operator is a non-target operator that does not require parameter filling operation, then a filling operator is inserted between the target operator and the non-target operator.

6. The method for processing the neural network model as described in claim 1, characterized in that, The step of inserting a padding operator and / or a pruning operator into the neural network model to be processed based on the target operator after the parameter padding operation includes: If the output channel mask array of the target operator after the parameter filling operation is not empty, and the output node corresponding to the target operator is a non-target operator that does not require the parameter filling operation, then a pruning operator is inserted between the target operator and the non-target operator.

7. A processing device for a neural network model, characterized in that, The device includes: The calculation module is used to calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed. The neural network model to be processed refers to the neural network model trained on a general-purpose chip that needs to be adapted and run on a dedicated AI chip. The target input channel and output channel values ​​of the target operator refer to the number of input channels and the number of output channels of the operator of the neural network model required by the dedicated AI chip. Storage module: used to store the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator in the form of masks in the input channel mask array and the output channel mask array of the target operator, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays. Filling module: used to obtain global information of the neural network model to be processed based on the input channel mask array and the output channel mask array which store the values ​​to be filled, and to perform parameter filling operation on the target operator based on the global information; Insertion module: Used to insert filling operators and / or pruning operators into the neural network model to be processed based on the target operator after the execution parameter filling operation to obtain the target neural network model.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it performs the following steps: Calculate the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed. The neural network model to be processed refers to the neural network model trained on a general-purpose chip that needs to be adapted and run on a dedicated AI chip. The target input channel and output channel values ​​of the target operator refer to the number of input channels and the number of output channels of the operator of the neural network model required by the dedicated AI chip. The input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are stored in the input channel mask array and the output channel mask array of the target operator in the form of masks, respectively. The input channel mask array and the output channel mask array of the target operator are both pre-initialized empty arrays. Based on the input channel mask array and output channel mask array storing the values ​​to be filled, global information of the neural network model to be processed is obtained, and parameter filling operation is performed on the target operator based on the global information; Based on the target operator after the parameter filling operation, the filling operator and / or pruning operator are inserted into the neural network model to be processed to obtain the target neural network model.

9. The electronic device as claimed in claim 8, characterized in that, The calculation of the input channel and output channel values ​​to be filled for the target operator of the neural network model to be processed includes: Obtain the target values ​​of the input channel and the target value of the output channel of the target operator of the neural network model to be processed; Obtain the initial values ​​of the input channel and the initial values ​​of the output channel of the target operator; Based on the topological order of the neural network model to be processed, the target values ​​of the input channel and output channel of the target operator, and the initial values ​​of the input channel and output channel of the target operator, the input channel values ​​to be filled and the output channel values ​​to be filled of the target operator are calculated.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the processing method for the neural network model as described in any one of claims 1 to 6.

Images