A method, apparatus, and electronic device for convolution computation based on sparse neural networks.

Abstract

This invention relates to the field of neural network technology, specifically to a convolution calculation method, apparatus, and electronic device based on sparse neural networks. The method includes: determining the splitting rules of a filter array according to the convolution operation specifications in the neural network and splitting the filter array into multiple sub-filter arrays; performing sparsification processing on each of the sub-filter arrays after splitting, so that the weights in the target weight column of each sub-filter array are reduced to zero; storing the weights in the effective weight columns of each sub-filter array sequentially in the order of the effective weight columns into a weight memory; during the convolution calculation, sequentially reading the weights in each of the effective weight columns of each sub-filter array contained in the weight memory, and performing convolution calculation using the read weights. By sparsifying and reducing the weights to zero, the amount of weight storage and multiplication-accumulation operations is reduced.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a convolution calculation method, apparatus, and electronic device based on sparse neural networks. Background Technology

[0002] The rapid development of neural network technology has brought great convenience to our lives, and many applications incorporating neural network technology have been integrated into our daily lives. The training of a neural network mainly involves using sample data to train the parameters within the neural network to find the optimal parameters so that the neural network achieves the expected computational effect. Parameter training includes training the weights in the convolutional kernel.

[0003] In neural networks, a filter is a multidimensional matrix composed of multiple convolutional kernels, such as... Figure 1 The image shows a filter array with dimensions F = 128, CI = 64, Kx = 3, and Ky = 3. Here, F represents the number of filters in the array (128), CI represents the number of channels in each filter (64 channels per channel, each channel corresponding to a convolutional kernel), and Kx*Ky represents the dimension of the convolutional kernel. Figure 1 The convolutional kernel has a dimension of 3*3, meaning each kernel contains 9 weights. Weights at the same position in all kernels of the same filter form a weight column. Figure 1 Each filter has 9 weight columns at different positions, K1-K9.

[0004] After the neural network training is completed, each weight in the convolution kernel needs to be stored in the weight memory for use in the neural network data processing. It is easy to understand that the larger the size of the filter, the more weights it contains, and the greater the space occupied by storing the weights and the amount of multiplication and addition operations when using the weights for convolution calculation. Based on this, how to reduce the space occupied by weight storage and the amount of multiplication and addition operations during convolution calculation has become the main research direction for technicians. Summary of the Invention

[0005] To address the problems existing in the prior art, embodiments of the present invention provide a convolution calculation method, apparatus, and electronic device based on sparse neural networks. The specific technical solution is as follows:

[0006] Firstly, a convolution computation method based on sparse neural networks is provided, including:

[0007] The splitting rules of the filter array are determined according to the convolution operation specifications in the neural network. The splitting rules include the number of filters in each sub-filter array obtained after splitting the filter array and the number of channels in each filter.

[0008] The filter array is split into multiple sub-filter arrays according to the determined splitting rules;

[0009] Each of the sub-filter arrays obtained after splitting is sparsified so that the weights in the target weight column of each sub-filter array are reduced to zero.

[0010] The weights in the effective weight columns of each sub-filter array are stored sequentially in the weight memory according to the order of the effective weight columns, wherein the effective weight columns are the weight columns in each sub-filter array other than the target weight column;

[0011] During the convolution calculation, the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory are read sequentially, and the read weights are used to perform convolution calculation.

[0012] Optionally, the step of performing sparsification processing on each of the sub-filter arrays obtained after splitting, so that the weights in the target weight column of each sub-filter array are reduced to zero, includes:

[0013] For each sub-filter array, the weights in the weight column at the same position in each of its constituent filters are summed.

[0014] Obtain a predetermined sparsity rate, and determine the number of weight columns in each of the sub-filter arrays whose weights are set to zero based on the predetermined sparsity rate;

[0015] For each of the sub-filter arrays, in ascending order of the weight summation result, a determined number of weight columns are selected and zeroed out.

[0016] Neural network operations were performed using the filter array after it was zeroed out and the filter array before it was zeroed out, respectively, to obtain the first operation result and the second operation result.

[0017] Calculate the error between the first calculation result and the second calculation result;

[0018] If the obtained error meets expectations, the weight column that has been zeroed out will be used as the target weight column;

[0019] If the obtained error does not meet expectations, reduce the predetermined sparsity rate and return to the step of determining the number of weight columns in each of the sub-filter arrays whose weights are set to zero according to the predetermined sparsity rate.

[0020] Optionally, before the step of storing the weights sequentially into the weight memory according to the order of the effective weight columns, the method further includes:

[0021] A sparsified table is created, in which the effective states of each weight column contained in each of the sub-filter arrays are recorded, and the sparsified table is stored in the weight memory.

[0022] Optionally, before the step of sequentially reading the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory, the method further includes:

[0023] Read the sparsed table.

[0024] Optionally, the step of performing convolution calculation using the read weights includes:

[0025] Based on the valid states recorded in the sparse table, the convolution calculations for invalid weight columns are skipped.

[0026] Secondly, a convolutional computation device based on a sparse neural network is provided, comprising:

[0027] The rule determination module is used to determine the splitting rules of the filter array according to the convolution operation specifications in the neural network. The splitting rules include the number of filters in each sub-filter array obtained after splitting the filter array and the number of channels in each filter.

[0028] An array partitioning module is used to partition the filter array into multiple sub-filter arrays according to the determined partitioning rules.

[0029] The sparsity processing module is used to perform sparsification processing on each of the sub-filter arrays obtained after splitting, so that the weights in the target weight column of each sub-filter array are reduced to zero.

[0030] The weight storage module is used to store the weights in the effective weight columns contained in each of the sub-filter arrays sequentially into the weight memory according to the order of the effective weight columns, wherein the effective weight columns are the weight columns in each of the sub-filter arrays other than the target weight column;

[0031] The convolution calculation module is used to sequentially read the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory during the convolution calculation process, and perform convolution calculation using the read weights.

[0032] Optional, sparse processing module, for

[0033] For each sub-filter array, the weights in the weight column at the same position in each of its constituent filters are summed.

[0034] Obtain a predetermined sparsity rate, and determine the number of weight columns in each of the sub-filter arrays whose weights are set to zero based on the predetermined sparsity rate;

[0035] For each of the sub-filter arrays, in ascending order of the weight summation result, a determined number of weight columns are selected and zeroed out.

[0036] Neural network operations were performed using the filter array after it was zeroed out and the filter array before it was zeroed out, respectively, to obtain the first operation result and the second operation result.

[0037] Calculate the error between the first calculation result and the second calculation result;

[0038] If the obtained error meets expectations, the weight column that has been zeroed out will be used as the target weight column;

[0039] If the obtained error does not meet expectations, reduce the predetermined sparsity rate and return to the step of determining the number of weight columns in each of the sub-filter arrays whose weights are set to zero according to the predetermined sparsity rate.

[0040] Optional, also includes:

[0041] A table creation module is used to create a sparsified table, record the effective status of each weight column contained in each of the sub-filter arrays in the sparsified table, and store the sparsified table in the weight memory.

[0042] Optionally, the convolution calculation module is also used to read the sparsed table.

[0043] Optionally, the convolution calculation module is also used to skip convolution calculations for invalid weight columns based on the valid states recorded in the sparse table.

[0044] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0045] Memory is used to store processor-executable instructions;

[0046] The processor, when executing instructions stored in memory, implements the above-mentioned convolution calculation method based on sparse neural networks.

[0047] The solution provided in this embodiment of the invention splits the filter array according to the maximum processing performance of convolution in the neural network, and performs sparsification processing on each sub-filter array obtained by the split. The weights that are reduced to zero after sparsification do not occupy the storage space of the weight memory and do not participate in the multiplication and addition operations in the convolution operation process, thereby reducing the weight storage and multiplication and addition operations. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of a filter array provided in an embodiment of the present invention;

[0050] Figure 2 This is a flowchart illustrating a convolution calculation method based on a sparse neural network provided in an embodiment of the present invention.

[0051] Figure 3 Yes Figure 1 A schematic diagram of the result after sparsification of the filter array;

[0052] Figure 4 This is a schematic diagram of a weight storage structure in a weight memory provided in an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of a sparse table provided in an embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of the structure of a convolutional computing device based on a sparse neural network provided in an embodiment of the present invention;

[0055] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0057] See Figure 2 The present invention provides a convolution calculation method based on a sparse neural network, comprising:

[0058] S200 determines the splitting rules of the filter array based on the convolution operation specifications in the neural network.

[0059] The splitting rules include the number of filters in each sub-filter array obtained after splitting the filter array, and the number of channels in each filter.

[0060] In implementation, convolution can be understood as taking a filter array of a specific size each time and then multiplying and adding it with the input. The convolution operation specification represents the maximum computational load of a single convolution operation. It determines the specification of the selected filter array, which in turn determines the number of filters used in each operation and the number of channels in each filter. For example, a convolution operation specification of 32*64 means that 64 filters and 32 channels in each filter can be used in a single convolution operation. The corresponding splitting rule is that after splitting the original filter array, each sub-filter array contains 64 filters and each filter has 32 channels. Figure 1 After the filter array is sparsified and split, four sub-filter arrays can be obtained.

[0061] S210, the filter array is split according to the determined splitting rules to obtain multiple sub-filter arrays.

[0062] In implementation, the initial splitting can be performed based on the number of filters contained in the sub-filter array. Taking a convolution operation specification of 32*64 as an example, that is, each sub-filter array contains 64 filters. During the splitting, the filter arrays to be split are traversed, and every group of 64 filters is grouped together to complete the initial splitting. Then, based on the criterion that each sub-filter array contains 32 channels, each sub-filter array obtained after the initial splitting is traversed and splitting is continued. Finally, the filter array is grouped into multiple sub-filter arrays of specification 32*64.

[0063] S220, each of the sub-filter arrays obtained after splitting is sparsified so that the weights in the target weight column of each sub-filter array are reduced to zero.

[0064] The target weight column can be understood as the weight column selected to minimize the error between the neural network operation result after the weights are zeroed and the neural network operation result without zeroing. In one embodiment, the weights in the weight columns at the same position in each of the filters contained in each sub-filter array can be summed. A predetermined sparsity rate is obtained, and the number of weight columns in each sub-filter array whose weights are zeroed is determined according to the predetermined sparsity rate. For each sub-filter array, the determined number of weight columns are selected and zeroed in ascending order of the weight summation result. Neural network operation is performed using the zeroed filter array and the non-zeroed filter array respectively to obtain a first operation result and a second operation result. The error between the first operation result and the second operation result is calculated. If the obtained error meets the expectation, the zeroed weight column is taken as the target weight column. If the obtained error does not meet the expectation, the predetermined sparsity rate is reduced, and the process returns to the step of determining the number of weight columns in each sub-filter array whose weights are zeroed according to the predetermined sparsity rate.

[0065] In implementation, sparsity can be understood as the ratio of the number of target weight columns that are set to zero to the total number of weight columns. The sparsity is used to determine the number of weight columns with zero weights in each sub-filter array. For example, using... Figure 1 In the example, when the sparsity rate is 10%, it can be determined that a weight column at the same position in the sub-filter array needs to be zeroed out. That is, a weight column at the same position in one of the nine different weight columns (K1-K9) needs to be zeroed out. The smaller the sum of the weights in a weight column, the lower its importance. Therefore, when selecting the target weight column, for each sub-filter array, the sum of the weights in the weight columns K1-K9 of each filter can be calculated first. Then, the sums of K1, K2, ..., K9 in each filter are added together. Finally, the sums of the weights in the nine different weight columns in the sub-filter array are obtained and compared. The one with the smallest sum is selected as the target weight column to be zeroed out. After zeroing out, the same data is processed using the sparsified neural network and the unsparsed neural network to compare the results. This determines whether the sparsified neural network meets the expectations. If it does, the current sparsification result is retained; if it does not meet the expectations, the sparsity rate needs to be reduced.

[0066] by Figure 1 Taking a filter array as an example, in a specific selection method when the sparsity is 50%, we can assume a situation where each of the 64 filters in Filter[0:63] has a sparsity of 50%.

[0067] The weight values ​​of the four target weight columns with the smallest sums, K2C[0:31], K4C[0:31], K5C[0:31], and K8C[0:31], are all set to 0.

[0068] The weight values ​​of the five target weight columns with the smallest sums, K1C[63:32], K3C[63:32], K4C[63:32], K7C[63:32], and K8C[63:32], are all set to zero.

[0069] Filter[64:127] is one of the 64 filters.

[0070] The weight values ​​of the five target weight columns with the smallest sums, K1C[0:31], K3C[0:31], K5C[0:31], K7C[0:31], and K8C[0:31], are all set to zero.

[0071] The weight values ​​of the four target weight columns with the smallest sums, K2C[63:32], K4C[63:32], K5C[63:32], and K7C[63:32], are all set to zero.

[0072] The sparsification result is as follows Figure 3 As shown, the sparsification effect of filters in the same sub-filter array is exactly the same. According to the principle of minimizing error, different sub-filter arrays can choose different target weight columns for sparsification. After sparsification, the weight compression ratio is 50%, which can reduce the storage space and multiply-accumulate operation by half.

[0073] S230: For the weights in the effective weight columns contained in each sub-filter array, store them sequentially into the weight memory according to the order of the effective weight columns.

[0074] Among them, the effective weight column is the weight column in each sub-filter array other than the target weight column.

[0075] During storage, the weights in each weight column can be stored sequentially in a table format, and only the effective weights are stored. Figure 3 Taking the sparsification result as an example, the sparsity unit of each group is CI=32, which is the number of channels in each filter after splitting, and CO=64, which is the number of filters in each group after splitting. Only valid weights are stored. For example, if the weights in the weight columns K1, K3, K6, K7, and K9 in the first group are not zeroed out, they are considered valid weights. Therefore, they are stored in a tightly packed manner according to the numbering order of the valid weight columns K1, K3, K6, K7, and K9. The storage result is as follows: Figure 4 As shown, one of the weight columns corresponds to 32 weights.

[0076] In one implementation, to more quickly determine which weight column in the filter has its weights set to zero, thus simplifying subsequent convolution calculations, a method can be created... Figure 5 The sparse table shown records the validity of each weight column; for example, 1 indicates a valid weight column, and 0 indicates an invalid weight column. Each sub-filter array occupies 9 bits, and the four sub-filter arrays occupy a total of 36 bits. Figure 5 In combination with the above Figure 4 Using a table to store weights reduces the storage space required for weights while facilitating subsequent convolution calculations.

[0077] S240: During the convolution calculation, the weights in each effective weight column contained in each sub-filter array in the weight memory are read sequentially, and the read weights are used to perform convolution calculation.

[0078] In implementation, the sparse table can be read first. During the convolution calculation process, the valid states recorded in the sparse table are parsed iteratively, skipping the convolution calculations performed on invalid weight columns (i.e., the target weight columns). Figure 5 Taking a sparse table as an example, starting from the first row of table-0, we determine the weight column Ky1 = {K3, K2, K1} = {1, 0, 1}. Then, during the convolution calculation, since the weight column K2 = 0, we can directly skip the convolution calculation for the weight column K2, thus reducing the amount of computation.

[0079] See Figure 6 The present invention provides a convolutional computation device based on a sparse neural network, comprising:

[0080] The rule determination module 600 is used to determine the splitting rules of the filter array according to the convolution operation specifications in the neural network, wherein the splitting rules include the number of filters contained in each sub-filter array after splitting and the number of channels contained in each filter.

[0081] The array partitioning module 610 is used to partition the filter array according to the determined partitioning rules to obtain multiple sub-filter arrays;

[0082] The sparsity processing module 620 is used to perform sparsification processing on each of the sub-filter arrays obtained after splitting, so that the weights in the target weight column of each sub-filter array are reduced to zero.

[0083] The weight storage module 630 is used to store the weights in the effective weight columns contained in each of the sub-filter arrays sequentially into the weight memory according to the order of the effective weight columns, wherein the effective weight columns are the weight columns in each of the sub-filter arrays other than the target weight column;

[0084] The convolution calculation module 640 is used to sequentially read the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory during the convolution calculation process, and to perform convolution calculation using the read weights.

[0085] In implementation, the sparse processing module 620 is used for

[0086] The weights contained in the weight column at the same position in each filter of each sub-filter array are summed.

[0087] Obtain a predetermined sparsity rate, and determine the number of weight columns in each of the sub-filter arrays whose weights are set to zero based on the predetermined sparsity rate;

[0088] For each of the sub-filter arrays, in ascending order of the weight summation result, a determined number of weight columns are selected and zeroed out.

[0089] Neural network operations were performed using the filter array after it was zeroed out and the filter array before it was zeroed out, respectively, to obtain the first operation result and the second operation result.

[0090] Calculate the error between the first calculation result and the second calculation result;

[0091] If the obtained error meets expectations, the zeroing weight column will be used as the target weight column;

[0092] If the obtained error does not meet expectations, reduce the predetermined sparsity rate and return to the step of determining the number of weight columns in each of the sub-filter arrays whose weights are set to zero according to the predetermined sparsity rate.

[0093] In implementation, it also includes: a table creation module, used to create a sparsified table, record the effective status of each weight column contained in each of the sub-filter arrays in the sparsified table, and store the sparsified table in the weight memory.

[0094] In implementation, the convolution calculation module 640 is also used to read the sparse table.

[0095] In implementation, the convolution calculation module 640 is also used to skip the convolution calculation for invalid weight columns based on the valid states recorded in the sparse table.

[0096] The solution provided in this embodiment of the invention splits the filter array according to the maximum processing performance of convolution in the neural network, and performs sparsification on each sub-filter array obtained by splitting. The weights that are reduced to zero after sparsification do not occupy the storage space of the weight memory, and do not participate in the multiplication and addition operations in the convolution operation process, thereby reducing the weight storage and multiplication and addition operations.

[0097] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 001, a communication interface 002, a memory 003, and a communication bus 004. The processor 001, communication interface 002, and memory 003 communicate with each other via the communication bus 004.

[0098] Memory 003 is used to store computer programs;

[0099] Processor 001, when executing the program stored in memory 003, implements the above-mentioned convolution calculation method based on sparse neural networks, the method comprising:

[0100] The splitting rules of the filter array are determined according to the convolution operation specifications in the neural network, wherein the splitting rules include the number of filters in each sub-filter array after splitting and the number of channels in each filter;

[0101] The filter array is split into multiple sub-filter arrays according to the determined splitting rules;

[0102] Each of the sub-filter arrays obtained after splitting is sparsified so that the weights in the target weight column of each sub-filter array are reduced to zero.

[0103] The weights in the effective weight columns of each sub-filter array are stored sequentially in the weight memory according to the order of the effective weight columns, wherein the effective weight columns are the weight columns in each sub-filter array other than the target weight column;

[0104] During the convolution calculation, the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory are read sequentially, and the read weights are used to perform convolution calculation.

[0105] The solution provided in this embodiment of the invention splits the filter array according to the maximum processing performance of convolution in the neural network, and performs sparsification on each sub-filter array obtained by splitting. The weights that are reduced to zero after sparsification do not occupy the storage space of the weight memory, and do not participate in the multiplication and addition operations in the convolution operation process, thereby reducing the weight storage and multiplication and addition operations.

[0106] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0107] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0108] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0109] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0110] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0111] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0112] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and electronic device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A convolution calculation method based on sparse neural networks, characterized in that, The convolution calculation method includes: The splitting rules of the filter array are determined according to the convolution operation specifications in the neural network. The splitting rules include the number of filters in each sub-filter array obtained after splitting the filter array and the number of channels in each filter. The filter array is split into multiple sub-filter arrays according to the determined splitting rules; Each of the sub-filter arrays obtained after splitting is sparsified so that the weights in the target weight column of each sub-filter array are reduced to zero. The weights in the effective weight columns of each sub-filter array are stored sequentially in the weight memory according to the order of the effective weight columns, wherein the effective weight columns are the weight columns in each sub-filter array other than the target weight column; During the convolution calculation process, the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory are read sequentially, and the read weights are used to perform convolution calculation; The step of performing sparsification processing on each of the sub-filter arrays obtained after splitting, so that the weights in the target weight column of each sub-filter array are reduced to zero, includes: For each sub-filter array, the weights in the weight column at the same position in each of its constituent filters are summed. Obtain a predetermined sparsity rate, and determine the number of weight columns in each of the sub-filter arrays whose weights are set to zero based on the predetermined sparsity rate; For each of the sub-filter arrays, in ascending order of the weight summation result, a determined number of weight columns are selected and zeroed out. Neural network operations were performed using the filter array after it was zeroed out and the filter array before it was zeroed out, respectively, to obtain the first operation result and the second operation result. Calculate the error between the first calculation result and the second calculation result; If the obtained error meets expectations, the weight column that has been zeroed out will be used as the target weight column; If the obtained error does not meet expectations, reduce the predetermined sparsity rate and return to the step of determining the number of weight columns in each of the sub-filter arrays whose weights are set to zero according to the predetermined sparsity rate.

2. The convolution calculation method based on sparse neural networks as described in claim 1, characterized in that, Before the step of storing the weights sequentially into the weight memory according to the order of the effective weight columns, the method further includes: A sparsified table is created, in which the effective states of each weight column contained in each of the sub-filter arrays are recorded, and the sparsified table is stored in the weight memory.

3. The convolution calculation method based on sparse neural networks as described in claim 2, characterized in that, Before the step of sequentially reading the weights from each of the effective weight columns contained in each of the sub-filter arrays in the weight memory, the method further includes: Read the sparsed table.

4. The convolution calculation method based on sparse neural networks as described in claim 3, characterized in that, The step of performing convolution calculation using the read weights includes: Based on the valid states recorded in the sparse table, the convolution calculations for invalid weight columns are skipped.

5. A convolutional computation device based on a sparse neural network, characterized in that, The convolution calculation device includes: The rule determination module is used to determine the splitting rules of the filter array according to the convolution operation specifications in the neural network. The splitting rules include the number of filters in each sub-filter array obtained after splitting the filter array and the number of channels in each filter. An array partitioning module is used to partition the filter array into multiple sub-filter arrays according to the determined partitioning rules. The sparsity processing module is used to perform sparsification processing on each of the sub-filter arrays obtained after splitting, so that the weights in the target weight column of each sub-filter array are reduced to zero. The weight storage module is used to store the weights in the effective weight columns contained in each of the sub-filter arrays sequentially into the weight memory according to the order of the effective weight columns, wherein the effective weight columns are the weight columns in each of the sub-filter arrays other than the target weight column; The convolution calculation module is used to sequentially read the weights in each of the effective weight columns contained in each of the sub-filter arrays in the weight memory during the convolution calculation process, and perform convolution calculation using the read weights; Sparse processing module, used for For each sub-filter array, the weights in the weight column at the same position in each of its constituent filters are summed. Obtain a predetermined sparsity rate, and determine the number of weight columns in each of the sub-filter arrays whose weights are set to zero based on the predetermined sparsity rate; For each of the sub-filter arrays, in ascending order of the weight summation result, a determined number of weight columns are selected and zeroed out. Neural network operations were performed using the filter array after it was zeroed out and the filter array before it was zeroed out, respectively, to obtain the first operation result and the second operation result. Calculate the error between the first calculation result and the second calculation result; If the obtained error meets expectations, the weight column that has been zeroed out will be used as the target weight column; If the obtained error does not meet expectations, reduce the predetermined sparsity rate and return to the step of determining the number of weight columns in each of the sub-filter arrays whose weights are set to zero according to the predetermined sparsity rate.

6. The convolutional computation device based on a sparse neural network as described in claim 5, characterized in that, Also includes: A table creation module is used to create a sparsified table, record the effective status of each weight column contained in each of the sub-filter arrays in the sparsified table, and store the sparsified table in the weight memory.

7. The convolutional computation device based on sparse neural networks as described in claim 6, characterized in that, The convolution calculation module is also used to read the sparsed table.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory is used to store processor-executable instructions; A processor, when executing instructions stored in memory, implements the convolution calculation method according to any one of claims 1-4.

Images