A convolution processing method, device and storage medium for convolutional neural network

Abstract

An embodiment of the present invention discloses a convolution processing method, device and storage medium for a convolutional neural network, wherein the method includes: classifying the weight values ​​in the sub-convolution kernels in the convolutional neural network; generating an indicator for storing a corresponding operation on the data and an address for representing the weight value for the corresponding weight value according to the classification result of each weight value; obtaining the corresponding data to be processed according to the address of the weight value; and performing a convolution operation on the data to be processed according to the indicator to obtain a convolution result.

Description

Technical Field

[0001] The present invention relates to a deep neural network learning technology, and more particularly to, but not limited to, a convolution processing method, device, and storage medium for a convolutional neural network. Background Art

[0002] At present, artificial intelligence technology has developed rapidly around the world. The industry has invested a lot of energy in the research of artificial intelligence technology and has achieved remarkable results. Especially in the fields of image detection and recognition, as well as language recognition, the recognition rate of artificial intelligence has exceeded that of humans.

[0003] At the same time, the scale of neural networks continues to grow. A typical neural network has dozens or even hundreds of layers, with hundreds of millions of connections between neurons. That is, as the performance indicators of the algorithm continue to improve, the amount of computing continues to increase, and the time required to complete reasoning is also getting longer and longer.

[0004] However, in actual applications, with such a high amount of computation, not all calculations are valid. For algorithms that have been fixed-point or sparse, that is, compressed, branched, or clustered through training, the points with zero weights do not need to be calculated in actual operation. Summary of the Invention

[0005] Embodiments of the present invention provide a convolution processing method, device, and storage medium for a convolutional neural network.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a convolution processing method for a convolutional neural network, the method comprising:

[0008] Classifying weight values ​​in subconvolution kernels of the convolutional neural network;

[0009] generating, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value;

[0010] Obtain the corresponding data to be processed according to the address of the weight value;

[0011] A convolution operation is performed on the data to be processed according to the indicator to obtain a convolution result.

[0012] In a second aspect, an embodiment of the present invention provides a convolution processing device for a convolutional neural network, the device comprising:

[0013] a classification unit configured to classify weight values ​​in the subconvolution kernels of the convolutional neural network;

[0014] a generating unit configured to generate, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value;

[0015] an acquiring unit, configured to acquire corresponding data to be processed according to the address of the weight value;

[0016] The convolution processing unit is configured to perform a convolution operation on the data to be processed according to the indicator to obtain a convolution result.

[0017] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are configured to execute the convolution processing device of the convolutional neural network provided in the above embodiment.

[0018] Embodiments of the present invention provide a convolution processing method, device, and storage medium for a convolutional neural network. The method includes: classifying weight values ​​in subconvolution kernels in the convolutional neural network; generating, based on the classification result of each weight value, an indicator for storing a corresponding operation on data and an address for representing the weight value; obtaining corresponding data to be processed based on the address of the weight value; and performing a convolution operation on the data to be processed based on the indicator to obtain a convolution result. In this way, through the combination of weight conversion and a new processing unit, the sparse characteristics of the weights can be fully utilized. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a convolution processing method for a convolutional neural network according to the first embodiment of the present invention;

[0020] Figure 2A A schematic flow chart of a convolution processing method for a convolutional neural network provided in the second embodiment of the present invention;

[0021] Figure 2B A schematic flow chart of another convolution processing method for a convolutional neural network provided in the second embodiment of the present invention;

[0022] Figure 2C A schematic flow chart of another convolution processing method for a convolutional neural network provided in the second embodiment of the present invention;

[0023] Figure 3 A schematic flow chart of a convolution processing method for a convolutional neural network provided in Example 3 of the present invention;

[0024] Figure 4 A schematic diagram of the process of weight classification and rearrangement provided in the third embodiment of the present invention;

[0025] Figure 5A schematic diagram of a structure for cutting an original convolution kernel into sub-convolution kernels provided in the third embodiment of the present invention;

[0026] Figure 6 A schematic diagram of the indicator table and mapping table generated according to the third embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the weight table structure after weight re-arrangement provided in the third embodiment of the present invention;

[0028] Figure 8 A schematic diagram of the weight table structure after weight re-arrangement in the second method provided in the third embodiment of the present invention;

[0029] Figure 9 This is a schematic diagram of the weight table entry structure after weight re-arrangement in the third embodiment of the present invention;

[0030] Figure 10 This is a schematic diagram of the structure of weight table entries after weight re-arrangement in the fourth method provided in the third embodiment of the present invention;

[0031] Figure 11 A schematic diagram of the convolution processing structure of a convolutional neural network provided in Example 3 of the present invention;

[0032] Figure 12 A schematic diagram of the convolution processing structure of another convolutional neural network provided in Example 3 of the present invention;

[0033] Figure 13 A schematic diagram of the convolution processing structure of another convolutional neural network provided in Example 3 of the present invention;

[0034] Figure 14 A schematic diagram of the composition structure of a convolution processing device for a convolutional neural network provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0035] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the specific technical solutions of the invention will be described in further detail below in conjunction with the accompanying drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0036] In an embodiment of the present invention, a new processing unit structure is provided. Compared to the multiplication-addition structure of a traditional processing unit, the processing unit is split into a classification accumulation unit and a classification multiplication unit. This structure has better scalability and can support functions such as early calculation termination, clock gating, and zero data stop calculation. Through weight conversion, an appropriate conversion strategy can be selected based on the characteristics of the neural network structure. The combination of weight conversion and the new processing unit can fully utilize the sparse characteristics of weights. Through the embodiment of the present invention, the hardware accelerator and processor of the neural network can utilize the sparse characteristics of weights to reduce the amount of calculation, reduce processing time, and improve efficiency. Example 1

[0037] The embodiment of the present invention provides a convolution processing method of a convolutional neural network. Figure 1 A flow chart of a convolution processing method of a convolutional neural network provided in the first embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0038] Step S101, classifying the weight values ​​in the sub-convolution kernels of the convolutional neural network.

[0039] Here, the sub-convolution kernels in the convolutional neural network are classified according to a preset strategy, and the weight values ​​are rearranged according to the classification results, and the rearranged weight values ​​are converted accordingly.

[0040] In other embodiments, the classifying the weight values ​​in the sub-convolution kernels in the convolutional neural network includes: classifying the same weight values ​​in the sub-convolution kernels into one category; or, classifying the weight values ​​with the same absolute values ​​into one category; or, classifying the weight values ​​that are multiples of a power of 2 into one category; or, classifying the positive weight values ​​and the negative weight values ​​into one category respectively.

[0041] Here, taking the classification of the same weight values ​​in the sub-convolution kernel into one category as an example, in an embodiment of the present invention, only the same weight values ​​in the sub-convolution kernel can be classified into one category, and only the same weight values ​​can be arranged together. Assuming that the column width, row width and number of channels of the sub-convolution kernel are R, S, and C respectively, there are 256 types of weight values. According to the technical solutions in the related art, R*S*C multiplication and addition operations are required, but through the embodiment of the present invention, only R*S*C+256 addition operations and 256 multiplication operations are required at most. In other embodiments, the weight types can be reduced to 64 or less by merging weights with the same positive and negative absolute values, or by merging weights that are multiples of 2 to the power of phu, or by merging positive weights and negative weights separately, with positive weight types arranged in front and negative weight types arranged in the back. The rearranged indicator table and mapping table are stored in the global cache.

[0042] Step S102 : generating, for the corresponding weight value, an indicator for storing a corresponding operation on data and an address for representing the weight value according to the classification result of each weight value.

[0043] Here, according to the classification and rearrangement results of the weight values, the weight values ​​can be converted into corresponding indicator tables and mapping tables. The indicator table is used to store indicators for performing corresponding operations on data; the mapping table is used to represent the addresses of the weight values.

[0044] In an embodiment of the present invention, the indicator represents an indicator for storing corresponding operations corresponding to all categories of weight values, each indicator corresponds to a category of weight values, and the mapping table corresponding to each indicator stores the addresses corresponding to all weight values ​​in this category of weight values, wherein the operations corresponding to the indicator include: weight value, end address, shift value, addition and subtraction type, extension information and classification indication; the mapping table includes the address of the weight value, that is, the parameters representing the row number, column number and channel number corresponding to the weight value.

[0045] Step S103: Obtain the corresponding data to be processed according to the address of the weight value.

[0046] Here, the address of the weight value corresponds to the address of the data to be processed, and the corresponding data to be processed can be obtained based on the corresponding relationship. The data to be processed can be images, vehicle operation data, language data, etc.

[0047] Step S104: performing a convolution operation on the data to be processed according to the indicator to obtain a convolution result.

[0048] Here, since the indicator includes: weight value, end address, shift value, addition / subtraction type, extension information and classification indication; the data to be processed can be processed accordingly according to the operation in the indicator, such as shifting or addition / subtraction operations.

[0049] In an embodiment of the present invention, the weights are compressed in advance through weight conversion, so that the time of the convolution operation is only related to the number of non-zero weights, and the sparsity rate of the weight convolution kernel is fully utilized, which greatly saves the inference running time. Example 2

[0050] The embodiment of the present invention provides a convolution processing method of a convolutional neural network. Figure 2A This is a flow chart of a convolution processing method for a convolutional neural network provided in the second embodiment of the present invention. Figure 2A Said method comprises:

[0051] Step S201a: cutting the original convolution kernel in the convolutional neural network to obtain at least two sub-convolution kernels.

[0052] Here, taking the use of the compiler for weight conversion as an example, the compiler can perform iterative operations to cut a convolution kernel of size R*S*CH into multiple sub-convolution kernels of size R*S*C (where R is the column width of the convolution kernel, S is the row width of the convolution kernel, and C is the number of channels of the sub-convolution kernel. Assuming that the number of channels of the convolution kernel of this layer is Ch, then C<=Ch).

[0053] Step S202a, performing fixed-point processing on the original weight value in each sub-convolution kernel to obtain the weight value in the sub-convolution kernel.

[0054] Step S203a: classify the weight values ​​in the sub-convolution kernels of the convolutional neural network.

[0055] Here, the sub-convolution kernels in the convolutional neural network are classified according to a preset strategy, and the weight values ​​are rearranged according to the classification results, and the rearranged weight values ​​are converted accordingly.

[0056] Step S204a: generating, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value.

[0057] Here, according to the classification and rearrangement results of the weight values, the weight values ​​can be converted into corresponding indicator tables and mapping tables. The indicators in the indicator table are used to store indicators for performing corresponding operations on the data; the mapping table is used to represent the addresses of the weight values.

[0058] In an embodiment of the present invention, the indicator represents an indicator for storing corresponding operations corresponding to all categories of weight values, each indicator corresponds to a category of weight values, and the mapping table corresponding to each indicator stores the addresses corresponding to all weight values ​​in this category of weight values, wherein the operations corresponding to the indicator include: weight value, end address, shift value, addition and subtraction type, extension information and classification indication; the mapping table includes the address of the weight value, that is, the parameters representing the row number, column number and channel number corresponding to the weight value.

[0059] Step S205a: Obtain the corresponding data to be processed according to the address of the weight value.

[0060] Here, the address of the weight value corresponds to the address of the data to be processed, and the corresponding data to be processed can be obtained according to the corresponding relationship.

[0061] Step S206a: performing a convolution operation on the data to be processed according to the indicator to obtain a convolution result.

[0062] Here, since the indicator includes: weight value, end address, shift value, addition / subtraction type, extension information and classification indication; the data to be processed can be processed accordingly according to the operation in the indicator, such as shifting or addition / subtraction operations.

[0063] In other embodiments, obtaining the corresponding data to be processed according to the address of the weight value includes: determining a data set to be processed that has the same row width, column width and number of channels as the sub-convolution kernel from the input original data set to be processed; establishing a correspondence between the address of each data to be processed in the data set to be processed and the address of the corresponding weight value in the sub-convolution kernel; and obtaining the data to be processed at the corresponding address in the data set to be processed according to the correspondence.

[0064] Figure 2B A flow chart of another convolution processing method of a convolutional neural network provided in the second embodiment of the present invention is shown as follows: Figure 2B As shown, the method includes:

[0065] Step S201b: cutting the original convolution kernel in the convolutional neural network to obtain at least two sub-convolution kernels.

[0066] Step S202b: performing fixed-point processing on the original weight value in each sub-convolution kernel to obtain the weight value in the sub-convolution kernel.

[0067] Step S203b: classify the weight values ​​in the sub-convolution kernels of the convolutional neural network.

[0068] Step S204b: generating, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value.

[0069] Step S205b: Determine a data set to be processed from the input original data set to be processed that has the same row width, column width, and number of channels as the sub-convolution kernel.

[0070] Here, the original data set to be processed is a set of all data to be processed. In an embodiment of the present invention, it is necessary to obtain a corresponding data set to be processed from the original data set to be processed. The row width, column width and number of channels of the data set to be processed are the same as the row width, column width and number of channels of the sub-convolution kernel.

[0071] Step S206b: establishing a correspondence between the address of each to-be-processed data in the to-be-processed data set and the address of the corresponding weight value in the sub-convolution kernel.

[0072] Here, each data to be processed in the data set to be processed has a one-to-one correspondence with the corresponding weight value in the sub-convolution kernel. The address of a corresponding data to be processed can be obtained according to the address of a weight value in the sub-convolution kernel.

[0073] Step S207b: acquiring the data to be processed of the corresponding address in the set of data to be processed according to the corresponding relationship.

[0074] Here, the address of a corresponding data to be processed can be obtained according to the address of a certain weight value in the sub-convolution kernel, and the corresponding data to be processed can be determined according to the address.

[0075] Step S208b: performing a convolution operation on the data to be processed according to the indicator to obtain a convolution result.

[0076] In other embodiments, the convolution operation is performed on the data to be processed according to the indicator to obtain a convolution result, including: obtaining the cumulative sum of all the data to be processed corresponding to all weight values ​​in each category of weight value according to the indicator; multiplying the cumulative sum with the corresponding weight value to obtain a partial sum; and adding all the partial sums corresponding to the weight values ​​of all categories to obtain a convolution result.

[0077] Figure 2C A flow chart of another convolution processing method of a convolutional neural network provided in the second embodiment of the present invention is shown as follows: Figure 2C As shown, the method includes:

[0078] Step S201c: cutting the original convolution kernel in the convolutional neural network to obtain at least two sub-convolution kernels.

[0079] Step S202c: performing fixed-point processing on the original weight value in each sub-convolution kernel to obtain the weight value in the sub-convolution kernel.

[0080] Step S203c: classify the weight values ​​in the sub-convolution kernels of the convolutional neural network.

[0081] Step S204c: generating, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value.

[0082] Here, according to the classification and rearrangement results of the weight values, the weight values ​​can be converted into corresponding indicator tables and mapping tables. The indicator table is used to store indicators for performing corresponding operations on data; the mapping table is used to represent the addresses of the weight values.

[0083] In an embodiment of the present invention, an indicator table is used to store indicators of corresponding operations corresponding to all categories of weight values, each indicator corresponds to a category of weight values, and a mapping table corresponding to each indicator stores addresses corresponding to all weight values ​​in this category of weight values, wherein the operations corresponding to the indicators include: weight value, end address, shift value, addition and subtraction type, extension information and classification indication; the mapping table includes the address of the weight value, that is, the parameters representing the row number, column number and channel number corresponding to the weight value.

[0084] Step S205c: determining a data set to be processed that has the same row width, column width, and number of channels as the sub-convolution kernel from the input original data set to be processed.

[0085] Step S206c: establishing a correspondence between the address of each to-be-processed data in the to-be-processed data set and the address of the corresponding weight value in the sub-convolution kernel.

[0086] Here, the address of the weight value corresponds to the address of the data to be processed, and the corresponding data to be processed can be obtained according to the corresponding relationship.

[0087] Step S207c: acquiring the data to be processed of the corresponding address in the set of data to be processed according to the corresponding relationship.

[0088] Step S208c: Obtain the cumulative sum of all the to-be-processed data corresponding to all the weight values ​​in each category of weight values ​​according to the indicator.

[0089] In an embodiment of the present invention, when reading the mapping table corresponding to the same indicator, the classification accumulation unit of the processing unit performs calculation preprocessing and addition / subtraction operations. When the mapping table within the same indicator is read, that is, when the end condition of the same weight is not reached, according to different implementation strategies, that is, weight reordering strategies, the classification accumulation unit of the processing unit performs calculation preprocessing, that is, shift operation and addition / subtraction operation, wherein the shift amount depends on the configuration in the indicator table, wherein the addition and subtraction depend on the configuration in the indicator table; when the mapping table corresponding to the same indicator is read, the classification end flag is checked, and if the classification end flag is not finished, the next indicator is read; when the mapping table within the same indicator is read, the classification end flag is checked, and if the classification end flag is finished, the result of the accumulation / subtraction, the corresponding weight value and the classification accumulation processing unit number are sent to the classification multiplication and addition processing unit; at the same time, the next indicator, that is, the indicator corresponding to the next category weight value, is read, and the accumulated value in the classification accumulation processor is cleared to zero, until the classification end flag in the indicator table indicates that the accumulation operation of the to-be-processed data corresponding to the weight values ​​of all categories is completed.

[0090] In other embodiments, obtaining the cumulative sum of all data to be processed corresponding to all weight values ​​in each category of weight values ​​according to the indicator includes: preprocessing all data to be processed corresponding to all weight values ​​in each category of weight values ​​according to the indicator; obtaining the cumulative sum of all preprocessed data to be processed, wherein the data to be processed is not zero.

[0091] Here, the preprocessing may be a shift operation, and the specific shift value is obtained from the shift value in the indicator.

[0092] Step S209c: multiply the accumulated sum by the corresponding weight value to obtain a partial sum.

[0093] In other embodiments, multiplying the accumulated sum by the corresponding weight value to obtain the partial sum includes: if it is determined that the accumulated sum is greater than or equal to a preset bias value, multiplying the accumulated sum by the corresponding weight value to obtain the partial sum; and if it is determined that the accumulated sum is less than the preset bias value, terminating the convolution operation.

[0094] In step S210c, all partial sums corresponding to the weight values ​​of all categories are added together to obtain a convolution result.

[0095] In other embodiments, a difference between the partial sum and a preset bias value is determined; if the difference between the partial sum and the preset bias value is determined to be greater than or equal to zero, all partial sums corresponding to the weight values ​​of all categories are added together to obtain a convolution result; if the difference between the partial sum and the preset bias value is determined to be less than zero, the convolution operation is terminated.

[0096] In other embodiments, the corresponding operation of the convolution operation is controlled by a clock gating signal, and the weight classification end point can be used as an enable signal to trigger the clock gating function, and the gating function can be turned off at other times.

[0097] In this embodiment of the present invention, weight conversion is used to pre-process non-zero weights, making the convolution operation time only related to the number of non-zero weights. This fully utilizes the sparsity of the weighted convolution kernel, significantly reducing the inference runtime. At the same time, the new processing unit structure is more scalable and can support functions such as early calculation termination, clock gating, and zero data stop calculation. This means that the sparse nature of data can be utilized to further reduce the amount of computation and reduce power consumption through clock gating. Example 3

[0098] The embodiment of the present invention provides a convolution processing method of a convolutional neural network. Figure 3 A flow chart of a convolution processing method for a convolutional neural network provided in the third embodiment of the present invention is shown in FIG. Figure 3 As shown, the method includes the following steps:

[0099] Step S301: The weight conversion unit performs weight conversion.

[0100] Here, the weight conversion unit can be a tool such as software encoding, script files, or a manual method, which is responsible for splitting and rearranging the convolution kernel. By simulating the processing unit, the convolution kernel is split and rearranged in an optimal way, and the rearranged indicator table and mapping table are stored in the global cache.

[0101] In an embodiment of the present invention, the weights are converted by a compiler, a script, or manually. Taking the use of a compiler for weight conversion as an example, the compiler performs iterative operations to cut a convolution kernel of size R*S*CH into multiple sub-convolution kernels of size R*S*C (where R is the column width of the convolution kernel, S is the row width of the convolution kernel, and C is the number of channels of the sub-convolution kernel. Assuming that the number of channels of the convolution kernel of this layer is Ch, then C<=Ch), and rearranges the fixed-point weight values ​​in each sub-convolution kernel, and converts the arranged weights into the form of an indicator table and a mapping table.

[0102] Figure 4 This is a flow chart of the weight classification and rearrangement provided in the third embodiment of the present invention. In the embodiment of the present invention, a compiler is used to perform weight conversion, wherein the compiler is internally divided into a convolution kernel cutting module, a rearrangement strategy selection module, a sub-convolution kernel rearrangement module, and a rearrangement performance simulation module. Figure 4 As shown, the weight classification rearrangement includes the following steps:

[0103] Step S401: The compiler obtains the original convolution kernel.

[0104] In step S402, the convolution kernel cutting module cuts the original convolution kernel.

[0105] Here, the original convolution kernel is cut by the convolution kernel cutting module to form a sub-convolution kernel. Figure 5 This is a schematic diagram of a structure for cutting an original convolution kernel into sub-convolution kernels provided in the third embodiment of the present invention. Figure 5 It shows the multiple sub-convolution kernels into which the original convolution kernel is divided and the row width 501, column width 502 and number of channels 503 of each sub-convolution kernel.

[0106] Step S403: The rearrangement strategy selection module selects a rearrangement strategy corresponding to the sub-convolution kernel.

[0107] In step S404, the sub-convolution kernel rearrangement module rearranges the sub-convolution kernels according to the rearrangement strategy.

[0108] Step S405 : The rearrangement performance simulation module generates a corresponding indicator table and mapping table according to the rearrangement result.

[0109] Figure 6 This is a schematic diagram of the indicator table and mapping table generated according to the third embodiment of the present invention, as shown in FIG. Figure 6 As shown, there is a corresponding association relationship between the indicator table 601 and the mapping table 602. Figure 7 This is a schematic diagram of the weight table structure after weight rearrangement provided in the third embodiment of the present invention. Figure 7As shown, the indicator table 71 is used to store the weight value 701, the end address 702, the shift value 703, the addition and subtraction type 704, the extension information 705 and the classification indication 706; the mapping table 72 represents the addressing method 707 for the rows, columns and channel numbers of the weights. The encoding method can be represented by L(iwx, jwx, kwx), which represents the addressing method for the rows, columns and channel numbers, where iwx represents the row number corresponding to the weight wx, and the value range is (0, R-1), where jwx represents the column number corresponding to the weight wx, and the value range is (0, R-1); kwx represents the channel number corresponding to the weight wx, and the value range is (0, R-1), and R is the column width of the sub-convolution kernel.

[0110] In other embodiments, the weights may be converted by compiling a script or performing manual operations.

[0111] In the embodiment of the present invention, different rearrangement strategies can be selected. Each rearrangement strategy corresponds to a rearrangement mode. These rearrangement strategies are described using the 8-bit fixed-point weight as an example:

[0112] Method 1: No policy rescheduling and no early stopping function.

[0113] In an embodiment of the present invention, the same weights are rearranged together. Assuming that there are 256 weights, the column width of the convolution kernel corresponding to the weights is R, the row width is S, and the number of channels is C. Through this embodiment of the present invention, the original R*S*C multiplication and addition operations can be converted into a maximum of R*S*C+256 addition operations and 256 multiplication operations.

[0114] Method 2: No policy rescheduling, and early stopping is enabled.

[0115] In this embodiment of the present invention, only identical weights are rearranged together, with positive weights arranged first and negative weights arranged later. For example, if there are 256 weights, and the convolution kernel corresponding to these weights has a column width of R, a row width of S, and a number of channels of C, then this embodiment of the present invention can reduce the original requirement of R*S*C multiplication and addition operations to a maximum of R*S*C+256 addition operations and 256 multiplication operations. Furthermore, when the cumulative sum of the convolution calculation is less than the bias, the convolution operation can be stopped early.

[0116] Figure 8 This is a schematic diagram of the weight table structure after weight rearrangement in the second embodiment of the present invention. Figure 8As shown, each row in the indicator table 81 represents an indicator corresponding to a weight classification. The first row includes: weight value 1, end address is weight 1 end address, no shift, add operation, extended information, weight and classification end indication; the second row includes: weight value 2, end address is weight 2 end address, no shift, add operation, extended information, weight and classification end indication; the third row includes: weight value 3, end address is weight 3 end address, no shift, add operation, extended information, weight and classification end indication; the fourth row includes: weight value -1, end address is weight -1 end address, no shift, subtraction operation, extended information, and rearrangement end indication. The mapping table 82 represents the addresses of the weight values ​​of each type of weight corresponding to each indicator.

[0117] Method 3: Prioritize reducing type reordering and disable early stopping.

[0118] In an embodiment of the present invention, the weight types can be reduced to 64 or fewer by merging weights with the same positive and negative absolute values, merging weights that are multiples of 2, and so on. Taking the reduction to 64 categories as an example, in this case, the early stopping function of the convolution is not enabled. Assuming that the column width of the convolution kernel corresponding to the weight is R, the row width is S, and the number of channels is C, through an embodiment of the present invention, the original R*S*C multiplication and addition operations can be converted to only R*S*C+64 addition operations and 64 multiplication operations at most.

[0119] Figure 9 This is a schematic diagram of the weight table structure after weight rearrangement in the third embodiment of the present invention. Figure 9 As shown, each row in the indicator table 91 represents an indicator corresponding to a weight classification. The first row includes: weight value 1, end address is weight 1 end address, no shift, add operation, extended information, and weight end indication; the second row includes: weight value -1, end address is weight -1 end address, no shift, subtraction operation, extended information, and weight end indication; the third row includes: weight value 2, end address is weight 2 end address, shift 1, add operation, extended information, and weight end indication; the fourth row includes: weight value -64, end address is weight -64 end address, shift 6, subtraction operation, extended information, and weight end indication; the fifth row includes: weight value 3, end address is weight 3 end address, no shift, add operation, extended information, and weight end indication; the sixth row includes: weight value n-1, end address is weight n-1 end address, shift x, add / subtract operation, extended information, and rearrangement end indication. The mapping table 92 represents the addresses of the weight values ​​of each type of weight corresponding to each indicator.

[0120] Method 4: Enable the early stopping function and merge types as much as possible.

[0121] In an embodiment of the present invention, positive weights and negative weights are merged separately, with positive weight types arranged in front and negative weights arranged in the back, which can be compatible with the characteristics of both functions. However, the utilization rate of multiplication for weight rearrangement in this manner is higher than that of multiplication in the third manner.

[0122] Figure 10 This is a schematic diagram of the weight table structure after weight rearrangement in the fourth embodiment of the present invention, as shown in FIG. Figure 10 As shown, each row in the indicator table 1001 represents an indicator corresponding to a weight classification. The first row includes: weight value 1, end address is weight 1 end address, no shift, add operation, extended information, weight end indication; the second row includes: weight value 2, end address is weight 2 end address, shift 1, add operation, extended information, weight end indication; the third row includes: weight value 64, end address is weight 64 end address, shift 6, add operation, extended information, weight and classification end indication; the fourth row includes: weight value 3, end address 4 The bundle address is the end address of weight 3, no shift, an addition operation is performed, extended information, and a weight end indication. The fifth row includes: a weight value of 127, an end address of weight 127, no shift, an addition operation is performed, extended information, and a weight and classification end indication. The sixth row includes: a weight value of -1, an end address of weight -1, no shift, a subtraction operation is performed, extended information, and a weight end indication. The seventh row includes: a weight value of -128, an end address of weight -128, no shift, a subtraction operation is performed, extended information, and a rearrangement end indication. Mapping table 1002 represents the addresses of the weight values ​​of various weight types corresponding to each indicator.

[0123] Step S302: The controller performs command parsing and parameter configuration.

[0124] here, Figure 11 A schematic diagram of the convolution processing structure of the convolutional neural network provided in the third embodiment of the present invention is shown in FIG. Figure 11 As shown, the controller in the control unit 1101 parses the instruction 1102 and then configures parameters for the processing unit 1103 , and configures functional parameters for the processing unit by selecting input data blocks from the data 1105 in the global cache in the cache unit 1104 .

[0125] Step S303: data blocks and weight indicators are imported.

[0126] Here, for a scheduled execution of a processing unit, data of the same size as the split sub-convolution kernel, that is, with a column width of R, a row width of S, and a number of channels of C (R*S*C), is read from the global cache and stored in the data cache (where R is the column width of the sub-convolution kernel, S is the row width of the sub-convolution kernel, and C is the number of channels of the sub-convolution kernel). The weight indicator of the sub-convolution kernel to be processed is read from the global cache into the indicator cache.

[0127] Step S304: read the data stream.

[0128] Here, the weighted re-arranged mapping table contents are read sequentially, and the mapping table contents are used as an address or selection condition to read data from the processor's internal cache. In an embodiment of the present invention, the controller of the processing unit reads the first indicator into the controller, and then reads the weighted re-arranged mapping table contents sequentially, and uses the mapping table contents as an address / selection condition to read data from the data cache.

[0129] Step S305: the classification accumulation unit performs classification accumulation operations.

[0130] Here, the classification accumulation unit mainly completes the accumulation operation of a certain weight classification, and also supports zero data skipping operation.

[0131] In the embodiment of the present invention, when reading the mapping table entries corresponding to the same indicator table entry, the classification accumulation unit of the processing unit performs calculation preprocessing and addition / subtraction operations. Figure 12 A schematic diagram of the convolution processing structure of another convolutional neural network provided in the third embodiment of the present invention is shown in FIG. Figure 12 As shown, the cache unit includes: a global cache 1201, a weight indicator cache 1202, a weight mapping table cache 1203 and a data cache 1204; the processing unit 122 includes a classification accumulation unit 1205 and a classification multiplication and partial sum accumulation unit 1206, and the control unit 123 is used to receive and send control instructions for controlling each unit.

[0132] In other embodiments, Figure 13 A schematic diagram of the convolution processing structure of another convolutional neural network provided in the third embodiment of the present invention is shown in FIG. Figure 13As shown, processing unit 13 includes a classification accumulation unit 131 and a classification multiplication and partial sum accumulation unit 132. Classification accumulation unit 131 includes a controller module 1301, a classification accumulation module 1302, a preprocessing module 1303, a zero determination stop module 1304, and an addition / subtraction operation module 1305; classification multiplication and partial sum accumulation unit 132 includes a stop determination module 1306, a multiplication module 1307, a partial sum module 1308, and an addition module 1309. Preprocessing and addition / subtraction operations are performed in classification accumulation unit 131 of processing unit 13. Depending on the implementation strategy, namely the weight reordering strategy, preprocessing, namely shifting operations and addition / subtraction operations, is performed in classification accumulation unit 131 of processing unit 13. The shift amount depends on the configuration in the indicator table; the addition / subtraction operation depends on the configuration in the indicator table. When the input data is 0, the addition / subtraction operation is paused or stopped. The classification multiplication and partial sum accumulation unit 132 is used to complete the multiplication operation of the accumulated sum output by the classification accumulation unit 131 and the corresponding weight, and to accumulate the multiplication result into the partial sum. It also supports clock gate control function and early stop function.

[0133] Step S306: The classification accumulation unit determines whether the classification accumulation is completed.

[0134] Here, when all mapping table entries corresponding to the same indicator table entry are read, the classification end flag is checked. If the classification end flag indicates that the classification is not ended, the next indicator table entry is read and step S305 is repeated.

[0135] Step S307: Category switching control.

[0136] Here, when the mapping table items in the same indicator are read, the classification end flag is checked. If the classification end flag is ended, the accumulation / subtraction result, the corresponding weight value and the classification accumulation processing unit number are sent to the classification multiplication and addition processing unit; at the same time, the next indicator table item is read, the accumulation value in the classification accumulation processor is cleared, and the operation of step S305 is restarted until the classification end flag in the indicator table item indicates that the entire convolution operation is completed.

[0137] Step S308: the classification multiplication and partial sum accumulation unit performs classification multiplication.

[0138] Here, the classification multiplication and partial sum accumulation unit mainly completes the multiplication operation of the accumulated sum output by the classification accumulation unit and the corresponding weight, and accumulates the multiplication result into the partial sum. It also supports clock gating control and early stopping functions.

[0139] In this embodiment of the present invention, the classification multiplication and addition processing unit performs a multiplication operation on the input weights and data and accumulates the multiplication result into the partial sum. If the configuration selects whether to determine whether to stop the operation early, the difference between the current partial sum value and the bias value is determined to be less than 0. If so, feedback is provided to the controller of the processing unit to stop the processing.

[0140] Clock gating uses the end point of the rearranged weight classification as the enable signal; the clock can be turned off at other times. Depending on the different implementations provided in step S301, the classification multiplication unit is configured to determine whether it can prematurely stop the operation. If premature stopping is enabled, a determination is made as to whether the difference between the current partial sum and the offset value is less than zero. If so, this difference is fed back to the processing unit controller, halting processing. If this function is not enabled, no determination of the partial sum and the offset value is performed.

[0141] In an embodiment of the present invention, an implementation device of a sparse convolutional neural network includes: a weight conversion unit, a cache unit, a control unit, and a processing unit.

[0142] The weight conversion unit can be a tool such as a software compiler, a script file, or can be manually used. It is responsible for splitting and rearranging the convolution kernel. By simulating the processing unit, the convolution kernel is split and rearranged in an optimal way. The rearranged indicator table and mapping table are stored in the global cache.

[0143] The processing unit is divided into two parts, the classification accumulation unit and the classification multiplication and partial sum accumulation unit.

[0144] The classification accumulation unit is used to complete the accumulation operation of a certain weight classification and supports zero data skipping operations.

[0145] The classification multiplication and partial sum accumulation unit is used to complete the multiplication operation of the partial sum output by the classification accumulation unit and the corresponding weight, and to accumulate the multiplication result into the partial sum. It also supports clock gating control and early stopping functions.

[0146] The cache unit can adopt a single-level structure or a multi-level cache structure. Take the second-level cache as an example:

[0147] The global cache provides a scheduling for the data cache, weight indicator cache and weight mapping cache, that is, the data and weight indicator table and weight mapping table required for the convolution operation of the sub-convolution kernel.

[0148] The weight mapping cache sends the weight mapping table value to the data cache as the address of the data cache, reads the data at the corresponding position, and sends it to the classification accumulation unit.

[0149] The weight indicator cache sends the indicator to the classification accumulation unit to control the internal operation of the classification accumulation unit and whether the calculation result of the classification accumulation unit needs to be sent to the classification multiplication unit for further calculation.

[0150] In this embodiment of the present invention, by combining weight conversion with a new processing unit, the weights are compressed in advance to non-zero values, making the convolution operation time only related to the number of non-zero weights. This fully utilizes the sparsity of the weighted convolution kernel, significantly saving inference runtime. At the same time, the new processing unit structure is more scalable and can support functions such as early calculation termination, clock gating, and zero data stop calculation. This means that the sparse nature of data can be utilized to further reduce the amount of computation and reduce power consumption through clock gating. Example 4

[0151] Based on the foregoing embodiments, an embodiment of the present invention provides a convolution processing device for a convolutional neural network. Each unit included in the device, and each sub-unit included in each unit, can be implemented by a processor of a computer device; of course, it can also be implemented by a logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc., wherein the computer device can be a mobile phone, a notebook, a desktop computer, or a server.

[0152] Figure 14 A schematic diagram of the structure of a convolution processing device for a convolutional neural network provided in the fourth embodiment of the present invention is shown in FIG. Figure 14 As shown, the device includes:

[0153] A classification unit 1401 is configured to classify the weight values ​​in the subconvolution kernels of the convolutional neural network;

[0154] A generating unit 1402 is configured to generate, for each weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value;

[0155] An acquiring unit 1403 is configured to acquire corresponding data to be processed according to the address of the weight value;

[0156] The convolution processing unit 1404 is configured to perform a convolution operation on the data to be processed according to the indicator to obtain a convolution result.

[0157] In other embodiments, the device further comprises:

[0158] A cutting unit, configured to cut the original convolution kernel in the convolutional neural network to obtain at least two sub-convolution kernels;

[0159] The fixed-point unit is configured to perform fixed-point processing on the original weight value in each sub-convolution kernel to obtain the weight value in the sub-convolution kernel.

[0160] In other embodiments, the acquiring unit is further configured to:

[0161] Determine a data set to be processed from the input original data set to be processed, which has the same column width, row width and number of channels as the subconvolution kernel;

[0162] Establishing a correspondence between the address of each to-be-processed data in the to-be-processed data set and the address of the corresponding weight value in the sub-convolution kernel;

[0163] The data to be processed of the corresponding address in the set of data to be processed is obtained according to the corresponding relationship.

[0164] In other embodiments, the convolution processing unit is further configured to:

[0165] Obtaining the cumulative sum of all to-be-processed data corresponding to all weight values ​​in each category of weight values ​​according to the indicator;

[0166] Multiplying the accumulated sum by the corresponding weight value to obtain a partial sum;

[0167] Add up all the partial sums corresponding to the weight values ​​of all categories to get the convolution result.

[0168] In other embodiments, the convolution processing unit is further configured to:

[0169] All the data to be processed corresponding to all the weight values ​​in each category of weight values ​​are preprocessed according to the indicator; and a cumulative sum of all the preprocessed data to be processed is obtained, wherein the data to be processed is not zero.

[0170] In other embodiments, the convolution processing unit is further configured to:

[0171] If it is determined that the accumulated sum is greater than or equal to the preset bias value, the accumulated sum is multiplied by the corresponding weight value to obtain a partial sum.

[0172] In other embodiments, the convolution processing unit is further configured to:

[0173] determining a difference between the partial sum and a preset offset value;

[0174] If it is determined that the difference between the partial sum and the preset bias value is greater than or equal to zero, all partial sums corresponding to the weight values ​​of all categories are added together to obtain a convolution result.

[0175] In other embodiments, the classification unit is further configured to:

[0176] Classify the same weight values ​​in the sub-convolution kernels into one category; or,

[0177] Group weights with the same absolute value into one category; or,

[0178] Group weight values ​​that are multiples of a power of 2 into one category; or,

[0179] Positive weight values ​​and negative weight values ​​are treated as one category each.

[0180] It should be noted that in the embodiment of the present invention, if the convolution processing method of the above-mentioned convolutional neural network is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device to execute all or part of the apparatus described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0181] Correspondingly, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the convolution processing method of the convolutional neural network provided in any of the above embodiments.

[0182] Correspondingly, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it implements the steps in the convolution processing method of the convolutional neural network provided in any of the above embodiments.

[0183] The description of the embodiment of the device of the present invention is similar to the description of the above-mentioned embodiment of the device and has similar beneficial effects as the embodiment of the device. For technical details not disclosed in the embodiment of the device of the present invention, please refer to the description of the embodiment of the device of the present invention for understanding.

[0184] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above device embodiment and has similar beneficial effects as the device embodiment. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the device embodiment of the present invention for understanding.

[0185] Of course, the device in the embodiment of the present invention may also have other similar protocol interaction implementation cases. Without departing from the spirit and essence of the present invention, technicians in this field can make various corresponding changes and deformations based on the embodiment of the present invention, but these corresponding changes and deformations should all fall within the scope of protection of the device of the present invention.

[0186] Those skilled in the art will appreciate that embodiments of the present invention may be provided as devices, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage) containing computer-usable program code.

[0187] The present invention is described with reference to flowcharts and / or block diagrams of apparatuses, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that performs the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0190] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0191] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or device comprising the element.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical, or other forms.

[0193] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0194] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0195] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned device embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned device embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0196] Alternatively, if the integrated units described above are implemented as software functional units and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a server to execute all or part of the apparatus described in each embodiment of the present invention. The aforementioned storage media include various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0197] The above description is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A convolution processing method for a convolutional neural network, characterized in that: Applied to a scenario of processing image data, the method includes: Classifying weight values ​​in subconvolution kernels of the convolutional neural network; generating, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value; Acquire corresponding data to be processed according to the address of the weight value, wherein the data to be processed is the image data; Performing a convolution operation on the data to be processed according to the indicator to obtain a convolution result; The operations corresponding to the indicators include: weight value, end address, shift value and addition and subtraction type; The classification of the weight values ​​in the sub-convolution kernels in the convolutional neural network includes: classifying weight values ​​with the same absolute value into one category; or classifying weight values ​​that are multiples of a power of 2 into one category; or classifying positive weight values ​​and negative weight values ​​into one category respectively.

2. The method according to claim 1, characterized in that The method further comprises: Cutting the original convolution kernel in the convolutional neural network to obtain at least two sub-convolution kernels; The original weight value in each of the sub-convolution kernels is fixed-point processed to obtain the weight value in the sub-convolution kernel.

3. The method according to claim 1 or 2, characterized in that The step of obtaining corresponding data to be processed according to the address of the weight value includes: Determine a data set to be processed from the input original data set to be processed, which has the same column width, row width and number of channels as the subconvolution kernel; Establishing a correspondence between the address of each to-be-processed data in the to-be-processed data set and the address of the corresponding weight value in the sub-convolution kernel; The data to be processed of the corresponding address in the set of data to be processed is obtained according to the corresponding relationship.

4. The method according to claim 3, characterized in that The performing a convolution operation on the data to be processed according to the indicator to obtain a convolution result includes: Obtaining the cumulative sum of all to-be-processed data corresponding to all weight values ​​in each category of weight values ​​according to the indicator; Multiplying the accumulated sum by the corresponding weight value to obtain a partial sum; Add up all the partial sums corresponding to the weight values ​​of all categories to get the convolution result.

5. The method according to claim 4, characterized in that The step of obtaining the cumulative sum of all to-be-processed data corresponding to all weight values ​​in each category of weight values ​​according to the indicator includes: All the data to be processed corresponding to all the weight values ​​in each category of weight values ​​are preprocessed according to the indicator; and a cumulative sum of all the preprocessed data to be processed is obtained, wherein the data to be processed is not zero.

6. The method according to claim 4, characterized in that The multiplying the accumulated sum by the corresponding weight value to obtain a partial sum includes: If it is determined that the accumulated sum is greater than or equal to the preset bias value, the accumulated sum is multiplied by the corresponding weight value to obtain a partial sum.

7. The method according to claim 4, characterized in that The convolution result is obtained by adding the partial sums corresponding to the weight values ​​of all categories, including: determining a difference between the partial sum and a preset offset value; If it is determined that the difference between the partial sum and the preset bias value is greater than or equal to zero, all partial sums corresponding to the weight values ​​of all categories are added together to obtain a convolution result.

8. The method according to claim 1, characterized in that The method further comprises classifying the weight values ​​in the subconvolution kernels of the convolutional neural network: The same weight values ​​in the sub-convolution kernels are classified into one category.

9. A convolution processing device for a convolutional neural network, characterized in that: Applicable to a scenario of processing image data, the device includes: a classification unit configured to classify weight values ​​in the subconvolution kernels of the convolutional neural network; a generating unit configured to generate, for the corresponding weight value, an indicator for storing a corresponding operation on the data and an address for representing the weight value according to the classification result of each weight value; an acquiring unit configured to acquire corresponding data to be processed according to the address of the weight value, wherein the data to be processed is the image data; a convolution processing unit, configured to perform a convolution operation on the data to be processed according to the indicator to obtain a convolution result; The operations corresponding to the indicators include: weight value, end address, shift value and addition and subtraction type; The classification unit is further configured to classify weight values ​​with the same absolute value into one category; or classify weight values ​​that are multiples of a power of 2 into one category; or classify positive weight values ​​and negative weight values ​​into one category respectively.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are configured to execute the convolution processing method of the convolutional neural network provided by any one of claims 1 to 8 above.

Images