Preprocessing method and acceleration method of convolutional neural network, acceleration system and medium

Abstract

The present disclosure provides a pre-processing method for a convolutional neural network. The convolutional neural network comprises a convolutional layer, which comprises a convolution kernel comprising weight values formed as a plurality of weight matrices. The pre-processing method comprises: dividing the convolution kernel into a plurality of weight groups; sorting the plurality of weight groups; determining a potential contribution value factor for each weight group. The present disclosure also provides an acceleration method for the pre-processed convolutional neural network, comprising: obtaining the sorting and the potential contribution factor for each weight group; generating a potential contribution value for each weight group; and performing convolution calculation on each weight group with input data based on the sorting, wherein, after each convolution calculation of a weight group with the input data is completed, it is determined whether the convolution operation can be terminated in advance. Furthermore, the present disclosure also relates to an acceleration system for a convolutional neural network, and a computer readable storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a preprocessing method and acceleration method for convolutional neural networks, an acceleration system for convolutional neural networks, and a computer-readable storage medium. Background Technology

[0002] Currently, convolutional neural networks (CNNs) are widely used in fields such as computer vision, object and speech recognition, medical diagnosis, games, and robotics due to their privacy, reliability, low latency, high efficiency, and customizability. The core of a CNN is the convolution operation, which requires processing a large number of parameters and employing numerous parallel multiplier units. After the convolution calculation, the CNN applies an activation function to the result; for example, the ReLU function (Rectified LinearUnit) is one of the most commonly used. Since the convolution results of the intermediate layers of a CNN may contain a large number of negative values, these negative values ​​become zero after processing by the ReLU function, making this part of the computation meaningless and wasting power. Therefore, there is a need to provide a way to accelerate the computation of CNNs, reduce system power consumption, and maintain the accuracy of the CNN. Summary of the Invention

[0003] According to one aspect of this disclosure, a preprocessing method for a convolutional neural network is provided, wherein the convolutional neural network includes a convolutional layer, the convolutional layer includes a convolutional kernel, the convolutional kernel includes weight values ​​formed as at least one weight matrix, and the preprocessing method includes the following steps: dividing the convolutional kernel into multiple weight groups, each weight group including at least one weight matrix; sorting the multiple weight groups based on the weight values ​​of each weight group; and determining the potential contribution factor of each weight group based on the extreme values ​​estimated by the weight values ​​of each weight group.

[0004] In some exemplary embodiments, the convolution kernel is divided into multiple weighted reassemblies based on the processing power of the hardware resources, and the number of weight matrices included in each weighted reassembly corresponds to the processing power of the hardware resources.

[0005] In some exemplary embodiments, the method further includes: dividing the parallel input channels of the convolutional neural network used for input data into multiple channel groups based on the processing power of hardware resources, wherein each channel group corresponds one-to-one with a multiple weight group of the convolutional kernel, wherein the number of input channels in each channel group is equal to the number of weight matrices in the corresponding weight group.

[0006] In some exemplary embodiments, the processing power of a hardware resource includes the number of parallel processing channels that the hardware resource has.

[0007] In some exemplary embodiments, the step of sorting the plurality of weighted sets based on the weight values ​​of each weight group includes: calculating the sum of all positive weight values ​​in each weighted set; calculating the sum of all negative weight values ​​in each weighted set; determining the priority of each weighted set based on the sum of all positive weight values ​​and the sum of all negative weight values ​​in each weighted set; and sorting the plurality of weighted sets based on the priority of each weighted set.

[0008] In some exemplary embodiments, the step of determining the priority of each weight group based on the sum of all positive weight values ​​and the sum of all negative weight values ​​in each weight group includes: weighting the absolute value of the sum of all negative weight values ​​in each weight group with the sum of all positive weight values ​​to determine the priority of each weight group.

[0009] In some exemplary embodiments, the step of determining the potential contribution factor of each weight reorganization based on the extreme values ​​estimated by the weight values ​​of each weight group includes determining the potential contribution factor as one of the following: a weighted sum of the sum of all positive weight values ​​and the sum of all negative weight values ​​in each weight reorganization; the positive weight value with the largest absolute value in each weight reorganization multiplied by the number of positive weight values ​​in that weight reorganization; or the negative weight value with the largest absolute value in each weight reorganization multiplied by the number of negative weight values ​​in that weight reorganization.

[0010] According to another aspect of this disclosure, an acceleration method for convolutional neural networks is provided, which is applied to a convolutional neural network preprocessed by the preprocessing method described above. The acceleration method includes the following steps: obtaining the ranking and potential contribution factors of each weight group; generating potential contribution values ​​for each weight group based on the input data and the potential contribution factors, or based on the maximum value of the input data and the potential contribution factors preset by the system specifications; performing convolution calculations on each weight group one by one with the corresponding part of the input data based on the ranking, wherein, when each weight group is completed with the corresponding part of the input data, the sum of the current part corresponding to all calculated weight groups is compared with the sum of the potential contribution values ​​of all remaining weight groups; when a predetermined early termination condition is met, the convolution operation is terminated and the sum of the current part is taken as the result of the convolution operation.

[0011] In some exemplary embodiments, the sign of the sum of the potential contribution values ​​of all residual right reorganizations is positive, so the predetermined early termination condition includes: if the sum of the current part and the sum of the potential contribution values ​​of all residual right reorganizations is less than or equal to a first threshold, then the convolution operation is terminated.

[0012] In some exemplary embodiments, the first threshold is zero.

[0013] In some exemplary embodiments, the sign of the sum of the potential contribution values ​​of all residual right reorganizations is negative, so the predetermined early termination condition includes: if the sum of the current partial sum and the sum of the potential contribution values ​​of all residual right reorganizations is greater than or equal to a second threshold, then the convolution operation is terminated.

[0014] In some exemplary embodiments, the second threshold is 255.

[0015] According to another aspect of this disclosure, an acceleration system for a convolutional neural network is provided, wherein the convolutional neural network includes convolutional layers, each convolutional layer includes a convolutional kernel, the convolutional kernel including weight values ​​formed as a plurality of weight matrices, the acceleration system comprising: a preprocessor configured to acquire input data, and for the convolutional neural network: dividing the convolutional kernel into a plurality of weight groups, each weight group including a plurality of weight matrices; sorting the plurality of weight groups based on the weight values ​​of each weight group; determining a potential contribution factor of each weight group based on the estimated extreme values ​​of the weight values ​​of each weight group; and a storage unit configured to store the input data and the convolutional layers of the convolutional neural network. The system includes: a ranking and potential contribution value of each weight group; and a calculation unit configured to: obtain the ranking and potential contribution factor of each weight group; generate the potential contribution value of each weight group based on the input data and the potential contribution factor, or based on the maximum value of the input data preset by the system specifications and the potential contribution factor; perform convolution calculation on each weight group with the input data one by one based on the ranking, wherein, when each weight group is completed and convolved with the input data, the current part sum corresponding to all calculated weight groups is compared with the sum of the potential contribution values ​​of all remaining weight groups; when a predetermined early termination condition is met, the convolution operation is terminated and the current part sum is taken as the result of the convolution operation.

[0016] In some exemplary embodiments, the number of weight matrices included in each weight reassembly is equal to the number of parallel input channels used for the input data.

[0017] In some exemplary embodiments, the processor is further configured to: calculate the sum of all positive weight values ​​in each weighting group; calculate the sum of all negative weight values ​​in each weighting group; perform a weighted summation of the absolute value of the sum of all negative weight values ​​in each weighting group with the sum of all positive weight values ​​to determine the priority of each weighting group; and sort the plurality of weighting groups based on the priority of each weighting group.

[0018] In some exemplary embodiments, the potential contribution factor of each weighting reorganization is determined as one of the following: a weighted sum of the sum of all positive weights and the sum of all negative weights in each weighting reorganization; the positive weight with the largest absolute value in each weighting reorganization multiplied by the number of positive weights in that weighting reorganization; the negative weight with the largest absolute value in each weighting reorganization multiplied by the number of negative weights in that weighting reorganization.

[0019] In some exemplary embodiments, the computing unit is further configured such that the sum of the potential contribution values ​​of all residual right reorganizations is positive; the predetermined early termination condition includes: if the sum of the current partial sum and the sum of the potential contribution values ​​of all residual right reorganizations is less than or equal to a first threshold, then the convolution operation is terminated.

[0020] In some exemplary embodiments, the first threshold is zero.

[0021] In some exemplary embodiments, the computing unit is further configured such that the sum of the potential contribution values ​​of all residual right reorganizations is negative; the predetermined early termination condition includes: if the sum of the current partial sum and the sum of the potential contribution values ​​of all residual right reorganizations is greater than or equal to a second threshold, then the convolution operation is terminated.

[0022] In some exemplary embodiments, the second threshold is 255.

[0023] In some exemplary embodiments, the acceleration system further includes a direct memory access unit configured to retrieve the input data and the ordering and potential contribution values ​​of the weight groups of the convolutional layers of the convolutional neural network from the storage unit.

[0024] In some exemplary embodiments, the acceleration system further includes a microcontroller configured to control the operation of the direct memory access unit and the computing unit.

[0025] According to another aspect of this disclosure, a computer-readable storage medium is provided, which is configured to store computer-executable instructions configured to, when executed on a processor, cause the processor to perform the preprocessing method described above.

[0026] According to another aspect of this disclosure, a computer-readable storage medium is provided, which is configured to store computer-executable instructions configured to, when executed on a processor, cause the processor to perform the acceleration method described above. Attached Figure Description

[0027] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, so as to provide a fuller understanding of the further details, features, and advantages of this disclosure; in the accompanying drawings:

[0028] Figure 1A and Figure 1B This schematically illustrates the process of three-dimensional convolution in a convolutional neural network's convolutional layers.

[0029] Figure 2A preprocessing method for a convolutional neural network according to an exemplary embodiment of the present disclosure is schematically illustrated in the form of a flowchart;

[0030] Figure 3 This schematically illustrates the case of grouping a single convolutional kernel;

[0031] Figure 4 The illustration shows the impact of positive and negative weight scores on the priority of a weight group.

[0032] Figure 5 The diagram schematically illustrates the grouping of parallel input channels for input data in a convolutional neural network, and the grouping and sorting of each convolutional kernel based on the channel groups.

[0033] Figure 6 A sorting method for sorting weight reorganizations according to an exemplary embodiment of the present disclosure is schematically illustrated in flowchart form, which can be applied to... Figure 2 The sorting step in the preprocessing method shown;

[0034] Figure 7 A preprocessing method for a convolutional neural network according to another exemplary embodiment of the present disclosure is schematically illustrated in the form of a flowchart;

[0035] Figure 8 A flowchart schematically illustrates an exemplary embodiment of an acceleration method for a convolutional neural network according to this disclosure;

[0036] Figure 9 It is illustrated schematically in the form of a flowchart. Figure 8 The steps included in the acceleration method for convolutional neural networks are shown below;

[0037] Figure 10 This schematically illustrates a commonly used activation function;

[0038] Figure 11 An acceleration system for a convolutional neural network according to an exemplary embodiment of the present disclosure is schematically illustrated in the form of a block diagram;

[0039] Figure 12 The diagram schematically illustrates the structure of a predictive processing element that can be applied to... Figure 11 The acceleration system shown.

[0040] It should be understood that the contents shown in the accompanying drawings are merely illustrative and therefore need not be drawn to scale. Furthermore, throughout all the accompanying drawings, identical or similar features are indicated by identical or similar reference numerals. Detailed Implementation

[0041] The following description provides detailed specifications of various exemplary embodiments of the present disclosure to enable those skilled in the art to fully understand and implement these exemplary embodiments. It should be understood that these detailed specifications are merely exemplary and do not constitute any limitation on the present disclosure.

[0042] See Figure 1A and Figure 1B It schematically illustrates the process of three-dimensional convolution in the convolutional layers of a convolutional neural network.

[0043] Figure 1A This diagram schematically illustrates the overall process of performing a 3D convolution operation on an input feature map to generate an output feature map. The input feature map is generally 3D, with a width W_in, a height H_in, and a depth Cin. In other words, the input feature map can be viewed as consisting of Cin input feature matrices, each with H_in rows and W_in columns, i.e., H_in × W_in input feature data. The convolutional layer can include Cout convolutional kernels, each of which can also be 3D, with a width R, a height S, and the same depth Cin. That is, each convolutional kernel can also be viewed as consisting of Cin weight matrices, each with S rows and R columns, i.e., S × R weight values. Similarly, the output feature map can also be 3D, with a width W_out, a height H_out, and a depth Cout. In other words, the output feature map can be viewed as consisting of Cout output feature matrices, each with H_out rows and W_out columns, that is, having H_out × W_out output feature data.

[0044] Figure 1B This schematically illustrates the process of convolving an input feature map with a convolution kernel to generate an output feature matrix in the output feature map. Figure 1B The input feature map and the three-dimensional form of the convolutional kernel are respectively compared with... Figure 1A The input feature map and the three-dimensional form of the convolution kernel are the same. As shown in the figure, the convolution kernel can be used to perform convolution operations starting from the top left corner of the input feature map to generate the corresponding output feature data y in the output feature matrix. For example, in Figure 1B In the case shown, the output feature data y can be calculated according to Equation 1 below:

[0045] Formula 1

[0046] Where y represents an output feature data in the output feature matrix, Cin represents the depth of the input feature map and the convolution kernel, R represents the width of the convolution kernel, S represents the height of the convolution kernel, w represents the weight value of the convolution kernel, and Fmap represents the input feature value of the input feature map. After calculating an output feature value, the convolution kernel can move along a predetermined stride. Figure 1B The convolution kernel moves in the direction indicated by the broken arrow and calculates the next output feature value. Figure 1B The direction indicated by the broken arrow, after moving through the entire input feature map, results in all calculated output feature values ​​forming the output feature matrix shown in the figure. This output feature matrix has H_out rows and W_out columns. As a non-restrictive example, with a stride of 1, H_out = H_in - R + 1, and W_out = W_in - S + 1. Subsequently, for the next convolutional kernel, the above steps are repeated, thereby generating a new output feature matrix. Therefore, with Cout convolutional kernels, the convolution process can generate Cout output feature matrices, and these Cout output feature matrices together constitute the output feature matrix. Figure 1A The output feature map is shown in the figure.

[0047] It is easy to recognize that the above convolution operation requires processing a large number of parameters and using a large number of parallel multiplier units, which places high demands on computing power and can easily become a bottleneck in the overall performance of convolutional neural networks.

[0048] See Figure 2 The flowchart schematically illustrates a preprocessing method for a convolutional neural network according to an exemplary embodiment of the present disclosure. Figure 2 As shown, the preprocessing method 100 may include steps 110, 120, and 130:

[0049] Step 110: Divide the convolution kernel into multiple weight reassemblies, each weight reassembly including multiple weight matrices;

[0050] Step 120: Sort the multiple weight groups based on their weight values;

[0051] Step 130: Based on the extreme values ​​estimated by the weight values ​​of each weight group, determine the potential contribution factors of each weight group.

[0052] In step 110, the convolutional kernel can be divided based on hardware characteristics. Each block constitutes a weighted reassembly, which may include multiple weight matrices. (See also...) Figure 3 As a non-restrictive example, it shows the case where a convolutional kernel is divided into four weight groups. Figure 3In the scenario shown, the convolutional kernel can be partitioned based on the processing power of hardware resources (e.g., a preprocessor). For example, it can be partitioned based on the number of parallel processing channels (Cgi) in the hardware resources used for parallel processing of data. That is, the depth of each weight group is equal to the number of parallel processing channels (Cgi) in the hardware resources, and the sum of the depths of all weight groups equals the depth of the convolutional kernel (Cin). Therefore, each weight group can be viewed as consisting of Cgi weight matrices, where each weight matrix has S rows and R columns, i.e., it can have S×R weight values.

[0053] Furthermore, it is easy to understand that when the convolution kernel is divided into multiple weighted groups, the output feature data y in the output feature matrix can be calculated according to the following Equation 2:

[0054] Formula 2

[0055] Where y represents an output feature data in the output feature matrix, Cin represents the depth of the input feature map and the convolution kernel, R represents the width of the convolution kernel, S represents the height of the convolution kernel, Cgi represents the number of parallel processing channels (i.e., the depth of a weight group), w represents the weight value of the convolution kernel, and Fmap represents the input feature value of the input feature map.

[0056] Therefore, Equation 2 describes how to use 3D convolution to calculate the output feature data in the output feature matrix when the convolution kernel is divided into multiple weight groups in a convolutional neural network, given the input feature map. Equation 2 can be further rewritten as follows:

[0057] Formula 3

[0058] Where y represents an output feature data in the output feature matrix, Cin represents the depth of the input feature map and the convolution kernel, R represents the width of the convolution kernel, S represents the height of the convolution kernel, Cgi represents the number of parallel processing channels (i.e., the depth of a weight group), w represents the weight value of the convolution kernel, Fmap represents the input feature value of the input feature map, and bias represents the bias amount.

[0059] Therefore, in Equation 3, This represents the sum of the convolution operations of the first to the Kth weight groups. Let represent the convolution operation portion of the weight groups from K+1 to Cin / Cgi. Therefore, it can be understood from Equation 3 that during the three-dimensional convolution calculation of the weight groups from 1 to Cin / Cgi, when the calculation of a weight group (e.g., the Kth weight group) is completed, the impact of the remaining weight groups (e.g., the K+1 to Cin / Cgi weight groups) on the result of the entire convolution operation can be estimated, so as to decide whether to continue the convolution operation of the remaining weight groups. In this way, the efficiency of the convolution operation can be improved.

[0060] See also Figure 2In step 120, the multiple weight sets are sorted based on the weight values ​​of each weight set. As a non-limiting example, each weight set can be sorted based on its individual weight values. For example, a positive weight score (Score_pos) can be calculated based on all positive weight values ​​in the weight set, and a negative weight score (Score_neg) can be calculated based on all negative weight values. In some cases, the positive weight score (Score_pos) may be related to the potential contribution value of the weight set, or it may be related to the partial sum of the convolution calculation of the weight set; therefore, a higher positive weight score (Score_pos) indicates a higher priority for the weight set. Similarly, in other cases, the negative weight score (Score_neg) may be related to the potential contribution value of the weight set, or it may be related to the partial sum of the convolution calculation of the weight set; therefore, a higher absolute value of the negative weight score (Score_neg) also indicates a higher priority for the weight set. Specifically, as a non-limiting example, when the activation function used in the convolutional neural network is the ReLU function (i.e., the Rectified Linear Unit function), since the ReLU function outputs 0 when the input value is less than or equal to 0, the threshold for prematurely terminating the convolution operation of the remaining weight groups can be set to 0. In the case where the threshold is 0, the positive weight score Score_pos can be correlated with the potential contribution value of the weight group; therefore, the higher the positive weight score Score_pos, the higher the priority. Similarly, the negative weight score Score_neg can be correlated with the partial sum of the convolution calculations for the weight group; therefore, the higher the absolute value of the negative weight score Score_neg, the higher the priority. Furthermore, as another non-limiting example, since feature data often has a limitation on the number of bits—for example, for 8-bit feature data, the maximum value is 255—if the convolution result exceeds this value, it will be truncated to 255. Therefore, in this case, the threshold for prematurely terminating the convolution operation of the remaining weight groups can be set to 255. In the case where the threshold is 255, the negative weight score Score_neg can be correlated with the potential contribution value of the weight group. Therefore, the higher the absolute value of the negative weight score Score_neg, the higher the priority. The positive weight score Score_pos can be correlated with the partial sum of the convolution calculation of the weight group. Therefore, the higher the positive weight score Score_pos, the higher the priority.

[0061] Based on the above analysis, the total weight score Score of each weight group can be determined based on the positive weight score Score_pos and the negative weight score Score_neg of each weight group. Then, the priority of each weight group can be determined based on the total weight score Score.

[0062] Figure 4 This illustration shows the impact of positive weight scores (Score_pos) and negative weight scores (Score_neg) on ​​the priority of a weight group. For example... Figure 4 As shown, when the absolute values ​​of both the positive weight score Score_pos and the negative weight score Score_neg of a weight group are high, the weight group has high priority; when one of the absolute values ​​of the positive weight score Score_pos and the negative weight score Score_neg of a weight group is high and the other is low, the weight group has medium priority; when the absolute values ​​of both the positive weight score Score_pos and the negative weight score Score_neg of a weight group are low, the weight group has low priority.

[0063] In some exemplary embodiments, the total weight score of the weight group can be calculated based on a weighted sum of the positive weight score Score_pos and the negative weight score Score_neg. As a non-limiting example, the total weight score of the weight group can be calculated by weighted summation according to Equation 3 below:

[0064] Formula 4

[0065] Where Score_pos represents the positive weight score of the weight group, Score_neg represents the negative weight score of the weight group, Score represents the total weight score of the weight group, and γ represents a coefficient greater than or equal to 0 and less than or equal to 1.

[0066] See Figure 5 This diagram schematically illustrates the grouping of parallel input channels for input data in a convolutional neural network, and the grouping and sorting of each convolutional kernel based on these channel groups. In some exemplary embodiments, each convolutional kernel can be divided into multiple weighted groups based on the processing power of hardware resources (e.g., a preprocessor), with the number of weight matrices included in each weighted group corresponding to the processing power of the hardware resource. As a non-limiting example, the processing power of the hardware resource may include the number of parallel processing channels that the hardware resource has for performing parallel processing. Figure 1A The input feature map shown typically indicates that a convolutional neural network (CNN) includes a number of parallel input channels equal to its depth Cin, i.e., Cin parallel input channels. It should be understood that hardware resources, such as a preprocessor, are limited in the number of parallel processing channels Cgi available for parallel data processing; generally, this number is less than the number of input channels Cin. Therefore, the Cin parallel input channels of the CNN can be grouped based on the number of parallel processing channels Cgi available for parallel data processing in the hardware resources. Figure 5As shown, the Cin input channels of the convolutional neural network are divided into four channel groups, meaning Cin / Cgi = 4, and each channel group contains Cgi input channels. Then, based on these channel groups, each convolutional kernel of the convolutional neural network can be grouped. For each convolutional kernel, its weight sets correspond one-to-one with the channel groups, and each weight set also contains Cgi weight matrices. For example... Figure 5 As shown, the four weight groups of each convolutional kernel can also be sorted. As a non-limiting example, the total weight score of the weight groups described above can be used to sort the four weight groups of each convolutional kernel. Since the weight matrices included in each convolutional kernel are different, the sorting of the weight groups of each convolutional kernel may also be different.

[0067] It should be understood that in some other exemplary embodiments, the Cin parallel input channels of the convolutional neural network may not be divisible by the number of parallel processing channels Cgi. In this case, the number of input channels included in a channel group in the grouping result may be less than the number of parallel processing channels Cgi, but the number of input channels is still equal to the number of weight matrices in the corresponding weight reassembly. Alternatively, in the above case, the number of parallel input channels Cin of the convolutional neural network can be expanded to a number divisible by the number of parallel processing channels Cgi by padding with zeros, thereby achieving the grouping of input channels. It should be understood that the method of expanding the number of parallel input channels Cin of the convolutional neural network by padding with zeros can be flexibly applied according to actual needs. For example, zero padding can be performed centrally after Cin input channels, or it can be performed intermittently in Cin input channels. This disclosure does not limit the method of padding input channels with zeros, as long as the number of input channels after zero padding is divisible by the number of parallel processing channels.

[0068] See Figure 6 The flowchart schematically illustrates a sorting method for sorting weight reorganizations according to an exemplary embodiment of the present disclosure, which can be applied to... Figure 2 The sorting step 120 in the preprocessing method shown. For example... Figure 6 As shown, sorting method 120' may include steps 121, 122, 123, and 124:

[0069] Step 121: Calculate the sum of all positive weights in each weighting reorganization;

[0070] Step 122: Calculate the sum of all negative weight values ​​in each weighting reorganization;

[0071] Step 123: Determine the priority of each weight group based on the sum of all positive weight values ​​and the sum of all negative weight values ​​in each weight group;

[0072] Step 124: Sort the multiple weight groups according to their priorities.

[0073] It should be understood that in step 121, the sum of all positive weight values ​​in each weight group is calculated to obtain the positive weight score Score_pos for that weight group; in step 122, the sum of all negative weight values ​​in each weight group is calculated to obtain the negative weight score Score_neg for that weight group; and in step 123, based on Equation 4 above, the weighted total score Score of the weight group can be calculated by weighted summation using the obtained positive weight score Score_pos and negative weight score Score_neg, thereby determining the priority of the weight group. However, it should be recognized that the above is merely a non-limiting example of calculating the positive weight score Score_pos, negative weight score Score_neg, and total weight score of a weight group. In fact, any suitable method for calculating the positive weight score Score_pos and negative weight score Score_neg of a weight group, and for calculating the overall score based on them, can be used in the preprocessing methods for preprocessing neural networks according to this disclosure, and this disclosure does not limit this.

[0074] See also Figure 2In step 130, the extreme values ​​estimated based on the weight values ​​of each weight group are used to determine the potential contribution factor of each weight group. As described above, it can be understood from Equation 3 that during the 3D convolution calculation of the 1st to Cin / Cgi weight groups, when the calculation of a weight group (e.g., the Kth weight group) is completed, the impact of the remaining weight groups (e.g., the K+1th to Cin / Cgi weight groups) on the result of the entire convolution operation can be estimated to determine whether to continue the convolution operation of the remaining weight groups. Based on the above understanding, this disclosure introduces the concept of a potential contribution factor for each weight group, so that in the 3D convolution calculation of the convolutional neural network, the concept of a potential contribution value can be introduced for each weight group. The potential contribution value can be equal to the potential contribution factor multiplied by the maximum feature value in the input feature map, so as to evaluate the calculation result of the entire 3D convolution operation. In another exemplary embodiment, the potential contribution value can also be equal to the potential contribution factor multiplied by the maximum value of the input data preset according to the system specifications, so as to evaluate the calculation result of the entire 3D convolution operation. Therefore, during the convolution operation, the overall potential contribution value of all remaining weight groups can be equal to the sum of the potential contribution values ​​of each remaining weight group. As a non-limiting example, the system specification can be a bit limit defined for the data being processed; for example, it can specify that all bits used for the input data are 8 bits. In the case where all bits used for the input data are 8-bit unsigned numbers, the maximum value of the input data preset according to the system specification can be 255. It should be understood that, depending on actual needs, the potential contribution value can also be equal to the potential contribution factor multiplied by other suitable values, and this disclosure does not impose any limitations on this.

[0075] Since the potential contribution value of each weight group can be calculated as the product of the maximum eigenvalue of the input feature map or the maximum value of the input data preset according to the system specifications and the potential contribution factor estimated by the weight value of the weight group, in the preprocessing method 100, the extreme value estimated based on the weight value of the weight group can be determined as the potential contribution factor PC_factor of the weight group.

[0076] As a non-limiting example, the potential contribution factor PC_factor of the weighted reorganization can be a weighted sum of the positive weight scores Score_pos and the negative weight scores Score_neg of the weighted group. For example, the potential contribution factor PC_factor of the weighted reorganization can be calculated according to Equation 5 below:

[0077] PC_factor = α×(Score_pos + β×Score_neg) Equation 5

[0078] Where α and β are coefficients greater than or equal to 0 and less than or equal to 1, respectively. Furthermore, as a non-limiting example, as detailed above, the positive weight score Score_pos in Equation 5 can be calculated as the sum of all positive weight values ​​in the weighted reorganization, and the negative weight score Score_neg can be calculated as the sum of all negative weight values ​​in the weighted reorganization. Additionally, in some other exemplary embodiments, the potential contribution factor PC_factor of the weighted reorganization can be calculated as the positive weight value with the largest absolute value in the weighted reorganization multiplied by the number of positive weight values ​​in that weighted reorganization, or the potential contribution factor PC_factor of the weighted reorganization can be calculated as the negative weight value with the largest absolute value in the weighted reorganization multiplied by the number of negative weight values ​​in that weighted reorganization.

[0079] It should be recognized that, depending on the calculation method, the extreme values ​​estimated based on the weight values ​​of the weight group can have different forms. Therefore, the potential contribution factor PC_factor of the weighted reorganization can also have different forms. Thus, any suitable calculation method for estimating the extreme values ​​based on the weight values ​​of the weight group, and the resulting extreme values, can be used to determine the potential contribution factor PC_factor of the weighted reorganization, and this disclosure does not impose any limitations on this.

[0080] See Figure 7 The flowchart schematically illustrates a preprocessing method for a convolutional neural network according to another exemplary embodiment of the present disclosure, wherein the convolutional neural network may include multiple convolutional layers, and each convolutional layer may include multiple convolutional kernels. Figure 7 As shown, preprocessing method 200 may include steps 210, 220, 230, 240, 250, 260, 270, and 280:

[0081] Step 210: Input the initial model information of the convolutional neural network;

[0082] Step 220: Determine the convolutional layer to be processed and increment the convolutional layer number;

[0083] Step 230: Determine the convolution kernel to be processed and increment the kernel number;

[0084] Step 240: Process the convolution kernel using preprocessing method 100;

[0085] Step 250: Determine whether all convolution kernels have been processed. If yes, proceed to step 260; otherwise, return to step 230.

[0086] Step 260: Update the order of weight groups and potential contribution factors of each convolutional kernel in the convolutional layer;

[0087] Step 270: Determine whether all convolutional layers have been processed. If yes, proceed to step 280; otherwise, return to step 220.

[0088] Step 280: Output the model information of the preprocessed convolutional neural network.

[0089] In step 210, the initial model information of the convolutional neural network may include, but is not limited to, information about multiple convolutional layers of the convolutional neural network and information about multiple convolutional kernels included in each convolutional layer. In step 220, processing is determined to start from the first convolutional layer. Each time a convolutional layer is determined, its index is incremented, for example, by one, to determine the next convolutional layer to be processed. Similarly, in step 230, processing is determined to start from the first convolutional kernel of the convolutional layer. Each time a convolutional kernel is determined, its index is incremented, for example, by one, to determine the next convolutional kernel to be processed. In step 240, the convolutional kernels are processed using the preprocessing method 100 described above. It should be understood that various exemplary embodiments described above regarding the preprocessing method 100 can be applied to step 240 to implement the preprocessing of the convolutional kernels. These exemplary embodiments will not be described in detail here. In step 250, it is determined whether all convolutional kernels have been processed; if not, the remaining convolutional kernels are processed; if so, subsequent steps are executed. In step 260, based on the sorting results in step 240 and the calculated potential contribution factors, the order of the weight groups and potential contribution factors of each convolutional kernel in the convolutional layer are updated to obtain updated model information. In step 270, it is determined whether all convolutional layers have been processed; if not, the remaining convolutional layers are processed; if so, subsequent steps are executed. In step 280, the preprocessed model information of the convolutional neural network can be output to a suitable memory (e.g., but not limited to, external memory) for storage.

[0090] Offline preprocessing of convolutional neural network models can increase the probability of the convolutional neural network system terminating the current convolution operation early, thereby accelerating computation, reducing power consumption, and without affecting the accuracy of model inference.

[0091] See Figure 8 The flowchart schematically illustrates an exemplary embodiment of an acceleration method for a convolutional neural network according to the present disclosure. Figure 8 The acceleration method 300 shown can be applied to convolutional neural networks preprocessed by the preprocessing methods 100 and 200 described above. As shown in the figure, the acceleration method 300 may include steps 310, 320, and 330:

[0092] Step 310: Obtain the ranking and potential contribution factors of each weight group;

[0093] Step 320: Based on the input data and potential contribution factors, generate the potential contribution value for each weighting reorganization; and

[0094] Step 330: Based on the sorting, each weight group is convolved with the input data one by one. When the convolution calculation of each weight group with the input data is completed, the current part sum corresponding to all calculated weight groups is compared with the sum of the potential contribution values ​​of all remaining weight groups. When a predetermined early termination condition is met, the convolution operation is terminated and the current part sum is taken as the result of the convolution operation.

[0095] In step 310, the order of the weight groups of the previously preprocessed convolutional neural network and their potential contribution factors can be obtained from, for example, external memory. As mentioned earlier, the preprocessing method according to this disclosure can be performed offline; that is, the convolutional neural network is preprocessed before input data such as input feature maps is processed, and the resulting model parameters, such as the order of the weight groups of the convolutional neural network and their potential contribution factors, can be stored in external memory. During the processing of input data such as input feature maps, the model parameters of the convolutional neural network can be read from external memory for application to the acceleration method 300 according to this disclosure.

[0096] In step 320, the potential contribution value of each weighted recombination is generated based on the input data and the potential contribution factor. In the preprocessing method according to this disclosure, the potential contribution factor of each weighted recombination has already been obtained based on the weight values ​​of each weight group. However, the potential contribution value of each weighted recombination depends not only on the potential contribution factor but also on the input data. As a non-limiting example, the potential contribution value of each weighted recombination can be calculated by multiplying the maximum value in the input data by the potential contribution factor of each weighted recombination, i.e.: PC_value(k) = Max(Fmap)×PC_factor(k), where PC_value(k) represents the potential contribution value of the k-th weighted recombination, Max(Fmap) represents the maximum value in the input feature map Fmap, which is the input data, and PC_factor(k) represents the potential contribution factor of the k-th weighted recombination.

[0097] In step 330, each weight group is convolved with the input data one by one based on the ranking. As analyzed in detail above, this ranking indicates the degree of influence of each weight group on the convolution result. The higher the ranking, the greater the influence of the weight group on the convolution result; the lower the ranking, the smaller the influence. In this step, after each weight group is convolved with the input data, the current sum corresponding to all calculated weight groups is compared with the sum of the potential contribution values ​​of all remaining weight groups. When a predetermined early termination condition is met, the convolution operation is terminated, and the current sum is taken as the result of the convolution operation. It should be understood that the concept of early termination in step 330 is essentially based on the understanding that when the sum of the potential contribution values ​​of all remaining weight groups based on extreme value estimation is found to be insufficient to change the output result of the convolution operation of each weight group after being subjected to the activation function, then the calculation of these remaining weight groups is meaningless, and therefore these meaningless calculations can be omitted.

[0098] See Figure 9 It is illustrated schematically in the form of a flowchart. Figure 8 Step 330 in the acceleration method shown includes the following steps. For example... Figure 9 As shown, Figure 8 Step 330 in the acceleration method shown may further include steps 331, 332, 333, 334, 335, and 336:

[0099] Step 331: Determine the weighting group for the calculation and sort the weighting groups in ascending order;

[0100] Step 332: Perform convolution calculation between the weighted group and the input data;

[0101] Step 333: Calculate the current partial sum corresponding to all calculated weight groups, and calculate the sum of the potential contribution values ​​of all residual weight reorganizations;

[0102] Step 334: Determine whether the predetermined early termination conditions are met. If not, proceed to step 335; if yes, continue to step 336.

[0103] Step 335: Determine whether all ownership reorganizations have been calculated. If not, return to step 331; if yes, continue to step 336.

[0104] Step 336: Output the current part as the result of the convolution operation.

[0105] In step 331, the calculation begins with the highest-ranked weight group in the weight reassembly. Each time a weight group is determined, its ranking is incremented (e.g., by one) to determine the next weight group to be calculated. In step 332, a convolution operation is performed using the weight group determined in step 331 and the input data. It should be understood that any suitable convolution operation method can be applied. In step 333, the current partial sum corresponding to all calculated weight groups and the sum of the potential contribution values ​​of all remaining weight reassemblies are calculated to determine whether the convolution operation needs to be terminated early. In a non-limiting example, starting with the highest-ranked weight group, after each weight group is calculated, the current calculation result is added to the previous calculation result to generate the current partial sum of all calculated weight groups. Simultaneously, the potential contribution value corresponding to the currently calculated weight group is subtracted from the sum of the potential contribution values ​​of all previous remaining weight reassemblies to generate the sum of the potential contribution values ​​of all remaining weight reassemblies. In step 334, the predetermined early termination condition can be set according to actual needs, as long as it is based on the sum of the current partial sums of all calculated weight groups and the sum of the potential contribution values ​​of all residual weight reorganizations. This will be described in further detail below. In step 335, it is determined whether all reorganizations have been calculated. In step 336, the current partial sum is output as the result of the convolution operation.

[0106] As described earlier, the concept of early termination is essentially based on the understanding that when the sum of the potential contribution values ​​of all remaining weight groups, estimated based on extreme values, is insufficient to change the output of the convolution operation after the activation function, the calculations performed on these residual weight recombinations are meaningless and can therefore be omitted. As a non-limiting example, when the sign of the sum of the potential contribution values ​​of all residual weight recombinations is positive—for example, when determining the potential contribution value based on the positive weight score Score_pos of the weight group and the maximum eigenvalue Max(Fmap) in the input feature map—the predetermined early termination condition can be set as follows: if the sum of the current partial sum and the sum of the potential contribution values ​​of all residual weight recombinations is less than or equal to a first threshold, then the convolution operation is terminated. Furthermore, as another non-limiting example, when the sum of the potential contribution values ​​of all residual weight recombinations is negative—for example, when determining the potential contribution value based on the negative weight score Score_neg of the weight group and the maximum eigenvalue Max(Fmap) in the input feature map—a predetermined early termination condition can be set as follows: if the sum of the current partial sum and the sum of the potential contribution values ​​of all residual weight recombinations is greater than or equal to a second threshold, then the convolution operation is terminated. It should be understood that the above two predetermined early termination conditions can be used in combination. For example, for all residual weight recombinations, a first potential contribution value with a positive sign and a second potential contribution value with a negative sign can be determined. The current partial sum is added to the sum of the first potential contribution values ​​of all residual weight recombinations; if the result is less than or equal to the first threshold, then the convolution operation is terminated. Similarly, the current partial sum is added to the sum of the second potential contribution values ​​of all residual weight recombinations; if the result is greater than or equal to the second threshold, then the convolution operation is also terminated.

[0107] In convolutional neural networks, the output of a convolutional layer is normalized by a batch normalization layer (also known as a BN layer) before being activated by an activation function. Therefore, the first and second thresholds can be set accordingly based on the activation function used in the convolutional neural network.

[0108] like Figure 10The diagram illustrates a commonly used activation function, ReLU. The ReLU function outputs 0 when the input value is less than or equal to 0. Therefore, when using ReLU as the activation function, a predetermined early termination condition can be set as follows: if the sum of the potential contribution values ​​of all residual weight recombinations is positive, and the sum of the current partial sum and the sum of the potential contribution values ​​of all residual weight recombinations is less than or equal to zero, then the convolution operation terminates. In other words, the first threshold can be set to zero. Furthermore, also with the ReLU function, due to the limitation on the number of digits in the feature data (e.g., for 8-bit feature data, the maximum value is 255), if the convolution result exceeds this value, it will be truncated to 255. Therefore, in this case, a predetermined early termination condition can be set as follows: if the sum of the potential contribution values ​​of all residual weight recombinations is negative, and the sum of the current partial sum and the sum of the potential contribution values ​​of all residual weight recombinations is greater than or equal to the maximum value corresponding to all digits of the feature data, then the convolution operation terminates. In other words, the second threshold can be set to the maximum value corresponding to all digits of the feature data. As a non-limiting example, the feature data can be 8 bits in total, and in this case, the maximum value is 255. Therefore, the second threshold can be set to 255 in this case.

[0109] Furthermore, in some other exemplary embodiments according to this disclosure, the predetermined early termination condition can be set as follows: if the current partial sum has the opposite sign to the sum of the potential contribution values ​​of all residual weight reorganizations, the current partial sum is added to the sum of the potential contribution values ​​of all residual weight reorganizations; if the result has the same sign as the current partial sum, the convolution operation is terminated. It should be understood that the predetermined early termination condition for the convolution operation can be set according to actual needs, as long as it is based on the current partial sum of all calculated weight groups and the sum of the potential contribution values ​​of all residual weight reorganizations; this disclosure does not impose any limitations on this.

[0110] Furthermore, it should be understood that the acceleration method for convolutional neural networks according to this disclosure can be used for real-time processing of input feature map data, while the preprocessing method for convolutional neural networks according to this disclosure can be used for offline preprocessing of convolutional neural networks. The result of offline preprocessing of the convolutional neural network can be stored in external storage so that it can be read and called by the acceleration method when processing the input feature map data in real time.

[0111] The acceleration method for convolutional neural networks disclosed herein can predict and terminate meaningless convolution operations in advance, thereby reducing the number of multiplication operations and the number of data reads, thus accelerating the computation of neural networks, reducing system power consumption, and maintaining the accuracy of the network.

[0112] For example, when the potential contribution value of reorganizing all residual rights is calculated according to the following formula:

[0113] Formula 6

[0114] Where PC_value(k) represents the potential contribution value of all residual weight reorganizations corresponding to the calculated k-th weight reorganization, α and β represent coefficients greater than or equal to 0 and less than or equal to 1, Score_pos represents the positive weight score of the weight group, which can be calculated as the sum of all positive weight values ​​in the weight reorganization, and Score_neg represents the negative weight score of the weight group, which can be calculated as the sum of all negative weight values ​​in the weight reorganization. Simultaneously, the weight reorganizations are sorted according to Equation 4 above. In an implementation without loss of accuracy, the operational savings for each existing neural network are as follows:

[0115]

[0116] In other exemplary embodiments, the coefficients α, β, and γ can be adjusted to further reduce computational complexity. For example, but not limited to, α = 0.75, β = 0.5, and γ = 0.125. Furthermore, in still other exemplary embodiments, the coefficient γ is variable in different sorting strategies; for example, for a first sorting strategy, coefficient γ = 1, while for other sorting strategies, γ = 0.

[0117] See Figure 11 The diagram schematically illustrates an acceleration system for a convolutional neural network according to an exemplary embodiment of the present disclosure. Figure 11 As shown, the acceleration system 400 may include: a preprocessor 410, a storage unit 420, a direct memory access unit (DMA) 430, a microcontroller unit (MCU) 440, an input buffer 450, an output buffer 460, and a computing unit 470. The processor 410, storage unit 420, and DMA unit 430 are interconnected via a data bus, and the MCU 440, DMA unit 430, input buffer 450, and output buffer 460 are also communicatively connected to each other for transmitting instructions and data. Furthermore, the computing unit 470 may be configured to receive instructions from the MCU 440 and data from the input buffer 450 and output buffer 460, and output the calculation results to the output buffer 460.

[0118] The preprocessor 410 can be used to acquire input data and generate instructions based on the model of the convolutional neural network. As a non-limiting example, the preprocessor 410 can also be configured to execute the preprocessing method described in detail above according to this disclosure and output the preprocessed model information of the convolutional neural network to the storage unit 420. The MCU 440 is used for configuring parameters and thresholds for each layer, as well as interrupt handling, etc., and can be configured, for example but not limited to, to control the operation of the DMA unit 430 and the computing unit 470. The preprocessor 410 and the MCU 440 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that operates based on operation instructions.

[0119] Storage unit 420 can be a non-transitory medium capable of storing computer-readable, processor-executable program instructions and data as computer-executable code, and it generally includes both volatile and non-volatile memory (e.g., RAM, ROM). Furthermore, it should be recognized that in some examples, storage unit 420 may also include hard disk drives, solid-state drives, removable media, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc.

[0120] The computation unit 470 can be used to perform operations such as convolution, normalization, activation functions, and pooling on the weight data and input data, and to determine whether to terminate the current convolution operation prematurely. Figure 10 As shown, the computation unit 470 may include a prediction module 471, a scalar module 472, and a pooling module 473. Specifically, the prediction module 471 may be configured to execute the acceleration method according to this disclosure described above, for performing convolution operations on the weight data and the input data, and determining whether to terminate the current convolution operation prematurely. The scalar module 472 may be configured to normalize the convolution operation result of the prediction module 471 and apply an activation function for activation. The pooling module 473 may be configured to perform pooling operations on the output result of the scalar module 472.

[0121] It should be understood that in some non-limiting exemplary embodiments, the preprocessor 410 and the computing unit 470 may be replaced by the same graphics processing unit (i.e., GPU) to perform their respective functions. The GPU can perform offline preprocessing of the convolutional neural network or online processing of the input data. In other non-limiting exemplary embodiments, the acceleration system for the convolutional neural network may omit the DMA unit 430 and its structure and functionality may be integrated into the storage unit 420. Furthermore, in still other non-limiting exemplary embodiments, the MCU 440 may be omitted for the convolutional neural network.

[0122] See Figure 12 The diagram schematically illustrates the structure of a prediction processing element 500, which can be applied to... Figure 10 The prediction module 471 in the acceleration system shown. Figure 11 As shown, the prediction processing element 500 may include a multiplier 510, an adder 520, a comparator 530, and a multiplexer 540 (i.e., a MUX). The multiplier 510, adder 520, comparator 530, and multiplexer 540 can be any suitable hardware device or software module, and this disclosure is not limited thereto. Figure 12 In the prediction processing element 500 shown, multiplier 510 multiplies the input weight values ​​with input data such as an input feature map; then, the product is sent to adder 520 for summation to generate a current partial sum; the current partial sum generated by adder 520 is sent to comparator 530 and multiplexer 540 respectively; comparator 530 receives the current partial sum and also receives a weight recombination index and potential contribution configuration (e.g., the weight recombination index and potential contribution configuration can be obtained from input buffer 450), and determines whether a predetermined early termination condition is met based on the current partial sum and the potential contribution value; if the predetermined early termination condition is met, the convolution operation is terminated, and an early termination message is sent to multiplexer 540; multiplexer 540 receives the current partial sum from adder 520, and also receives a threshold set as needed, so as to determine whether to output the current partial sum or output the input threshold as the result based on the output of comparator 530. Specifically, when multiplexer 540 receives a message indicating no early termination is required, it outputs the current partial sum as the output feature map; when it receives a message indicating early termination, it does not output the current partial sum but instead outputs the input threshold as the result. It should be understood that the input threshold received by multiplexer 540 can be different values ​​as needed, for example, for... Figure 9 The ReLU function shown can have an input threshold of 0 or 255, depending on the actual needs. Regarding convolution operations and the determination of early termination conditions, the prediction processing element 500 can apply any of the aforementioned exemplary embodiments, which will not be elaborated further in this disclosure.

[0123] It should be understood that the above regarding Figure 11 , Figure 12The various modules, devices, units, elements, etc., described can be implemented in hardware or in hardware combined with software and / or firmware. For example, these modules, devices, units, elements, etc., can be implemented as computer-executable code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, in some exemplary embodiments, one or more of these modules, devices, units, elements, etc., can be implemented together in a system-on-a-chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry), and may optionally execute received program code and / or include embedded firmware to perform functions.

[0124] Although the preprocessing method and acceleration method according to this disclosure are in Figure 11 , Figure 12 The preprocessing method and acceleration method, or parts thereof, may be implemented using any form of computer-readable storage medium accessible by a computing device, such as, but not limited to, a computer-readable storage medium storing computer-executable instructions which, when executed on a processor, cause the processor to perform the preprocessing method and acceleration method according to this disclosure.

[0125] It should be understood that computer-readable storage media can include volatile and non-volatile, removable and non-removable storage media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD), or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other storage medium that can be used to store information for access by a computing device.

[0126] Therefore, the acceleration system according to this disclosure requires less additional real-time processing and hardware resources, reduces the number of operations, and can utilize high-speed parallel hardware. Furthermore, the technical solution according to this disclosure proactively and systematically reduces computation while maintaining accuracy by utilizing the potential contributions of offline convolutional kernel grouping and estimation during optimization, as well as early stopping in the convolutional neural network acceleration system.

[0127] The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to limit this disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the terms “comprising” and “including,” when used in this disclosure, refer to the presence of the mentioned feature but do not exclude the presence of one or more other features or the addition of one or more other features. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. It will be understood that although the terms “first,” “second,” “third,” etc., may be used herein to describe various features, these features should not be limited by these terms. These terms are used only to distinguish one feature from another.

[0128] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It shall also be understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0129] In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example that is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0130] This paper describes various techniques within the general context of software and hardware components or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc., that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," and "component" generally refer to software, firmware, hardware, or a combination thereof. The techniques described herein are characterized as platform-independent, meaning that these techniques can be implemented on a variety of computing platforms with various processors.

[0131] The logic and / or steps illustrated in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). Furthermore, it should be understood that the various steps of the methods illustrated in the flowcharts or otherwise described herein are merely exemplary and do not imply that the steps of the illustrated or described methods must be performed in accordance with the steps shown or described. Rather, the various steps of the methods illustrated in the flowcharts or otherwise described herein may be performed in a different order than that shown in this disclosure, or may be performed simultaneously. Additionally, the methods illustrated in the flowcharts or otherwise described herein may include other additional steps as needed.

[0132] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0133] Those skilled in the art should understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware associated with program instructions, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes performing one or a combination of the steps of the method embodiments.

[0134] Although this disclosure has been described in detail with reference to several embodiments, it is not intended to be limited to the specific forms set forth herein. Rather, the scope of this disclosure is defined only by the appended claims.

Claims

1. A pre-processing method for a convolutional neural network, wherein, The convolutional neural network comprises a convolutional layer, the convolutional layer comprises convolution kernels, the convolution kernels comprise weight values formed into at least one weight matrix, and the preprocessing method comprises the following steps: dividing the convolution kernels into a plurality of weight groups based on the processing capacity of a hardware resource, each weight group comprising a number of weight matrices corresponding to the processing capacity of the hardware resource; sorting the plurality of weight groups based on the weight values of the weight groups; the sorting indicates the degree of influence of each weight group on the result of the convolution operation, and the higher the sorting, the greater the influence of the weight group on the result of the convolution operation, and the lower the sorting, the smaller the influence of the weight group on the result of the convolution operation; determining potential contribution factors of the weight groups based on extreme values estimated based on the weight values of the weight groups; wherein the step of determining the potential contribution factors of the weight groups based on the extreme values estimated based on the weight values of the weight groups comprises determining the potential contribution factors as one of the following: a weighted sum of a sum of all positive weight values and a sum of all negative weight values in each weight group; a product of a positive weight value with the largest absolute value in each weight group and a number of positive weight values in the weight group; a product of a negative weight value with the largest absolute value in each weight group and a number of negative weight values in the weight group.

2. The pre-treatment method of claim 1, wherein, The step of sorting the plurality of weight groups based on the weight values of the weight groups comprises: calculating a sum of all positive weight values in each weight group; calculating a sum of all negative weight values in each weight group; determining a priority of each weight group based on the sum of all positive weight values and the sum of all negative weight values in the weight group; sorting the plurality of weight groups based on the priorities of the weight groups.

3. An acceleration method for a convolutional neural network, which is applied to a convolutional neural network preprocessed by the preprocessing method according to any one of claims 1 to 2, and the acceleration method comprises the following steps: obtaining the sorting and the potential contribution factors of the weight groups; generating potential contribution values of the weight groups based on input data and the potential contribution factors, or based on a maximum value of the input data preset according to system specifications and the potential contribution factors; performing convolution calculation on the input data and each weight group in turn based on the sorting, wherein, after completing the convolution calculation of one weight group and the corresponding part of the input data, a sum of a current part corresponding to all the calculated weight groups and potential contribution values of all the remaining weight groups is compared, and when a predetermined early termination condition is met, the convolution operation is terminated, and the current part sum is taken as the result of the convolution operation.

4. The acceleration method according to claim 3, wherein: the sum of the potential contribution values of all the remaining weight groups is positive; the predetermined early termination condition comprises that, when the current part sum added with the sum of the potential contribution values of all the remaining weight groups is less than or equal to a first threshold value, the convolution operation is terminated.

5. The acceleration method according to claim 3, wherein: the sum of the potential contribution values of all the remaining weight groups is negative; The predetermined early termination condition comprises: when the sum of the current partial sum and the sum of the potential contribution values of all the remaining weight groups is greater than or equal to a second threshold value, the convolution operation is terminated.

6. An acceleration system for a convolutional neural network, wherein, The convolutional neural network comprises a convolution layer, the convolution layer comprises convolution kernels, the convolution kernels comprise weight values formed as a plurality of weight matrices, and the acceleration system comprises: a preprocessor configured to obtain input data and, for the convolutional neural network: based on the processing capability of the hardware resource, divide each convolution kernel into a plurality of weight groups, each weight group comprising a number of weight matrices corresponding to the processing capability of the hardware resource; based on the weight values of each weight group, sort the plurality of weight groups; based on the extreme value estimated from the weight values of each weight group, determine the potential contribution factor of each weight group; wherein the potential contribution factor of each weight group is determined as one of: the weighted sum of the sum of all positive weight values and the sum of all negative weight values in each weight group; the maximum positive weight value in each weight group multiplied by the number of positive weight values in the weight group; the maximum negative weight value in each weight group multiplied by the number of negative weight values in the weight group; a storage unit configured to store the input data and the sorting and potential contribution values of each weight group of the convolution layer of the convolutional neural network; a calculation unit configured to: obtain the sorting and potential contribution factors of each weight group from the storage unit; based on the input data and the potential contribution factors, or based on the maximum value of the input data and the potential contribution factors preset by the system specification, generate the potential contribution values of each weight group; based on the sorting, perform convolution calculation on each weight group with the input data one by one, wherein when the convolution calculation of each weight group with the input data is completed, the current partial sum corresponding to all the calculated weight groups and the sum of the potential contribution values of all the remaining weight groups are compared, and when the predetermined early termination condition is met, the convolution operation is terminated and the current partial sum is taken as the convolution operation result.

7. The acceleration system of claim 6, wherein, The processor is further configured to: calculate the sum of all positive weight values in each weight group; calculate the sum of all negative weight values in each weight group; weight the absolute value of the sum of all negative weight values in each weight group with the sum of all positive weight values to determine the priority of each weight group; based on the priority of each weight group, sort the plurality of weight groups.

8. The acceleration system of claim 6, wherein, The calculation unit is further configured to: the sum of the potential contribution values of all the remaining weight groups is positive; the predetermined early termination condition comprises: when the sum of the current partial sum and the sum of the potential contribution values of all the remaining weight groups is greater than or equal to a second threshold value, the convolution operation is terminated.

9. The acceleration system of claim 6, wherein, The calculation unit is further configured to: the sum of the potential contribution values of all the remaining weight groups is negative; the predetermined early termination condition comprises: when the sum of the current partial sum and the sum of the potential contribution values of all the remaining weight groups is greater than or equal to a second threshold value, the convolution operation is terminated. The calculation unit is further configured to: the sum of the potential contribution values of all the remaining weight groups is negative; the predetermined early termination condition comprises: when the sum of the current partial sum and the sum of the potential contribution values of all the remaining weight groups is greater than or equal to a second threshold value, the convolution operation is terminated.

Images