Sparsification target layer determination device, sparsification target layer determination method, and program

Abstract

Provided is a to-be-sparsified layer determination device that determines whether or not to apply sparsification to the weights of a neural network (NN) model in an implementation target (actual machine). This to-be-sparsified layer determination device is provided with: an individual layer sparsity speed contribution examination unit that receives input of a neural network model including a plurality of layers, each having weights, and input of one or a plurality of sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of each layer, and examines, for each layer, the execution time of the neural network model and the execution time of the one or plurality of sparse weight neural network models; and a to-be-sparsified layer determination unit that, on the basis of the examination result, determines whether or not to apply sparsification to the weights of each layer of the neural network model.

Description

[Technical field]

[0001] The present invention relates to an apparatus for determining a layer to be sparsified, a method for determining a layer to be sparsified, and a program. [Background technology]

[0002] In neural network (NN) models, the weights in each layer are generally dense, meaning that there are many non-zero values, and in many cases nearly 100% of the values ​​are non-zero. In contrast, it may be possible to speed up the execution of NN models with sparse weights, in which the weights have many zero values. Making the weights sparse results in some degradation in accuracy, but it is known that the proportion of zero values ​​in the weights can be increased by devising a learning method. A method has been proposed that takes advantage of this "sparseness" of the weights (i.e., the large number of zero values) to increase speed.

[0003] Patent Document 1 relates to a method for determining a processing unit (tile size) in executing an already-sparsed neural network model.

[0004] Patent Document 2 relates to a method for providing a sparse network model while minimizing compromise in model accuracy.

[0005] Patent Document 3 relates to a high-speed sparse optimization device.

[0006] Patent Document 4 relates to a method for quickly executing a sparsified neural network model. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2021-093131 [Patent Document 2] Patent Publication No. 2021-006980 [Patent Document 3] JP 2020-102073 A [Patent Document 4] Special Publication No. 2019-522850 Summary of the Invention [Problem to be solved by the invention]

[0008] The following analysis is provided by the present invention.

[0009] However, it may happen that the sparsity of weights cannot be fully utilized to speed up the execution speed. For example, even if the sparsity of weights is 90%, i.e., the ratio of non-zero weights is 10%, the execution speed is not necessarily 10 times faster than when sparsification is not performed. This is because the cases in which sparsity can be utilized are limited due to constraints such as parameters such as batch size (N), number of channels (C), height (H), and width (W), and the relationship with the parallelism of hardware calculations and memory access. In addition, the sparsity (the ratio of zero values) obtained for each layer may be different, such as 90% in some layers and 70% in others. On the other hand, the closer the sparsity is to 100%, the greater the effect of speeding up the execution speed in some cases. In addition, when the sparsity is below a certain level, there may be cases in which the execution speed is not accelerated at all.

[0010] The present invention aims to provide an apparatus for determining a layer to be sparsified, a method for determining a layer to be sparsified, and a program that contribute to determining whether or not to apply sparsification to the weights of a neural network (NN) model in an implementation target (actual device). [Means for solving the problem]

[0011] According to a first aspect of the present invention, a layer-by-layer sparsity speed contribution investigation unit receives as input a neural network model including a plurality of layers, each of which has a weight, and one or more sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of the layers, and investigates, for each of the layers, an execution time of the neural network model and an execution time of the one or more sparse weight neural network models; a sparsification target layer determination unit that determines, for each of the layers of the neural network model, whether or not to apply sparsification to the weights based on a result of the investigation.

[0012] According to a second aspect of the present invention, a method for calculating a weight distribution of a neural network model including a plurality of layers, each of which has a weight, and one or more sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of each of the layers are input, and a running time of the neural network model and the one or more sparse weight neural network models are investigated for each of the layers; and determining, for each layer of the neural network model, whether or not to apply sparsification to the weights based on the result of the search.

[0013] According to a third aspect of the present invention, a computer is provided with: A process of inputting a neural network model including a plurality of layers, each of which has a weight, and one or more sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of each of the layers, and investigating the execution time of the neural network model and the execution time of the one or more sparse weight neural network models for each of the layers; and determining, for each layer of the neural network model, whether or not to apply sparsification to the weights based on the results of the investigation. The program can be recorded in a computer-readable storage medium. The storage medium can be a non-transient medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present invention can also be embodied as a computer program product. Effect of the Invention

[0014] According to the present invention, it is possible to provide a sparsification target layer determination device, a sparsification target layer determination method, and a program that contribute to determining whether or not to apply sparsification to the weights of a neural network (NN) model in an implementation target (actual device). [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a sparsification target layer determination device according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a configuration of a sparsification target layer determination device according to a first exemplary embodiment of the present invention. [Diagram 3] FIG. 11 is a diagram illustrating an example of generating uniformly random sparse weights according to the first embodiment of the present invention. [Figure 4] FIG. 11 is a diagram illustrating an example of generating random sparse weights according to a specific pattern in the first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram showing an example of an outline of a sparsification application layer list output by the sparsification target layer determination device according to the first exemplary embodiment of the present invention. [Figure 6] FIG. 4 is a diagram illustrating an example of an improvement rate of the execution speed with respect to the sparseness and the execution speed with respect to the Dense ratio in the sparsification target layer determination device according to the first embodiment of the present invention. [Figure 7]FIG. 11 is a diagram showing another example of the outline of the sparsification application layer list output by the sparsification target layer determination device according to the first embodiment of the present invention. [Figure 8] FIG. 11 is a diagram illustrating an example of a configuration of a sparsification target layer determination device according to a second exemplary embodiment of the present invention. [Figure 9] FIG. 11 is a diagram illustrating an example of a configuration of a dense weight / sparse weight execution speed measurement result database according to the second exemplary embodiment of the present invention. [Figure 10] FIG. 11 is a flow diagram showing an example of an algorithm of an outline of the operation of an each-layer sparsity speed contribution investigating unit of the sparsification target layer determining device according to the second exemplary embodiment of the present invention. [Figure 11] FIG. 13 is a flow chart showing another example of an outline algorithm of the operation of the each-layer sparsity speed contribution investigating unit of the sparsification target layer determination device according to the modified example of the second embodiment of the present invention. [Figure 12] FIG. 2 is a diagram showing the configuration of a computer constituting the sparsification target layer determination device of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] First, an overview of one embodiment of the present invention will be described with reference to the drawings. Note that the reference numerals in the drawings attached to this overview are attached to each element for convenience as an example to aid in understanding, and are not intended to limit the present invention to the illustrated form. Furthermore, the connection lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. The unidirectional arrows are intended to diagrammatically indicate the flow of the main signal (data), and do not exclude bidirectionality.

[0017] FIG. 1 is a diagram showing an example of the configuration of a sparsification target layer determination device 100 according to an embodiment of the present invention. The sparsification process 10 shown in FIG. 1 shows a process of creating an input to the sparsification target layer determination device 100 according to an embodiment of the present invention in advance. The sparsification process 10 executes a process of applying weight sparsification 12 to each layer of weights (dense weights) of a neural network model 11 to create one or more sparse weight neural network models 13 having sparse weights. The sparsification target layer determination device 100 according to an embodiment of the present invention receives input of the neural network model 11 and one or more sparse weight neural network models 13 generated in advance by the sparsification process 10. Note that the sparsity of the weights means that there are many zero values ​​in the weights. It is assumed that the sparse weight neural network model 13 containing many zero values ​​in the weights is created by the weight sparsification. Note that when an implementation target (actual device) that executes a model containing zero values ​​in the weights has a mechanism for skipping the zero values, the calculation of the sparse weight neural network model is accelerated. Therefore, the speed at which the model runs depends on the actual machine.

[0018] 1, the neural network model 11 can be configured as a four-layer neural network including, for example, a conv1 layer, a conv2 layer, a conv3 layer, and a conv4 layer. The conv1 layer, the conv2 layer, the conv3 layer, and the conv4 layer may be, for example, a convolutional layer in the case of a convolutional neural network (CNN, Convolutional Neural Netork), etc. Each sparse weight neural network model 13 also includes the conv1 layer, the conv2 layer, the conv3 layer, and the conv4 layer, which are the same as the neural network model 11.

[0019] 1, a sparsification target layer determination device 100 according to an embodiment of the present invention includes an each-layer sparsity speed contribution investigation unit 110 and a sparsification target layer determination unit 120. The each-layer sparsity speed contribution investigation unit 110 inputs a neural network model 11 including a plurality of layers, each of which has a weight, and one or a plurality of sparse weight neural network models 13 having sparse weights obtained by applying sparsification 12 to the weights for each layer. Each layer may be, for example, a conv1 layer, a conv2 layer, a conv3 layer, or a conv4 layer.

[0020] As described above, the calculation speed of the sparsification weight neural network model is increased by a mechanism for skipping zero values ​​of weights held by the actual machine, and therefore, in order to evaluate whether the calculation speed is increased, at least the layer sparsity speed contribution investigation unit 110 of the sparsification target layer determination device 100 according to one embodiment of the present invention is configured and executed on the actual machine. Note that the entire sparsification target layer determination device 100 may be configured and executed on the implementation target (actual machine).

[0021] Here, calculations are performed for all weights (dense weights) in each of the conv1 layer, conv2 layer, conv3 layer, and conv4 layer of the neural network model 11. In contrast, each of the conv1 layer, conv2 layer, conv3 layer, and conv4 layer of one or more sparse weight neural network models 13 is a neural network model including weights that are zero values ​​sparsified by weight sparsification 12. Therefore, when an actual device that executes the calculation of this sparsified weight neural network model has a mechanism for skipping the zero values, the sparsified weight neural network model is calculated using the mechanism for skipping the zero values.

[0022] The each layer sparsity speed contribution investigating unit 110 further investigates the execution time of the neural network model 11 and the execution time of each of the one or more sparse weight neural network models 13 for each layer.

[0023] The sparsification target layer determination unit 120 determines whether or not to apply sparsification to weights for each layer of the neural network model 11, based on the results of the investigation by the layer sparsity speed contribution investigation unit 110. The sparsification target layer determination unit 120 also outputs a sparsification application layer list 130 indicating whether or not to apply the sparsification, as determined above.

[0024] The sparsification target layer determination device 100 according to one embodiment of the present invention can provide a sparsification target layer determination device that contributes to determining whether or not to apply sparsification to weights of a neural network (NN) model in an implementation target (actual device). Also, it is possible to output a sparsification application layer list 130 that displays the determined whether or not to apply. The sparsification application layer list 130 may display whether or not to apply weight sparsification for each of the conv1 layer, conv2 layer, conv3 layer, and conv4 layer, for example.

[0025] [First embodiment] Next, a sparsification target layer determination device according to a first embodiment of the present invention will be described with reference to the drawings. Fig. 2 is a diagram showing an example of the configuration of a sparsification target layer determination device 100 according to a first embodiment of the present invention. In Fig. 2, components with the same reference numerals as those in Fig. 1 are assumed to be the same components, and descriptions thereof will be omitted.

[0026] 2, a sparsification target layer determination device 100 according to a first embodiment of the present invention includes an each-layer sparsity speed contribution investigation unit 110 and a sparsification target layer determination unit 120. The each-layer sparsity speed contribution investigation unit 110 includes a dense weighting execution speed measurement unit 111, a sparse weighting execution speed measurement unit 112, and an execution speed comparison unit 113.

[0027] 2, the sparsification process 10 includes a process of executing weight sparsification 12. The weight sparsification 12 may use a method of searching for and sparsifying weights that can be set to zero while minimizing deterioration in the accuracy of calculations performed by each layer, for example, the conv1 layer, the conv2 layer, the conv3 layer, and the conv4 layer of a neural network (NN) model that has undergone normal learning.

[0028] In addition, in the sparsification process 10, weight sparsification 12 may be applied to the weights of each layer of the NN model that has been subjected to normal learning, or to a predetermined position of the weights of each layer without determining the weights of each layer, to generate one or more sparse weight neural network models 13 having sparse weights with the weights set to zero.

[0029] Furthermore, the sparsification process 10 may generate one or more sparse weight neural network models 13 for each layer, including sparse weights that have been sparsified to different degrees, e.g., by randomly setting X% of the weights to zero.

[0030] Fig. 3 is a diagram showing an example of the process of weight sparsification 12, and shows an example of generating uniformly random sparse weights. Fig. 3 shows an example of sparsification in which weights in a matrix 300 indicating weights are set to zero values ​​at random positions 301 to 306 at a rate of X%.

[0031] Fig. 4 is a diagram showing another example of the process of weight sparsification 12, and is a diagram showing an example of generating X% random sparse weights according to a specific pattern. Fig. 4 shows an example of a case where the weights in a matrix 400 indicating the weights are sparsified by setting the weights to zero values ​​at a rate of X% in specific patterns 401 to 404 and specific patterns 405 to 408. Note that the examples shown in Figs. 3 and 4 are merely examples, and do not exclude sparsification other than uniform randomness or randomness according to a specific pattern, and the uniform randomness or specific pattern is not limited to being arranged as described above.

[0032] The sparsification target layer determination device 100 of one embodiment of the present invention receives input of a neural network model 11 and one or more sparse weight neural network models 13 generated in advance by the sparsification process 10. In addition, the calculation speed of the sparsification weight neural network model is increased by a mechanism for skipping zero values ​​of weights held by the actual machine, etc., so that in order to evaluate whether the calculation speed is increased, at least the dense weight execution speed measurement unit 111 and the sparse weight execution speed measurement unit 112 of the each layer sparsity speed contribution investigation unit 110 of one embodiment of the present invention are configured and executed on the implementation target (actual machine). Note that the each layer sparsity speed contribution investigation unit 110 or the entire sparsification target layer determination device 100 may be configured and executed on the implementation target (actual machine).

[0033] The operation of the sparsification target layer determination device 100 according to an embodiment of the present invention will be described below with reference to FIG.

[0034] Referring to FIG. 2, the dense weight execution speed measurement unit 111 of the each layer sparsity speed contribution investigation unit 110 performs calculations for all weights (dense weights) in each layer of the neural network model 11, i.e., the conv1 layer, the conv2 layer, the conv3 layer, and the conv4 layer, and measures the execution time of the calculations for each layer.

[0035] On the other hand, each of the conv1 layer, conv2 layer, conv3 layer, and conv4 layer of one or more sparse weight neural network models 13 is a neural network model configured to have weights made zero-valued by weight sparsification 12, so when the actual machine that executes the calculation of this sparse weight neural network model has a mechanism for skipping zero values, the sparse weight execution speed measurement unit 112 executes the calculation using the mechanism for skipping zero values. In other words, since the speed at which the sparse weight neural network model is executed depends on the actual machine, the sparse weight execution speed measurement unit 112 executes the calculation of the sparse weight neural network model on the actual machine using the mechanism for skipping zero values, and measures the execution time of the calculation of each of the one or more sparse weight neural network models 13 for each layer.

[0036] The execution speed comparison unit 113 compares, for each layer, the measured execution time of the calculations made by the dense weight execution speed measurement unit 111 with the measured execution time of the calculations made by the sparse weight execution speed measurement unit 112, and based on the results of the comparison, investigates the improvement rate of the calculation execution speed of the sparse weight execution speed measurement unit 112 for each layer.

[0037] The sparsification target layer determination unit 120 determines that sparsification is to be applied to the weights of a layer in the neural network model 11 for which the reduction in execution time is equal to or greater than a predetermined value.

[0038] An example of a method for determining whether or not to apply sparsification will be described below, but the method is not limited to the method described below.

[0039] [Example of the first decision method] FIG. 5 is a diagram showing an example of a schematic of a sparsification application layer list output by the sparsification target layer determination device of the first embodiment of the present invention. FIG. 5 shows an example of a sparsification application layer list 130 that displays whether or not sparsification is applied by the first determination method. Note that FIG. 5 shows an example of the sparsification application layer list 130 when only one sparse weight neural network model 13 is input. Referring to FIG. 5, the sparsification application layer list 130 includes columns of, for example, model structure 501, number of channels 502, sparsity degree (proportion of zero values) 503, execution time in dense mode 504, execution time in sparse mode 505, improvement rate of execution speed of dense ratio 506, and sparsification application 507. Also, rows 510 to 540 correspond to the conv1 layer to the conv4 layer shown in FIG. 1, respectively.

[0040] 5, the improvement rate 506 of the execution speed of the Dense ratio of the conv1 layer is 0.7 times (0.7×), the improvement rate 506 of the execution speed of the Dense ratio of the conv2 layer is 1.0 times (1.0×), the improvement rate 506 of the execution speed of the Dense ratio of the conv3 layer is 1.4 times (1.4×), and the improvement rate 506 of the execution speed of the Dense ratio of the conv4 layer is 2.1 times (2.1×). For example, as a method of determining whether to apply sparsification, if it is determined that sparsification is to be applied when the improvement rate 506 of the execution speed of the Dense ratio is 1.4 times or more, in sparsification application 507, it is determined that sparsification is to be applied to the conv3 layer and the conv4 layer of the neural network model 11, and it is determined that sparsification is not to be applied to the conv1 layer and the conv2 layer of the neural network model 11, and application / non-application is displayed, respectively.

[0041] [Example of the second decision method] Furthermore, when multiple sparse weight neural network models 13 are input, the dense weight execution speed measurement unit 111 of the each layer sparsity speed contribution investigation unit 110 measures the execution time of the neural network model 11 for each layer. Meanwhile, the sparse weight execution speed measurement unit 112 measures the execution time of each of the multiple sparse weight neural network models for each layer. The execution speed comparison unit 113 compares the execution time of the neural network model 11 with the execution time of each of the multiple sparse weight neural network models for each layer, and investigates the improvement rate of each execution speed for each layer based on the comparison result.

[0042] The sparsification target layer determination unit 120 can also determine to apply sparsification to the weights of a layer of the neural network model 11 corresponding to a layer in which any one of the improvement rates of the execution speed of a plurality of sparse weight neural network models is equal to or greater than a predetermined value.

[0043] [Example of the third decision method] In addition, when a plurality of sparse weight neural network models 13 are input, it is also possible to determine which layers are not to be sparsified as follows.

[0044] 6 is a diagram showing an example of the execution speed versus sparseness and the improvement rate of the execution speed versus the Dense ratio for the conv1 layer of the neural network model 11. A sparseness of 0% indicates a Dense case, that is, 10 msec (milliseconds) of the execution time 603 indicates the execution speed of the conv1 layer of the neural network model 11.

[0045] On the other hand, examples of sparsity of 70%, 80%, and 90% show the execution times 603 when the conv1 layer is executed at each sparsity by a plurality of sparse weight neural network models 13 including sparse weights to which sparsification with different sparsity degrees is applied. In the example shown in FIG. 6, when the sparsity is 70%, the execution time of the conv1 layer is 13 msec, and the improvement rate of the execution speed of the dense ratio is 0.7 times (0.7×). When the sparsity is 80%, the execution time of the conv1 layer is 12 msec, and the improvement rate of the execution speed of the dense ratio is 0.8 times (0.8×). When the sparsity is 90%, the execution time of the conv1 layer is 11 msec, and the improvement rate of the execution speed of the dense ratio is 0.9 times (0.9×).

[0046] As shown in the example of FIG. 6, in any sparsity level, if the execution speed is not increased compared to the Dense level, it can be determined not to apply sparsification to the weights of that layer of the neural network model 11.

[0047] [Example of the fourth decision method] When a target execution time for the entire neural network model 11 has been determined, it is possible to adopt a criterion for determining whether or not sparsification should be applied, such that sparsification is applied only to (at least) the minimum number of layers that can achieve the target execution time.

[0048] Fig. 7 is a diagram showing another example of the outline of the sparsification application layer list 130 output by the sparsification target layer determination device 100 according to the first embodiment of the present invention. Fig. 7 shows an example of the sparsification application layer list 130 that displays whether or not sparsification is to be applied, according to the fourth determination method. In Fig. 7, components with the same reference numerals as those in Fig. 5 are considered to be the same components, and description thereof will be omitted.

[0049] For example, assume that in order to meet the target execution time, speed-up by reducing the execution time of the neural network model 11 as a whole is required. Note that FIG. 7 shows an example of the sparsification application layer list 130 when only one sparse weight neural network model 13 is input, similar to FIG. 5. Referring to FIG. 7, the reduction in the sparse execution time relative to the dense execution time of the conv4 layer is 52 msec, and the reduction in execution time exceeds 50 msec by sparsification of only the conv4 layer. Therefore, if sparsification is applied only to the conv4 layer, a reduction in the execution time of the neural network model 11 as a whole can be achieved by 50 msec or more. In such a case, in sparsification application 507, it is displayed that sparsification is applied to the conv4 layer of the neural network model 11, and it is displayed that sparsification is not applied to the conv1 layer, the conv2 layer, and the conv3 layer of the neural network model 11.

[0050] In this way, even if sparsifying other layers is effective in increasing the execution speed of the neural network model 11, by not making the layers sparsifying more than necessary, it is possible to reduce the possibility of deterioration in calculation accuracy.

[0051] [Second embodiment] Next, a sparsification target layer determination device 200 according to a second embodiment of the present invention will be described with reference to the drawings. Fig. 8 is a diagram showing an example of the configuration of the sparsification target layer determination device 200 according to the second embodiment of the present invention. In Fig. 8, components with the same reference numerals as those in Fig. 2 are assumed to be the same components, and descriptions thereof will be omitted.

[0052] The sparsification target layer determination device 200 according to the second embodiment of the present invention is configured and executed on a mounting target (actual device).

[0053] 8, the sparsification target layer determination device 200 of the second embodiment of the present invention includes an each layer sparsity speed contribution investigation unit 110 and a sparsification target layer determination unit 120. The each layer sparsity speed contribution investigation unit 110 includes a dense weight execution speed measurement unit 111, a sparse weight execution speed measurement unit 112, an execution speed comparison unit 113, a parameter investigation unit 210, and a dense weight / sparse weight execution speed measurement result database (DB) 220. The dense weight / sparse weight execution speed measurement result database (DB) 220 may be disposed outside the sparsification target layer determination device 200.

[0054] 9 is a diagram showing an example of the configuration of the dense weight / sparse weight execution speed measurement result database 220 according to the second embodiment of the present invention. The dense weight / sparse weight execution speed measurement result database 220 is a database that stores an execution time 909 in sparse mode and an execution speed improvement rate 910 using a device 901, a layer type 902, a batch size (N) 903, a number of input channels (Cin) 904, a number of output channels (Cout) 905, a height (H) 906, a width (W) 907, and a sparsity degree 908 as input parameters. The device 901 is a parameter corresponding to an implementation target (actual device).

[0055] 9, row 921 shows the case of sparsity 0.0, i.e., dense without sparsification. This corresponds to the case of neural network (NN) model 11 in FIG. 8. In contrast, row 922 shows the case of sparsity 0.1, row 923 shows the case of sparsity 0.2, and row 924 shows the case of sparsity 0.9. These correspond to the case of sparse weight neural network (NN) model 13 in FIG. 8. The other parameters other than the sparsity are the same as those in row 921.

[0056] Also, lines 925 to 928 store the execution time 909 in the sparse state and the execution speed improvement rate 910 for parameters different from those in lines 921 to 924. Line 925 shows the case of a sparsity of 0.0, that is, dense without sparsification. With reference to FIG. 8, this corresponds to the case of the neural network (NN) model 11. On the other hand, line 926 shows the case of a sparsity of 0.1, line 927 shows the case of a sparsity of 0.2, and line 928 shows the case of a sparsity of 0.9. With reference to FIG. 8, these correspond to the case of the sparse weight neural network (NN) model 13. The parameters other than the sparsity are the same as those in line 925.

[0057] Next, an example of an outline of the operation of the sparsification target layer determination device 200 according to the second embodiment of the present invention will be described with reference to the drawings.

[0058] Fig. 10 is a flow diagram showing an example of an outline algorithm of the operation of the parameter investigation unit 210 of the each-layer sparsity speed contribution investigation unit 110 of the sparsification target layer determination device 200 according to the second embodiment of the present invention. The algorithm shown in Fig. 10 shows an example of the operation when each of the neural network model 11 and one or more sparse weight neural network models 13 can be executed for each layer.

[0059] 10 starts in step S1001, and in step S1002, the parameter research unit 210 refers to the dense weight / sparse weight execution speed measurement result database 220 for each layer of one or more sparse weight neural network (NN) models 13. Specifically, based on the parameters exemplified in FIG. 9, the parameter research unit 210 searches the dense weight / sparse weight execution speed measurement result database 220 for whether or not there is a record corresponding to, for example, the conv1 layer of one or more sparse weight neural network models 13.

[0060] In step S1003, if the parameter investigation unit 210 determines that a record corresponding to the conv1 layer exists in the database 220 (Y), the process proceeds to step S1004, and instructs the execution speed comparison unit 113 to apply the execution speed improvement rate stored in the database 220 to the conv1 layer.

[0061] In step S1003, if the parameter investigation unit 210 determines that there is no record corresponding to the conv1 layer (N), the process proceeds to step S1005, where the parameter investigation unit 210 instructs the dense weighting execution speed measurement unit 111, the sparse weighting execution speed measurement unit 112, and the execution speed comparison unit 113 to execute the conv1 layer of the neural network model 11 and the sparse weighting neural network model 13 to evaluate (investigate) the speed improvement rate.

[0062] Next, in step S1006, the execution speed comparison unit 113 registers the speed improvement rate of the conv1 layer in the database 220 together with the parameters.

[0063] Next, in step S1007, the parameter research unit 210 judges whether evaluation (research) of all layers is completed. If evaluation (research) of all layers is completed, that is, if evaluation (research) of the conv1 layer to the conv4 layer of one or more sparse weight neural network models 13 in FIG. 8 is completed, the algorithm ends in step S1008.

[0064] On the other hand, in step S1007, if the evaluation (investigation) of all layers is not completed, that is, if the evaluation of the conv1 layer to the conv4 layer of one or more sparse weight neural network models 13 in FIG. 8 is not completed, the process returns to step S1002, and the parameter investigation unit 210 repeats the above steps for the remaining layers (conv2 layer to conv4 layer).

[0065] According to the sparsification target layer determination device 200 of the second embodiment of the present invention, the dense weight / sparse weight execution speed measurement result database 220 can be used to speed up the calculation of the improvement rate of the execution speed for each layer.

[0066] [Modification of the second embodiment] Next, an example of the outline of the operation of the sparsification target layer determination device 200 according to the modified example of the second embodiment of the present invention will be described with reference to the drawings. Note that in the modified example of the second embodiment, the outline of the configuration of the sparsification target layer determination device 200 is the same as the configuration of the sparsification target layer determination device 200 according to the second embodiment, and therefore a description thereof will be omitted.

[0067] Fig. 11 is a flow diagram showing an example of an outline algorithm of the operation of the parameter investigation unit 210 of the each-layer sparsity speed contribution investigation unit 110 of the sparsification target layer determination device 200 in the modified example of the second embodiment of the present invention. The algorithm shown in Fig. 11 shows an example of the operation when the neural network model 11 and one or more sparse weight neural network models 13 cannot be executed for each layer, that is, when the neural network model 11 and the sparse weight neural network model 13 can only be executed as a whole.

[0068] 11 starts in step S1101, and in step S1102, the parameter research unit 210 refers to the dense weight / sparse weight execution speed measurement result database 220 for each layer of one or more sparse weight neural network (NN) models 13. Specifically, based on the parameters exemplified in FIG. 9, the parameter research unit 210 searches the dense weight / sparse weight execution speed measurement result database 220 for records corresponding to, for example, the conv1 layer to the conv4 layer of one or more sparse weight neural network models 13.

[0069] In step S1103, if the parameter investigation unit 210 determines that all records corresponding to all layers, for example, the conv1 layer to the conv4 layer, exist (N), the algorithm proceeds to step S1104 and instructs the execution speed comparison unit 113 to apply the improvement rate of the execution speed stored in the database 220 to each layer, and the algorithm ends in step S1107.

[0070] In step S1103, if the parameter investigation unit 210 determines that there is no record corresponding to at least one layer, for example, at least one layer among the conv1 layer to the conv4 layer (Y), the process proceeds to step S1105, where the parameter investigation unit 210 instructs the dense weight execution speed measurement unit 111, the sparse weight execution speed measurement unit 112, and the execution speed comparison unit 113 to execute all layers, for example, the conv1 layer to the conv4 layer, of the neural network model 11 and one or more sparse weight neural network models 13, and evaluate (investigate) the improvement rate of the execution speed.

[0071] Next, in step S1106, the execution speed comparison unit 113 registers the improvement rate of the execution speed of all the layers evaluated (investigated), for example, the conv1 layer to the conv4 layer, in the database 220 together with the parameters.

[0072] The algorithm then ends in step S1107.

[0073] According to the modified example of the second embodiment of the present invention, even when the neural network model 11 and the one or more sparse weight neural network models 13 cannot be executed layer by layer, i.e., even when the neural network model 11 and the sparse weight neural network model 13 can only be executed as a whole, it can contribute to speeding up the calculation of the improvement rate of the execution speed for each layer.

[0074] Although each embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiments, and further modifications, substitutions, and adjustments can be made without departing from the basic technical idea of ​​the present invention. For example, the system configuration, the configuration of each element, and the expression form of messages shown in each drawing are examples to help understand the present invention, and the present invention is not limited to the configurations shown in these drawings. In the following description, "A and / or B" is used to mean at least either A or B.

[0075] Furthermore, the procedures shown in the modified examples of the first and second embodiments described above can be realized by a program that causes a computer (9000 in FIG. 12) functioning as the sparseness target layer determination device 100, 200 to realize the functions of the sparseness target layer determination device 100, 200. Such a computer is exemplified by a configuration including a CPU (Central Processing Unit) 9010, a communication interface 9020, a memory 9030, and an auxiliary storage device 9040 in FIG. 12. That is, the CPU 9010 in FIG. 12 may execute a sparseness target layer determination program and perform an update process for each calculation parameter held in the auxiliary storage device 9040 or the like.

[0076] The memory 9030 is a RAM (Random Access Memory), a ROM (Read Only Memory), or the like.

[0077] In other words, each unit (processing means, functions) of the sparsification target layer determination device shown in the modified examples of the first and second embodiments described above can be realized by a computer program that causes the processor of the computer to execute each of the above-mentioned processes using its hardware.

[0078] Finally, preferred embodiments of the present invention will be summarized. [First form] (See the above-mentioned device for determining layers to be sparsified according to the first aspect.) [Second Form] In the sparsification target layer determination device according to the first aspect, the each-layer sparsity speed contribution investigation unit compares, for each layer, an execution time of the neural network model with an execution time of each of the one or more sparse weight neural network models, and investigates, for each layer, an improvement rate of the execution speed of each of the one or more sparse weight neural network models based on a result of the comparison; It is preferable that the sparsification target layer determination unit determines to apply the sparsification to the weights of a layer of the neural network model in which any of the improvement rates of execution speed is equal to or greater than a predetermined value. [Third Form] In the sparsification target layer determination device described in the second aspect, it is desirable that the sparsification target layer determination unit determines not to apply the sparsification to the weights of a layer of the neural network model in which any of the improvement rates of the execution speed of the layer is smaller than a predetermined value. [Fourth Form] In the sparsification target layer determination device described in the second aspect, it is desirable that the sparsification target layer determination unit determines, for each of the layers of the neural network model, whether or not to apply the sparsification to the weights of the layer so that a total execution time of each layer of the neural network model is reduced to a predetermined value or less. [Fifth Form] In the sparsification target layer determination device according to any one of the second to fourth aspects, the each layer sparsity speed contribution investigation unit further includes an execution speed measurement result database storing an improvement rate of the execution speed of the sparse weight neural network model; When an improvement rate of the execution speed of a layer having the same parameters as the target layer of the sparse weight neural network model exists in the execution speed measurement result database, the improvement rate of the execution speed of the target layer is obtained from the execution speed measurement result database; If the improvement rate of the execution speed of a layer having the same parameters as the target layer of the sparse weight neural network model is not present in the execution speed measurement result database, it is desirable to compare the execution time of the target layer of the neural network model with the execution time of the target layer of the sparse weight neural network model, investigate the improvement rate of the execution speed of the target layer, and store the parameters of the sparse weight neural network model and the improvement rate of the execution speed in the execution speed measurement result database. [Sixth Form] In the sparsification target layer determination device according to any one of the second to fourth aspects, the each layer sparsity speed contribution investigation unit further includes an execution speed measurement result database storing an improvement rate of the execution speed of the sparse weight neural network model; When the improvement rate of the execution speed of layers having the same parameters for all layers of the sparse weight neural network model exists in the execution speed measurement result database, the improvement rate of the execution speed of all layers of the sparse weight neural network model is obtained from the execution speed measurement result database; When the improvement rate of the execution speed of a layer having the same parameters for at least one layer of the sparse weight neural network model does not exist in the execution speed measurement result database, it is desirable to compare the execution time of each layer of the neural network model with the execution time of each layer of the sparse weight neural network model for all layers of the sparse weight neural network model, investigate the improvement rate of the execution speed for each layer, and store the parameters of the sparse weight neural network model and the improvement rate of the execution speed in the execution speed measurement result database. [7th form] (See the second perspective for determining which layers to sparsify.) [8th form] In a seventh aspect of the sparsification target layer determination method, the investigating step includes a step of comparing, for each of the layers, an execution time of the neural network model with an execution time of each of the one or more sparse weight neural network models, and investigating, for each of the layers, an improvement rate of the execution speed of each of the one or more sparse weight neural network models based on a result of the comparison; It is desirable that the determining step includes a step of determining to apply the sparsification to the weights of a layer of the neural network model in which any of the improvement rates of execution speed is equal to or greater than a predetermined value. [9th form] (See the third perspective program above) [10th form] In a ninth aspect of the program, the investigating process includes a process of comparing, for each of the layers, an execution time of the neural network model with an execution time of each of the one or more sparse weight neural network models, and investigating, for each of the layers, an improvement rate of the execution speed of each of the one or more sparse weight neural network models based on a result of the comparison; It is desirable to include a process of determining that the sparsification is to be applied to the weights of a layer of the neural network model for which either the determining process or the rate of improvement in execution speed is equal to or greater than a predetermined value. The seventh and ninth embodiments can be expanded into the third to sixth embodiments in the same manner as the first embodiment.

[0079] The disclosures of the above patent documents are incorporated herein by reference. Within the framework of the entire disclosure of the present invention (including the scope of claims), modifications and adjustments of the embodiments and examples are possible based on the basic technical ideas. Furthermore, within the framework of the disclosure of the present invention, various combinations and selections of the various disclosed elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art would be able to make in accordance with the entire disclosure including the scope of claims and the technical ideas. In particular, with regard to the numerical ranges described in this specification, any numerical value or subrange included in the range should be interpreted as being specifically described even if not otherwise specified. [Explanation of symbols]

[0080] 10 Sparsification process 11 Neural Network (NN) Model 12 Weight Sparsification 13 Sparse weight neural network (NN) model 100, 200 Sparsification target layer determination device 110 Each Layer Sparsity Speed ​​Contribution Investigation Section 111 Dense Weight Execution Speed ​​Measurement Unit 112 Sparse Weight Execution Speed ​​Measurement Unit 113 Execution Speed ​​Comparison Section 120 Sparsification target layer decision unit 130 Sparsification Application Layer List 210 Parameter Research Department 220 Dense weight / Sparse weight execution speed measurement result database (DB) 9000 Computers 9010 CPU 9020 Communication Interface 9030 Memory 9040 Auxiliary storage device

Claims

1. a layer-by-layer sparsity speed contribution investigation unit that receives as input a neural network model including a plurality of layers, each layer having a weight, and one or more sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of the layers, and investigates, for each layer, the execution time of the neural network model and the execution time of the one or more sparse weight neural network models; and a sparsification target layer determination unit that determines, for each of the layers of the neural network model, whether or not to apply sparsification to the weights based on a result of the investigation.

2. The each layer sparsity speed contribution investigation unit compares, for each layer, an execution time of the neural network model with an execution time of each of the one or more sparse weight neural network models, and investigates, for each layer, an improvement rate of the execution speed of each of the one or more sparse weight neural network models based on a result of the comparison; 2. The sparsification target layer determination device according to claim 1, wherein the sparsification target layer determination unit determines to apply the sparsification to the weights of a layer of the neural network model in which any of the improvement rates of execution speed is equal to or greater than a predetermined value.

3. 3. The sparsification target layer determination device according to claim 2, wherein the sparsification target layer determination unit determines not to apply the sparsification to the weights of a layer of the neural network model in which any of the improvement rates of the execution speed of the layer is smaller than a predetermined value.

4. 3. The sparsification target layer determination device according to claim 2, wherein the sparsification target layer determination unit determines, for each of the layers of the neural network model, whether or not to apply the sparsification to the weights of the layer so that a sum of execution times of the layers of the neural network model is reduced to a predetermined value or less.

5. The each layer sparsity speed contribution investigation unit further includes an execution speed measurement result database storing an improvement rate of the execution speed of the sparse weight neural network model; When an improvement rate of the execution speed of a layer having the same parameters as the target layer of the sparse weight neural network model exists in the execution speed measurement result database, the improvement rate of the execution speed of the target layer is obtained from the execution speed measurement result database; 5. The sparseness target layer determination device according to claim 2, wherein, when an improvement rate of an execution speed of a layer having the same parameters as the target layer of the sparse weight neural network model is not present in the execution speed measurement result database, the device compares an execution time of the target layer of the neural network model with an execution time of the target layer of the sparse weight neural network model to investigate an improvement rate of the execution speed of the target layer, and stores the parameters of the sparse weight neural network model and the improvement rate of the execution speed in the execution speed measurement result database.

6. The each layer sparsity speed contribution investigation unit further includes an execution speed measurement result database storing an improvement rate of the execution speed of the sparse weight neural network model; When the improvement rate of the execution speed of layers having the same parameters for all layers of the sparse weight neural network model exists in the execution speed measurement result database, the improvement rate of the execution speed of all layers of the sparse weight neural network model is obtained from the execution speed measurement result database; 5. The device for determining a layer to be sparsed according to claim 2, wherein, when an improvement rate of an execution speed of a layer having the same parameters for at least one layer of the sparse weight neural network model is not present in the execution speed measurement result database, the device compares an execution time for each layer of the neural network model with an execution time for each layer of the sparse weight neural network model for all layers of the sparse weight neural network model, investigates an improvement rate of the execution speed for each layer, and stores the parameters of the sparse weight neural network model and the improvement rate of the execution speed in the execution speed measurement result database.

7. Executed by a computer having a processor and a storage device, A step of inputting a neural network model including a plurality of layers, each of which has a weight, and one or more sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of each of the layers, and investigating the execution time of the neural network model and the execution time of the one or more sparse weight neural network models for each of the layers; and determining, for each layer of the neural network model, whether or not to apply sparsification to the weights based on a result of the examination.

8. The step of investigating includes a step of comparing, for each of the layers, an execution time of the neural network model with an execution time of each of the one or more sparse weight neural network models, and investigating, for each of the layers, an improvement rate of the execution speed of each of the one or more sparse weight neural network models based on a result of the comparison; 8. The method for determining a layer to be sparsified according to claim 7, wherein the determining step includes a step of determining that the sparsification is to be applied to the weights of a layer of the neural network model in which any of the improvement rates of execution speed is equal to or greater than a predetermined value.

9. On the computer, A process of inputting a neural network model including a plurality of layers, each of which has a weight, and one or more sparse weight neural network models having sparse weights obtained by applying sparsification to the weights of each of the layers, and investigating the execution time of the neural network model and the execution time of the one or more sparse weight neural network models for each of the layers; and determining, for each of the layers of the neural network model, whether or not to apply sparsification to the weights based on a result of the investigation.

10. The investigating process includes a process of comparing, for each of the layers, an execution time of the neural network model with an execution time of each of the one or more sparse weight neural network models, and investigating, for each of the layers, an improvement rate of the execution speed of each of the one or more sparse weight neural network models based on a result of the comparison; 10. The program according to claim 9, wherein the determining process includes a process of determining to apply the sparsification to the weights of a layer of the neural network model in which any of the improvement rates of execution speed is equal to or greater than a predetermined value.

Images