Information processing device, method, and program

Abstract

Lighten the size of the neural network. [Solution] The information processing device 1X mainly comprises an acquisition means 31X, a pair generation means 32X, a selection means 33X, and a model generation means 34X. The acquisition means 31X acquires the input transformation matrix and the output transformation matrix obtained by singular value decomposition of each convolutional layer included in the neural network. The pair generation means 32X generates pairs of the column vector of each output transformation matrix of a convolutional layer and the row vector of the corresponding input transformation matrix of the next convolutional layer. The selection means 33X selects pairs to be pruned from the pairs based on the impact on the error of the neural network when the pairs are deleted. The model generation means 34X generates a lightweight model of the neural network by pruning the pairs to be pruned from the neural network.

Description

[Technical Field]

[0001] This disclosure relates to the technical field of information processing devices, methods, and programs for information processing devices related to training data of machine learning models. [Background technology]

[0002] There are pruning techniques that reduce the processing load at each convolutional layer of a convolutional neural network. For example, Patent Document 1 discloses a technique for decomposing each convolutional layer using singular value decomposition in image processing using a trained neural network. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-039483 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] In addition to dimensionality reduction of the convolutional filters in each convolutional layer of a neural network, it is desirable to simultaneously achieve a reduction in the number of channels between convolutional layers. On the other hand, if one attempts to achieve both, simply executing the methods for achieving each simultaneously will result in separate optimizations for each method, making it difficult to optimize the overall network.

[0005] One of the purposes of this disclosure is to provide an information processing device, method, and program that can reduce the size of neural networks, in view of the above-mentioned problems. [Means for solving the problem]

[0006] One aspect of an information processing device is: An acquisition means for obtaining the input transformation matrix and output transformation matrix obtained by singular value decomposition of each convolutional layer included in a neural network, Pair generation means for generating pairs of the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer of the said convolutional layer, A selection means for selecting a pair to be pruned from the pair, based on the impact on the error of the neural network when the pair is deleted, A model generation means for generating a lightweight model of the neural network obtained by pruning the pair of branches to be pruned from the neural network, It is an information processing device.

[0007] One aspect of the method is: Computers The input and output transformation matrices obtained by singular value decomposition of each convolutional layer in the neural network are acquired. A pair is generated between the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer. Based on the impact on the error of the neural network when the aforementioned pair is deleted, select a pair from the aforementioned pair to be pruned. This method generates a lightweight model of the neural network by pruning the aforementioned pair of branches from the neural network.

[0008] One aspect of the program is: The input and output transformation matrices obtained by singular value decomposition of each convolutional layer in the neural network are acquired. A pair is generated between the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer. Based on the impact on the error of the neural network when the aforementioned pair is deleted, select a pair from the aforementioned pair to be pruned. This program causes a computer to perform a process of generating a lightweight model of the neural network by pruning the aforementioned pair of branches from the neural network. [Effects of the Invention]

[0009] One example of the effects of this disclosure is that it can reduce the size of neural networks. [Brief explanation of the drawing]

[0010] [Figure 1] This shows the schematic configuration of a lightweight model generation system. [Figure 2] This shows the hardware configuration of the information processing device. [Figure 3] This is a functional block diagram of an information processing device. [Figure 4] This diagram shows the processing flow before and after singular value decomposition in the L-th and L+1-th layer convolutional layers, respectively, arranged vertically. [Figure 5] The processing flow of the convolutional layers in the Lth and L+1th layers after singular value decomposition, and the processing flow of the convolutional layers in the Lth and L+1th layers after pruning by the pruning selection unit 33, are shown side by side. [Figure 6] This is an example of a flowchart illustrating the overview of the processes performed by an information processing device. [Figure 7] This is a block diagram of an information processing device. [Figure 8] This is an example of a flowchart executed by an information processing device. [Modes for carrying out the invention]

[0011] The following describes embodiments of the information processing device, method, and program with reference to the drawings.

[0012] <First Embodiment> (1) System Configuration Figure 1 shows the schematic configuration of the lightweight model generation system 100. The lightweight model generation system 100 generates a lightweight model by pruning a machine learning model that has a convolutional neural network architecture. The lightweight model generation system 100 mainly consists of an information processing device 1 and a storage device 2 that stores pre-lightweight model information D1 and dataset D2. Hereafter, the machine learning model of the convolutional neural network that is the target of lightweighting will be called the "pre-lightweight model," and the lightweight model generated by the lightweight model generation system 100, which is a lightweight version of the pre-lightweight model, will be called the "lightweight model."

[0013] The information processing device 1 performs a lightweighting of the pre-lightening model, which is constructed by referring to the pre-lightening model information D1, and outputs information about the lightweight model generated by the lightweighting. In the lightweighting of the pre-lightening model, as described later, the information processing device 1 simultaneously achieves dimensionality reduction of the convolutional filters in each layer and reduction of the number of channels between layers. Then, the information processing device 1 performs fine-tuning of the parameters of the lightweight model by further machine learning the pre-lightening model, to which dimensionality reduction of the convolutional filters and reduction of the number of channels between layers have been applied, using the dataset D2. Then, the information processing device 1 considers the parameters of the model that has undergone the fine-tuning described above as the machine-learned parameters of the lightweight model, and outputs model information representing the architecture of the lightweight model and the machine-learned parameters. In this case, the information processing device 1 may store the model information of the lightweight model in the memory of the information processing device 1 or in the storage device 2, or it may transmit it to a device other than the storage device 2.

[0014] The storage device 2 is a memory that stores information necessary for processing by the information processing device 1. The storage device 2 may be an external storage device such as a hard disk connected to or built into the information processing device 1, a storage medium such as flash memory, or a server device that communicates data with the information processing device 1. Furthermore, the storage device 2 may be composed of multiple storage devices, with each of the above-mentioned storage units distributed among them. The storage device 2 contains pre-lightweight model information D1 and a dataset D2.

[0015] The pre-lightened model information D1 is the information necessary to construct the pre-lightened model that has undergone machine learning, and includes the parameters of the pre-lightened model obtained through machine learning. For example, the pre-lightened model information D1 includes various parameters (including hyperparameters) such as the layer structure, the neuron structure of each layer, the number and size of filters in each layer, and the weights of each element of each filter. The pre-lightened model is a model that outputs data in a predetermined format when data in a predetermined format (specifically, tensor-format data such as images) is input. For example, the pre-lightened model may be a classification model or a regression model. The data output by the pre-lightened model may be the confidence level for each candidate class to be classified, or it may be a confidence map (heatmap).

[0016] Dataset D2 is a machine learning dataset for an inference model that has the same input and output formats as the pre-lightweight model. Dataset D2 includes, for example, a set of pairs of input data to be input to the inference model and ground truth data representing the correct output that the inference model should produce when the input data is input to the inference model. Dataset D2 also includes test data to calculate the effect of errors when pruning is performed and training data used for fine-tuning the lightweight model. Note that the sets of input data and ground truth data pairs may be prepared separately for the test data and training data, or a common set of input data and ground truth data pairs may be prepared.

[0017] The configuration of the lightweight model generation system 100 shown in Figure 1 is an example, and various modifications may be made. For example, the information processing device 1 and the storage device 2 may be implemented by the same device. In another example, the information processing device 1 may be composed of multiple devices. In this case, the multiple devices constituting the information processing device 1 exchange information necessary to perform pre-assigned processing between the devices by direct communication via wired or wireless connections, or by communication via a network.

[0018] (2) Hardware configuration Figure 2 shows an example of the hardware configuration of the information processing device 1. The information processing device 1 includes a processor 11, memory 12, and interface 13 as hardware components. The processor 11, memory 12, and interface 13 are connected via a data bus 19.

[0019] The processor 11 functions as a controller (arithmetic unit) that controls the entire information processing device 1 by executing programs stored in memory 12. The processor 11 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may consist of multiple processors. The processor 11 is an example of a computer.

[0020] Memory 12 is composed of various volatile and non-volatile memories, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The program executed by the information processing device 1 is stored in Memory 12. Some of the information stored in Memory 12 may be stored in an external storage device, such as a storage device 2 that can communicate with the information processing device 1, or in a storage medium that is removable from the information processing device 1. Furthermore, Memory 12 may store information that would otherwise be stored in the storage device 2.

[0021] Interface 13 is an interface for electrically connecting the information processing device 1 with other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data with other devices, or hardware interfaces for connecting with other devices via cables, etc. Interface 13 may also include a user interface, and may perform interface operations between a device functioning as a user interface and the information processing device 1. Examples of user interfaces include an input device that accepts user input, which is an external input, and a display device that displays information. Examples of input devices include touch panels, buttons, keyboards, and voice input devices, and examples of display devices include displays and projectors.

[0022] (3) Overview of lightweight model generation Next, the process of generating a lightweight model by the information processing device 1 will be described. In general terms, the information processing device 1 performs singular value decomposition on the convolution of each layer of the pre-lightweight model, generates pairs of singular vectors obtained from the singular value decomposition between adjacent layers, and determines whether or not to prune each pair based on the impact of the error. This generates a lightweight model that simultaneously achieves dimensionality reduction of the convolutional filter and reduction of the number of channels between layers. As a result, the information processing device 1 generates a model that enables real-time inference processing.

[0023] Figure 3 is a functional block diagram of the information processing device 1. Functionally, the processor 11 of the information processing device 1 includes a singular value decomposition unit 31, a pair generation unit 32, a pruning selection unit 33, a lightweight model generation unit 34, and a fine-tuning unit 35. In Figure 3, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to Figure 3. The same applies to the diagrams of other functional blocks described later.

[0024] The singular value decomposition unit 31 performs singular value decomposition on the convolution operations in each layer of the pre-lightweight model and supplies the information about the pre-lightweight model that has been singular value decomposed to the pruning selection unit 33. For example, if the matrix representing the convolution operation (convolution filter) in a certain convolution layer of the pre-lightweight model is "W", the singular value decomposition unit 31 represents the matrix W using the input transformation matrix "U", the output transformation matrix "V", and the matrix "P" representing the dimensionally reduced convolution operation, as follows. W=P·U·V

[0025] Here, the input transformation matrix U is a matrix consisting of left singular vectors, and the output transformation matrix V is a matrix consisting of right singular vectors. The singular value decomposition unit 31 supplies information about these matrices U, V, and P, which are the results of the singular value decomposition, to the pruning selection unit 33. The singular value decomposition of convolution operations is disclosed in the following non-patent document. Jun Wang, Shengchen Li, Wenwu Wang,SVD-Based Channel Pruning for Convolutional Neural Network in Acoustic Scene Classification Model, 2019, IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2019, P390-395.

[0026] The pair generation unit 32 generates pairs of singular vectors between adjacent convolutional layers of the pre-lightening model. In this case, for each adjacent convolutional layer, the pair generation unit 32 generates pairs of column vectors from the output transformation matrix V of the previous layer and the corresponding row vectors from the input transformation matrix U of the immediately following layer. Specifically, the pair generation unit 32 performs the process of generating pairs of the r-th column vector of the output transformation matrix V of the previous layer and the r-th row vector of the input transformation matrix U of the subsequent layer for "r=1,...,R". Here, "r" is a sequential index, and "R" represents the number of column vectors in the output transformation matrix V of the previous layer and the number of row vectors in the input transformation matrix U of the subsequent layer.

[0027] Here, if the pre-lightening model contains "N" (where N is an integer greater than or equal to 2) convolutional layers, then there are N-1 pairs of adjacent layers. Therefore, the pair generation unit 32 generates pairs for N-1 layers by combining each column vector of the output transformation matrix V of the previous layer with the corresponding row vector of the input transformation matrix U of the later layer.

[0028] The pair generation unit 32 then supplies the generated pair of singular vectors to the pruning selection unit 33 as candidates for pruning. Hereafter, the pair of singular vectors generated by the pair generation unit 32 will also be referred to as a "pruning candidate pair".

[0029] The pruning selection unit 33 selects pairs to be pruned from the candidate pruning pairs based on the impact on the error of the pre-lightweight model when the singular vectors constituting each candidate pruning pair generated by the pair generation unit 32 are actually pruned (i.e., deleted). The pruning selection unit 33 then notifies the lightweight model generation unit 34 of the candidate pruning pairs to be pruned, along with the information output by the singular value decomposition unit 31.

[0030] Specifically, the pruning selection unit 33 uses the dataset D2 to calculate the error of the pre-lightened model for each pruning candidate pair, both when the singular vectors constituting the pruning candidate pair are pruned and when they are not.

[0031] The error here refers to the difference between the data output by the pre-lightened model when input data is fed into the pre-lightened model and the ground truth data, and may be calculated using any loss function. For example, if the pre-lightened model outputs confidence scores for each class, the error corresponds to the difference between the confidence scores for each class output by the pre-lightened model and the confidence scores of the ground truth shown by the ground truth data. Also, if the pre-lightened model outputs a confidence map, the error corresponds to the difference between the confidence map output by the pre-lightened model and the confidence map of the ground truth shown by the ground truth data. Note that the error may also be the difference between the pre-lightened model and the ground truth of an inference task. For example, in a classification task, the error may be an index representing the agreement or disagreement between the most likely class identified based on the output of the pre-lightened model and the ground truth class shown by the ground truth data.

[0032] The pruning selection unit 33 then determines whether the error when pruning a candidate pair of pruning candidates worsens by a predetermined degree or more compared to the error when the candidate pair of pruning candidates are not pruned. The predetermined degree is a threshold for error increase used to determine whether the impact of the error is within an acceptable range, and is stored in advance in, for example, the storage device 2 or memory 12. If the pruning selection unit 33 determines that the error when pruning a candidate pair of pruning candidates increases by a predetermined degree or more compared to the error when the candidate pair of pruning candidates is not pruned, the candidate pair of pruning candidates is not selected for pruning. On the other hand, if the pruning selection unit 33 determines that the increase in error of the pre-lightweight model when actually pruning the singular vectors constituting the candidate pair of pruning candidates is less than the predetermined degree, the candidate pair of pruning candidates is selected for pruning. In another example, the pruning selection unit 33 may select a predetermined number of pruning candidate pairs as targets for pruning, in order of the smallest increase in error that would result from pruning those pairs. In other words, the pruning selection unit 33 calculates the increase in error that would result from pruning each of the pruning candidate pairs, and selects a predetermined number of pruning candidate pairs as targets for pruning the ones with the smallest increase.

[0033] The lightweight model generation unit 34 generates a lightweight model by pruning the candidate pruning pairs selected by the pruning selection unit 33 as targets for pruning. Preferably, the lightweight model generation unit 34 also determines whether or not to integrate the convolutional operations decomposed by singular value decomposition for the convolutional layers of the lightweight model, based on the computational complexity of the lightweight model before and after the integration of the convolutional operations. Hereafter, the mode in which the convolutional operations decomposed by singular value decomposition are not integrated, that is, the mode in which they remain decomposed into an input transformation matrix, a matrix representing a dimensionally reduced convolution operation, and an output transformation matrix, will be called the "first mode". The mode in which the convolutional operations decomposed by singular value decomposition are integrated, that is, the mode in which the three matrices described above are integrated into a single matrix representing a convolution operation, will be called the "second mode". In this case, the lightweight model generation unit 34 selects whether to adopt the first or second configuration for the convolutional layers of the lightweight model, based on the computational complexity of each configuration. Specifically, the lightweight model generation unit 34 calculates the computational complexity required for inference of the lightweight model when the first configuration is adopted, and the computational complexity required for inference of the lightweight model when the second configuration is adopted. If the computational complexity when the first configuration is adopted is smaller than that when the second configuration is adopted, the lightweight model generation unit 34 adopts the first configuration, in which each convolutional layer of the lightweight model remains decomposed. On the other hand, if the computational complexity when the second configuration is adopted is smaller than that when the first configuration is adopted, the lightweight model generation unit 34 adopts the second configuration, in which each convolutional layer of the lightweight model is integrated. The lightweight model generation unit 34 supplies model information for constructing the generated lightweight model to the fine-tuning unit 35.

[0034] The fine-tuning unit 35 constructs a lightweight model based on the lightweight model information supplied by the lightweight model generation unit 34, and updates the parameters of the constructed lightweight model using the dataset D2. Specifically, the fine-tuning unit 35 uses pairs of input data and ground truth data included in the dataset D2 to update the parameters of the lightweight model so that the error between the data output by the lightweight model when input data is input to the lightweight model and the ground truth data corresponding to the input data is minimized. The algorithm used to determine the parameters in order to minimize the error may be any learning algorithm used in machine learning, such as gradient descent or backpropagation. The fine-tuning unit 35 then outputs the model information of the lightweight model after fine-tuning.

[0035] The singular value decomposition unit 31, pair generation unit 32, pruning selection unit 33, lightweight model generation unit 34, and fine-tuning unit 35 can be implemented, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to implement each component. At least a portion of these components may be implemented not only by software programs, but also by any combination of hardware, firmware, and software. At least a portion of these components may also be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program composed of the above components may be implemented using this integrated circuit. At least a portion of each component may also be implemented by an ASSP (Application Specific Standard Produce), ASIC (Application Specific Integrated Circuit), or quantum processor (quantum computer control chip). Thus, each component may be implemented by various hardware. The same applies to other embodiments described later. Furthermore, each of these components may be realized through the collaboration of multiple computers, for example, using cloud computing technology.

[0036] (4) Specific example First, we will explain a specific example of singular value decomposition performed by the singular value decomposition unit 31.

[0037] FIG. 4 is a diagram showing, one above the other, the processing flow before singular value decomposition and the processing flow after singular value decomposition in the convolutional layers of the L-th layer and the (L + 1)-th layer among the first to N-th convolutional layers of the pre-lightweight model. In other words, the upper part of FIG. 4 shows the processing flow of the convolutional layers in the L-th layer and the (L + 1)-th layer before singular value decomposition, and the lower part of FIG. 4 shows the processing flow of the convolutional layers in the L-th layer and the (L + 1)-th layer after singular value decomposition. Here, "L" represents any integer from "1" to "N - 1".

[0038] Here, the convolution weight "T" of the L-th layer before singular value decomposition is represented by a 4D tensor as follows, using the vertical size "w" (L) in the L-th layer, the horizontal size "h" (L) in the L-th layer, the channel size "C" (L) in the L-th layer, and the channel size "C" (L+1) in the (L + 1)-th layer. w (L) ×h (L) ×C (L) ×C (L+1)

[0039] Also, as shown in FIG. 4, the input feature map input to the convolutional layer of the L-th layer has a size of "W" (L) ×H (L) ×C (L) . Also, the output feature map output from the convolutional layer of the L-th layer (i.e., the input feature map to the convolutional layer of the (L + 1)-th layer) has a size of "W" (L+1) ×H (L+1) ×C (L+1) . Also, the output feature map output from the convolutional layer of the (L + 1)-th layer (i.e., the input feature map to the convolutional layer of the (L + 2)-th layer) has a size of "W" (L+2) ×H (L+2) ×C (L+2) .

[0040] The size of the input conversion matrix "U" (L) of the L-th layer obtained by singular value decomposition is "C" (L) ×n (L) , and the size of the output conversion matrix "V" (L) of the L-th layer is "m" (L)×C (L+1) If we assume that the weights "T'" of the convolution of the Lth layer after singular value decomposition are "n (L) " and "m (L) Using this, it can be represented by the following 4-dimensional tensor. w (L) ×h (L) ×n (L) xm (L)

[0041] In the L-th convolutional layer, the input transformation matrix U (L) The input is transformed by this, and the output transformation matrix V (L) The output is transformed by the input transformation matrix U. (L) This is the element "C" of the convolution weight T' of the Lth layer. (L) ×(w (L) ×h (L) ×C (L+1) This corresponds to a matrix consisting of left singular vectors obtained by singular value decomposition of the matrix representation in ). Also, the output transformation matrix V (L) This is the element of the convolution weight T' of the Lth layer, "(w (L) ×h (L) ×C (L) )×C (L+1) This corresponds to a matrix consisting of right singular vectors obtained by singular value decomposition of the matrix representation in "[ ]".

[0042] Similarly, the weights of the L+1th layer convolution are "n (L+1) " and "m (L+1) Using this, it can be represented by the following 4-dimensional tensor. w (L+1) ×h (L+1) ×n (L+1) xm (L+1)

[0043] Then, in the L+1th convolutional layer, the input transformation matrix U (L+1) The input is transformed by this, and the output transformation matrix V (L+1) The output is being converted by this process.

[0044] In the configuration after singular value decomposition shown in Figure 4, the output transformation matrix V is generated at the output of the L layer. (L) The number of channels is m (L) From C(L+1) is expanded, and when inputting to the (L + 1)-th layer, C (L+1) ×n (L+1) input conversion matrix U (L+1) to compress the channel number from C (L+1) to n (L+1) .

[0045] In addition, the following relational expressions hold due to the properties of singular value decomposition. n (L) < min(C (L) , w (L) ×h (L) ×C (L+1) ) That is, n (L) ≤ C (L) m (L) < min(w (L) ×h (L) ×C (L) , C (L+1) ) That is, m (L) ≤ C (L+1)

[0046] By such singular value decomposition, it becomes possible to reduce the number of parameters and the amount of calculation.[[ID=...]] [[ID=...]]

[0047] [[ID=...]] For example, if the dimensions of n and m after pruning are represented by n' and m' respectively, the number of parameters before singular value decomposition is "w (L) ×h (L) ×C (L) ×C (L+1) ", while the number of parameters after singular value decomposition is "C (L) ×n' (L) + w (L) ×h (L) ×n' (L) ×m' (L) + m' (L) ×C (L+1) ". Also, assuming a complete convolutional layer, the amount of calculation (number of multiplications) before singular value decomposition is "W (L) ×H (L) ×C (L+1) ×w (L) ×h (L) ×C (L) ", and the amount of calculation after singular value decomposition is "W (L) ×H(L) × (C (L) × n')[[' (L) + m')[[' (L+1) × w')[[' (L) × h')[[' (L) × n')[[' (L) + m')[[' (L) × C')[[' (L+1) )".')[['

[0048] FIG. 5 shows vertically side by side the processing flow of the convolutional layer in the L-th layer and the (L + 1)-th layer after singular value decomposition and the processing flow of the convolutional layer in the L-th layer and the (L + 1)-th layer after pruning. That is, the upper part of FIG. 5 shows the processing flow of the convolutional layer in the L-th layer and the (L + 1)-th layer after singular value decomposition, and the lower part of FIG. 5 shows the processing flow of the convolutional layer in the L-th layer and the (L + 1)-th layer after pruning.

[0049] In this case, the pair generation unit 32 generates, as pruning candidate pairs, pairs of the r-th column vector of the output conversion matrix V of the L-th layer having the size of "m (L) × C (L+1) " and the r-th row vector of the input conversion matrix U of the (L + 1)-th layer having the size of "C (L) × n (L+1) ". Then, the lightweight model generation unit 34 replaces all elements of the pruning candidate pairs with 0 when the influence on the error of the pre-lightweight model is smaller than a predetermined degree.

[0050] Through the above pruning process, the number of channels is suitably reduced. In FIG. 5, the size of the output conversion matrix V of the L-th layer is "m (L) × C (L) ", and the size of the input conversion matrix U of the (L + 1)-th layer is "C ~(L+1) × n (L) ". Here, "C ~(L+1) " is the number of effective (i.e., not all elements are 0) vectors in the output conversion matrix V (L+1) and the input conversion matrix U​​​​​​​​​​​​​The following is true: The size of the output feature map is "W (L+1) ×H (L+1) ×C (L+1) From " to "W (L+1) ×H (L+1) ×C ~(L+1) It will be reduced to .

[0051] The above pruning process can reduce the number of parameters and the computational complexity. Specifically, the number of parameters is "m (L) ×C (L+1) +C (L+1) ×n (L+1) " to "m (L) ×C ~(L+1) +C ~(L+1) ×n (L+1) The computational complexity (in this case, the number of multiplications) is "W (L+1) ×H (L+1) ×(m (L) ×C (L+1) +C (L+1) ×n (L+1) )" to "W (L+1) ×H (L+1) ×(m (L) ×C ~(L+1) +C ~(L+1) ×n (L+1) )”

[0052] (5) Processing flow Figure 6 is an example of a flowchart showing an overview of the processes performed by the information processing device 1.

[0053] First, the information processing device 1 performs singular value decomposition of the N convolutional layers of the pre-lightweight model, which was constructed by referring to the pre-lightweight model information D1 (step S11). Then, if the index L of the convolutional layer to be processed is not set (i.e., if step S12 is executed for the first time), the information processing device 1 sets L to "L=1", and in all other cases, it increments the index L by 1 (step S12).

[0054] Next, the information processing device 1 generates pairs of pruning candidate pairs, which are each column vector of the output transformation matrix of the L layer and the corresponding row vector of the input transformation matrix of the L+1 layer (i.e., the same index as the column vector) (step S13).

[0055] Next, the information processing device 1 selects from the pruning candidate pairs generated in step S13 that the effect of the error in the pre-lightening model when deleted is within an acceptable range as the pair to be pruned (step S14).

[0056] Then, the information processing device 1 determines whether or not "L=N-1" is true (step S15). If "L=N-1" is true (step S15; Yes), the information processing device 1 determines that it has determined pruning candidate pairs for all adjacent layers of the convolutional layer and proceeds to step S16. On the other hand, if "L=N-1" is not true (step S15; No), the information processing device 1 determines that there are adjacent layers of the convolutional layer for which pruning candidate pairs have not been determined and returns to step S12.

[0057] After determining that "L=N-1" holds true, the information processing device 1 generates a lightweight model by deleting the pruning candidate pairs selected as pruning targets (step S16). In this case, preferably, the information processing device 1 may choose whether to adopt a first mode in which each convolutional layer of the lightweight model is left decomposed, or a second mode in which the decomposed matrices are integrated, based on the computational complexity of each. Next, the information processing device 1 updates the parameters of the lightweight model using the dataset D2 (step S17). This allows the information processing device 1 to fine-tune the lightweight model.

[0058] <Second Embodiment> Figure 7 is a block diagram of the information processing device 1X. The information processing device 1X mainly comprises an acquisition means 31X, a pair generation means 32X, a selection means 33X, and a model generation means 34X. The information processing device 1X can be, for example, the information processing device 1 of the first embodiment. The information processing device 1X may be composed of multiple devices.

[0059] The acquisition means 31X acquires the input transformation matrix and the output transformation matrix obtained by the singular value decomposition of each convolutional layer included in the neural network. The acquisition means 31X can be, for example, the singular value decomposition unit 31 in the first embodiment.

[0060] The pair generation means 32X generates pairs of column vectors of the output transformation matrix of each convolutional layer and row vectors of the corresponding input transformation matrix of the next convolutional layer. The pair generation means 32X can be, for example, the pair generation unit 32 in the first embodiment.

[0061] The selection means 33X selects pairs to be pruned from the pairs based on the impact of deleting pairs on the error of the neural network. The selection means 33X can be, for example, the pruning selection unit 33 in the first embodiment.

[0062] The model generation means 34X generates a lightweight neural network model by pruning the neural network with the pairs to be pruned. The model generation means 34X can be, for example, the lightweight model generation unit 34 in the first embodiment.

[0063] Figure 8 is an example of a flowchart executed by the information processing device 1X. The acquisition means 31X acquires the input transformation matrix and output transformation matrix obtained by singular value decomposition of each convolutional layer included in the neural network (step S21). The pair generation means 32X generates pairs of the column vector of each output transformation matrix of a convolutional layer and the row vector of the corresponding input transformation matrix of the next convolutional layer (step S22). The selection means 33X selects pairs to be pruned from the pairs based on the impact of deleting the pairs on the error of the neural network (step S23). The model generation means 34X generates a lightweight model of the neural network by pruning the pairs to be pruned from the neural network (step S24).

[0064] According to the second embodiment, the information processing device 1X performs pruning on pairs of row vectors of the input transformation matrix and column vectors of the output transformation matrix obtained by singular value decomposition, taking into account the influence of errors. As a result, the information processing device 1X can suitably generate a lightweight model.

[0065] In each of the embodiments described above, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a processor. Non-transitory computer-readable mediums include various types of tangible storage mediums. Examples of non-transitory computer-readable mediums include magnetic storage mediums (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage mediums (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to the computer using various types of transient computer-readable mediums. Examples of transient computer-readable mediums include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable mediums can supply the program to the computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0066] Furthermore, some or all of the above embodiments (including modified versions, hereinafter the same) may also be described as follows, but are not limited to the following. Also, some or all of the configurations described in Appendices 2 to 8, which are dependent on Appendice 1 above, may also be dependent on Appendices 9 and 10 in the same way as Appendices 2 to 8. Moreover, not limited to the devices, methods, and storage media described in the appendices, some or all of the configurations described in the appendices may also be dependent on methods, various hardware, software, various recording means for recording software (including storage media), or systems, without departing from the above embodiments.

[0067] [Note 1] An acquisition means for obtaining the input transformation matrix and output transformation matrix obtained by singular value decomposition of each convolutional layer included in a neural network, Pair generation means for generating pairs of the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer of the said convolutional layer, A selection means for selecting a pair to be pruned from the pair, based on the impact on the error of the neural network when the pair is deleted, A model generation means for generating a lightweight model of the neural network obtained by pruning the pair of branches to be pruned from the neural network, An information processing device having [Note 2] The information processing apparatus according to Appendix 1, wherein the model generation means determines, after pruning the pair of branches to be pruned, whether or not to integrate the convolution processes decomposed by the singular value decomposition with respect to the convolutional layer of the lightweight model, based on the computational complexity of the lightweight model before and after the integration of the convolution processes. [Note 3] The effect on the error is the increase in the error when the pair is deleted. The information processing apparatus according to Appendix 1, wherein the selection means selects a pair such that the increase in the error is less than or equal to a predetermined degree, or a predetermined number of pairs in order of increasing error, as the pairs to be pruned. [Note 4] The acquisition means is an information processing apparatus according to Appendix 1, which acquires the input transformation matrix and the output transformation matrix for each of the convolutional layers. [Note 5] The information processing apparatus according to Appendix 1, wherein the pair generation means generates pairs of each column vector of the output transformation matrix of each of the convolutional layers and row vectors having the same index as each column vector of the input transformation matrix of the next convolutional layer. [Note 6] The information processing apparatus as described in Appendix 1, wherein the row vectors of the input transformation matrix are left singular vectors obtained by the singular value decomposition, and the column vectors of the output transformation matrix are right singular vectors obtained by the singular value decomposition. [Note 7] The information processing apparatus according to Appendix 1, further comprising an update means for updating the parameters of the lightweight model based on the training data of the neural network. [Note 8] The information processing device described in Appendix 1, wherein the model generation means generates the lightweight model in which all elements of the pair to be pruned are set to 0. [Note 9] Computers The input and output transformation matrices obtained by singular value decomposition of each convolutional layer in the neural network are acquired. A pair is generated between the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer. Based on the impact on the error of the neural network when the aforementioned pair is deleted, select a pair from the aforementioned pair to be pruned. A method for generating a lightweight model of the neural network by pruning the pair of branches to be pruned from the neural network. [Note 10] The input and output transformation matrices obtained by singular value decomposition of each convolutional layer in the neural network are acquired. A pair is generated between the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer. Based on the impact on the error of the neural network when the aforementioned pair is deleted, select a pair from the aforementioned pair to be pruned. A program that causes a computer to perform a process of generating a lightweight model of the neural network by pruning the aforementioned pair of branches from the neural network. [Note 11] A storage medium containing the program described in Appendix 10.

[0068] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the structure and details of the present invention can be made that are understandable to those skilled in the art within the scope of the present invention. That is, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that those skilled in the art could make in accordance with the technical idea. Furthermore, each disclosure of the above-mentioned patent documents and other references is incorporated herein by reference. [Explanation of Symbols]

[0069] 1. 1X Information Processing Device 2 Storage device 11 processors 12 memory 13 Interfaces 100 Lightweight Model Generation Systems D1 Pre-Lightweight Model Information D2 dataset

Claims

1. An acquisition means for obtaining the input transformation matrix and output transformation matrix obtained by singular value decomposition of each convolutional layer included in a neural network, Pair generation means for generating pairs of the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer of the said convolutional layer, A selection means for selecting a pair to be pruned from the pair, based on the impact on the error of the neural network when the pair is deleted, A model generation means for generating a lightweight model of the neural network obtained by pruning the pair of branches to be pruned from the neural network, An information processing device having

2. The information processing apparatus according to claim 1, wherein the model generation means determines, after pruning the pair of branches to be pruned, whether or not to integrate the convolution processes decomposed by the singular value decomposition with respect to the convolution layer of the lightweight model, based on the computational complexity of the lightweight model before and after the integration of the convolution processes.

3. The effect on the error is the increase in the error when the pair is deleted. The information processing apparatus according to claim 1, wherein the selection means selects a pair such that the increase in the error is less than or equal to a predetermined degree, or a predetermined number of pairs in order of increasing error, as the pairs to be pruned.

4. The information processing apparatus according to claim 1, wherein the acquisition means acquires the input transformation matrix and the output transformation matrix for each of the convolutional layers.

5. The information processing apparatus according to claim 1, wherein the pair generation means generates pairs of each column vector of the output transformation matrix of each of the convolutional layers and row vectors having the same index as each column vector of the input transformation matrix of the next convolutional layer.

6. The information processing apparatus according to claim 1, wherein the row vectors of the input transformation matrix are left singular vectors obtained by the singular value decomposition, and the column vectors of the output transformation matrix are right singular vectors obtained by the singular value decomposition.

7. The information processing apparatus according to claim 1, further comprising an update means for updating the parameters of the lightweight model based on the training data of the neural network.

8. The information processing apparatus according to claim 1, wherein the model generation means generates the lightweight model in which all elements of the pair to be pruned are set to 0.

9. Computers The input and output transformation matrices obtained by singular value decomposition of each convolutional layer in the neural network are acquired. A pair is generated between the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer. Based on the impact on the error of the neural network when the aforementioned pair is deleted, select a pair from the aforementioned pair to be pruned. A method for generating a lightweight model of the neural network by pruning the pair of branches to be pruned from the neural network.

10. The input and output transformation matrices obtained by singular value decomposition of each convolutional layer in the neural network are acquired. A pair is generated between the column vector of the output transformation matrix of each of the convolutional layers and the row vector of the input transformation matrix of the next convolutional layer. Based on the impact on the error of the neural network when the aforementioned pair is deleted, select a pair from the aforementioned pair to be pruned. A program that causes a computer to perform a process of generating a lightweight model of the neural network by pruning the pair of branches to be pruned from the neural network.

Images