Learning device, trained model generation method, computation device, computation method, and program

Abstract

According to the present invention, a pruning position determination unit determines an edge pruning position pattern for a neural network model. An update unit updates the parameters of a neural network model in which the weights of the edges at pruning positions are set to zero, on the basis of training data consisting of pairs of input and output samples, in such a way that the neural network model outputs an output sample when an input sample is input to the neural network model.

Description

[Technical field]

[0001] The present invention relates to a learning device, a method for generating a trained model, a computing device, a computing method, and a program. [Background technology]

[0002] In machine learning, there is a known attack method called Adversarial Examples, which induces misjudgments by adding a certain amount of noise to input data. In this method, noise is calculated based on the tendency of the trained model's calculations so that the features of the data to be misjudged are close to each other. There is a known technology called PEDM (Promoting Ensemble Diversity Method) that uses multiple neural network models to make robust predictions against adversarial samples (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Tianyu Pang, Taufik Xu, Chao Du, Ning Chen, Jun Zhu. “Improving Adversarial Robustness via Promoting Ensemble Diversity” ICML, 2019 Summary of the Invention [Problem to be solved by the invention]

[0004] Since PEDM uses multiple neural network models, the number of parameters increases in proportion to the number of neural network models used, which means that the memory capacity required for calculation increases in proportion to the number of neural network models used. An example of the objective of the present disclosure is to provide a learning device, a method for generating a trained model, a computing device, a computing method, and a program that can increase robustness against adversarial samples and suppress an increase in memory capacity, in order to solve the above-mentioned problems. [Means for solving the problem]

[0005] According to a first aspect of the present invention, a learning device includes a mowing position determination unit that determines a pattern of mowing positions of edges of a neural network model, and an update unit that updates parameters of the neural network model based on learning data which is a pair of an input sample and an output sample, so that the output sample is output when the input sample is input to an edge of the neural network model having a weight of zero for the edge related to the mowing position.

[0006] According to a second aspect of the present invention, a program causes a computer to execute a step of determining a pattern of edge pruning positions of a neural network model, and a step of updating parameters of the neural network model based on learning data which is a pair of input samples and output samples, so that the output sample is output when the input sample is input to an edge of the neural network model in which a weight of the edge related to the pruning position among the edges of the neural network model is set to zero.

[0007] According to a third aspect of the present invention, a method for generating a trained model includes a step of determining a pattern of pruning positions of edges of a neural network model, a step of updating parameters of the neural network model based on training data which is a pair of input samples and output samples so that the output sample is output when the input sample is input to an edge of the neural network model in which a weight of the edge related to the pruning position is set to zero, and a step of outputting the trained model which is the neural network model with the updated parameters.

[0008] According to a fourth aspect of the present invention, a calculation device includes a trained model acquisition unit that acquires a trained model generated by the trained model output method related to the above aspect, a mowing position determination unit that determines a pattern of mowing positions of edges of the trained model, and a calculation unit that obtains output data by inputting input data to edges of the trained model in which the weight of the edges related to the mowing positions is set to zero.

[0009] According to a fifth aspect of the present invention, a program is a program for causing a computer to execute the steps of obtaining a trained model generated by the trained model output method relating to the above aspect, determining a pattern of mowing positions of edges of the trained model, and obtaining output data by inputting input data to edges of the trained model in which the weight of the edges relating to the mowing positions is set to zero.

[0010] According to a sixth aspect of the present invention, a calculation method includes a step of acquiring a trained model generated by the trained model output method relating to the above aspect, a step of determining a pattern of pruning positions of edges of the trained model, and a step of obtaining output data by inputting input data to edges of the trained model in which the weight of the edges relating to the pruning positions is set to zero. Effect of the Invention

[0011] According to at least one of the above aspects, it is possible to improve robustness against adversarial samples and suppress an increase in storage capacity. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic block diagram showing a configuration of an authentication system according to a first embodiment. [Diagram 2] FIG. 2 is a configuration diagram of a classification model trained by the learning device according to the first embodiment. [Diagram 3]4 is a flowchart showing a feature model learning method performed by the learning device according to the first embodiment. [Figure 4] 4 is a flowchart showing an authentication method performed by the authentication device according to the first embodiment. [Diagram 5] FIG. 2 is a schematic block diagram showing a basic configuration of a learning device. [Figure 6] FIG. 2 is a schematic block diagram showing a basic configuration of a calculation device. [Figure 7] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] First embodiment <Authentication system configuration> Hereinafter, the embodiments will be described in detail with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of an authentication system 1 according to the first embodiment. The authentication system 1 includes a learning device 10 and an authentication device 20. The learning device 10 learns parameters of a feature extraction model so that when biometric data is input, the learning device 10 outputs features of the biometric data. Examples of the biometric data include face images, vein images, fingerprint data, and voice data. The feature extraction model is represented by a machine learning model such as a neural network. The authentication device 20 uses a feature extraction model (trained model) having parameters trained by the learning device 10 to perform user authentication based on biometric data. Note that, although the authentication system 1 according to the first embodiment includes the learning device 10 and the authentication device 20 as separate devices, this is not limited thereto. For example, in the authentication system 1 according to other embodiments, the authentication device 20 may have the functions of the learning device 10.

[0014] Configuration of learning device 10 FIG. 2 is a configuration diagram of a classification model M0 trained by the learning device 10 according to the first embodiment. The learning device 10 includes a classification model storage unit 11, a dataset acquisition unit 12, a harvesting position determination unit 13, a calculation unit 14, a perturbation generation unit 15, an update unit 16, a model extraction unit 17, and an output unit 18.

[0015] The classification model storage unit 11 stores a classification model M0 configured by a neural network. As shown in FIG. 2, the classification model M0 includes an input layer M01, an intermediate layer M02, and an output layer M03. The input layer M01 outputs input biometric data as a vector to the intermediate layer M02. The intermediate layer M02 is configured by a neural network of two or more layers and has one or more fully connected layers. The intermediate layer M02 may be configured by a CNN (Convolutional Neural Network). The intermediate layer M02 converts the vector input from the input layer M01 into a low-dimensional feature vector and outputs it to the output layer M03. The feature vector is an example of a feature. The output layer M03 converts the feature vector input from the intermediate layer M02 into a P-dimensional vector indicating the posterior probability of the person represented by the feature vector.

[0016] The dataset acquisition unit 12 acquires a learning dataset in which biometric data, which is an input sample, is associated with a person label, which is an output sample. The person label is represented by a P-dimensional one-hot vector, where P is the number of people in the dataset.

[0017] The reaping position determination unit 13 randomly determines a pattern of reaping positions of edges related to the fully connected layer of the intermediate layer M02. Specifically, the reaping position determination unit 13 generates a mask vector that masks the edges of the classification model M0 based on a predetermined reaping rate. The mask vector is a vector having elements corresponding to the edges of the classification model M0. Among all elements of the mask vector, the value of the element corresponding to the reaping rate is 0, and the values ​​of the other elements are 1.

[0018] The calculation unit 14 calculates an output vector by inputting the input data to a temporary model in which the edges of the classification model M0 are masked by the mask vector generated by the harvesting position determination unit 13.

[0019] The perturbation generation unit 15 calculates a perturbation vector for generating an adversarial sample based on the temporary model. The perturbation vector is a vector for changing the element having the largest value among the vectors output by the temporary model by adding it to the input data. The perturbation vector can be generated, for example, using the PGD Attack method shown in Reference 1 (Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu, “Towards Deep Learning Models Resistant to Adversarial Attacks”, ICLR 2018).

[0020] The update unit 16 uses the learning dataset acquired by the dataset acquisition unit 12 to train parameters of the classification model M0 so that when biometric data is input into the classification model M0 whose edges are masked by the mask vector generated by the harvesting position determination unit 13, a P-dimensional vector indicating the posterior probability of a person corresponding to the biometric data is output.

[0021] The model extraction unit 17 extracts a partial model consisting of the input layer M01 and the intermediate layer M02 from the classification model M0 trained by the update unit 16 as a feature extraction model. The output unit 18 outputs the feature extraction model extracted by the model extraction unit 17 to the authentication device 20. The output may be performed by communication via a network or the like, or may be performed via a portable storage device.

[0022] 《Learning Method》 FIG. 3 is a flowchart showing a feature model learning method performed by the learning device 10 according to the first embodiment. The data set acquisition unit 12 of the learning device 10 acquires a data set prepared in advance from a database (not shown) (step S1). The reaping position determination unit 13 generates a plurality of mask vectors indicating the reaping positions of the edges based on random numbers (step S2). At this time, the reaping position determination unit 13 generates a plurality of mask vectors so that no completely matching mask vector pair is generated.

[0023] The learning device 10 selects the generated mask vectors one by one (step S3), and executes the following processing from step S4 to step S7 for each selected mask vector. The calculation unit 14 applies the mask vector selected in step S3 to the classification model M0 stored in the classification model storage unit 11, and generates a temporary model in which the transfer functions of some edges in the intermediate layer M02 are replaced with zero (step S4). That is, the calculation unit 14 prunes the edges of the intermediate layer M02.

[0024] The calculation unit 14 selects pairs of input samples and output samples included in the acquired data set one by one (step S5), and executes the following processing from step S6 to step S7 for each selected pair. Note that the processing from step S5 to step S7 may be executed as a batch process.

[0025] The calculation unit 14 inputs the input sample selected in step S5 to the temporary model generated in step S4, and calculates an output vector (step S6). The perturbation generation unit 15 generates a perturbation vector based on the temporary model generated in step S4 and the input sample selected in step S5 (step S7).

[0026] The update unit 16 calculates a loss function L based on the following formula (1) using the output vector and the perturbation vector calculated for the multiple input samples and the multiple temporary models (step S8).

[0027]

number

[0028] In formula (1), D denotes a set of data sets. M k denotes a set of mask vectors with a clipping ratio k. K denotes the number of mask vectors generated by the clipping position determination unit 13. θ denotes a classification model M0. x denotes an input sample. y denotes an output sample. L n L indicates the error criterion function expressed by the following equation (2). d denotes the diversity criterion function expressed by the following equation (3).

[0029]

number

[0030] In equation (2), f(θ, x) represents an output vector obtained by inputting data x to the model θ. That is, in the first embodiment, the error criterion function L n is the cross-entropy error.

[0031]

number

[0032] In equation (3), ∇f(θ, x) represents a perturbation vector for a combination of the model θ and the data x. That is, in the first embodiment, the diversity criterion function L d represents the average value of the cosine distance between the perturbation vectors of the temporary models with different mask positions. The diversity criterion function L d is an evaluation criterion for updating the parameters of the classification model M0 so that the difference between the calculation results for each harvesting position pattern becomes larger. One example of a method for expressing the magnitude of the difference between the calculation results is the distance between the calculation results.

[0033] The update unit 16 updates the parameters of the classification model M0 stored in the classification model storage unit 11 so as to reduce the calculated loss function L (step S9). At this time, the update unit 16 updates the classification model M0 instead of the temporary model. In this way, the update unit 16 can train the parameters of the classification model M0 so that, when biometric data is input to the temporary model, a P-dimensional vector indicating the posterior probability of a person corresponding to the biometric data is output.

[0034] The update unit 16 determines whether the parameter update process of the classification model M0 has been performed a predetermined number of epochs (step S10). If the number of times the parameter update process of the classification model M0 has been performed is less than the predetermined number of epochs (step S10: NO), the learning device 10 returns the process to step S2 and repeats the parameter update process.

[0035] On the other hand, if the number of times the parameter update process of the classification model M0 has been performed reaches a predetermined number of epochs (step S10: YES), the model extraction unit 17 extracts a partial model consisting of the input layer M01 and the intermediate layer M02 from the classification model M0 stored in the classification model storage unit 11 as a feature extraction model (step S11). The output unit 18 outputs the feature extraction model extracted in step S11 to the authentication device 20 (step S12). The temporary model is discarded when the learning process ends.

[0036] Configuration of Authentication Device 20 Referring to FIG. 1, the authentication device 20 includes a user data storage unit 21, a model acquisition unit 22, an extraction model storage unit 23, a biometric data acquisition unit 24, a reaping position determination unit 25, a feature extraction unit 26, a similarity calculation unit 27, and an authentication unit .

[0037] The user data storage unit 21 stores the account data of a user in association with the biometric data of the user. The model acquisition unit 22 acquires a trained feature extraction model from the learning device 10. The extraction model storage unit 23 stores the trained feature extraction model acquired by the model acquisition unit 22.

[0038] The biometric data acquisition unit 24 acquires biometric data to be authenticated from a sensor or the like provided in the authentication device 20 . The reaping position determination unit 25 randomly determines a pattern of the reaping positions of the edges related to the feature extraction model. Specifically, the reaping position determination unit 25 generates a mask vector that masks the edges of the feature extraction model based on a predetermined reaping rate. That is, the reaping position determination unit 25 performs the same process as the reaping position determination unit 13. Note that the reaping rate used by the reaping position determination unit 25 may be different from the reaping rate used by the reaping position determination unit 13 of the learning device 10. For example, the reaping rate used by the reaping position determination unit 25 may be 0%.

[0039] The feature extraction unit 26 generates a temporary model by masking edges of the feature extraction model stored in the extraction model storage unit 23 with the mask vector generated by the reaping position determination unit 13. The feature extraction unit 26 extracts feature vectors from the biometric data stored in the user data storage unit 21 and the biometric data acquired by the biometric data acquisition unit 24 using the generated temporary model.

[0040] The similarity calculation unit 27 calculates the similarity between two feature vectors. Examples of the similarity scale include L2 distance, cosine similarity, and PLDA (Probabilistic Linear Discriminant Analysis).

[0041] The authentication unit 28 performs authentication as to whether or not the user is a user stored in the user data storage unit 21, based on the similarity calculated by the similarity calculation unit 27. When the authentication unit 28 determines that the user is a user stored in the user data storage unit 21, it returns the account data of the user.

[0042] Authentication method 4 is a flowchart showing an authentication method by the authentication device 20 according to the first embodiment. It is assumed that, prior to execution of the authentication method, the model acquisition unit 22 acquires a trained feature extraction model from the learning device 10 and records it in the extraction model storage unit 23.

[0043] The biometric data acquisition unit 24 of the authentication device 20 acquires biometric data from a sensor or the like connected to the authentication device 20 (step S21). The reaping position determination unit 13 generates one mask vector indicating the reaping position of the edge based on a random number (step S22). The feature extraction unit 26 applies the mask vector to the feature extraction model stored in the extraction model storage unit 23, and generates a temporary model in which the transfer functions of some edges are replaced with zero (step S23).

[0044] The feature extraction unit 26 inputs the biometric data acquired in step S21 into the temporary model generated in step S23 to calculate a feature vector (step S24). Next, the authentication device 20 selects the users stored in the user data storage unit 21 one by one (step S25), and executes steps S26 to S27 described below.

[0045] First, the feature extraction unit 26 inputs the biometric data associated with the user selected in step S25 into the temporary model generated in step S23 to calculate a feature vector (step S26). Next, the similarity calculation unit 27 calculates the similarity between the feature vector calculated in step S24 and the feature vector calculated in step S26 (step S27).

[0046] The authentication unit 28 calculates the similarity between the acquired biometric data and each user stored in the user data storage unit 21, and then determines whether any of the calculated similarities exceeds a predetermined authentication threshold (step S28). If all similarities are equal to or less than the authentication threshold (step S28: NO), the authentication unit 28 determines that the authentication using the biometric data acquired in step S21 has failed (step S29), and ends the process.

[0047] On the other hand, if at least one similarity exceeds the authentication threshold (step S28: YES), the authentication unit 28 identifies the user with the highest similarity (step S30) and outputs the account data of the user (step S31). Note that the temporary model is discarded when the authentication process ends.

[0048] Actions and Effects In this way, according to the authentication system 1 according to the first embodiment, by randomly masking one feature extraction model, it is possible to generate an infinite number of temporary models capable of extracting features in the same way as the feature extraction model. This is because the parameters of the classification model M0 are updated so that an output sample is output when an input sample is input to the temporary model obtained by pruning edges of the classification model M0 that are related to the randomly determined pruning position. Since it is difficult to generate an adversarial sample for an infinitely generable temporary model, it can be said that the authentication system 1 according to the first embodiment has high robustness against adversarial samples. In addition, since the infinite number of temporary models in the first embodiment are generated from one feature extraction model, the authentication system 1 needs to hold only one feature extraction model, and therefore the authentication system 1 according to the first embodiment can suppress an increase in storage capacity even though it is capable of performing calculations using an infinite number of temporary models.

[0049] <Experimental Example> The inventors conducted a comparative experiment between the method according to the first embodiment and the method described in Non-Patent Document 1. In the comparative experiment, CIFAR10 (https: / / www.cs.toronto.edu / ~kriz / cifar.html[searched May 14, 2021]) was used as a learning dataset. In the comparative experiment, a classification model M0 was used as a model of the method according to the first embodiment. The number of ensembles K in the first embodiment and Non-Patent Document 1 was set to 3. The PGD Attack method in Reference Document 1 was used to generate adversarial perturbations. The magnitude of perturbation (the ratio of the image from which the adversarial samples are generated and the L of the adversarial samples) was determined as follows: ∞ The distance) was set to 0.02. The pruning rate in the learning device 10 was set to 20%, and the number K of mask vectors was set to 2. The pruning rate in the authentication device 20 was set to 0%.

[0050] As a result, when using the model obtained by the method described in Non-Patent Document 1, the rate at which normal data (data to which no perturbation vector was added) was correctly classified was 95.4%, and the rate at which adversarial samples (data to which no perturbation vector was added) were correctly classified was 36.5%. In contrast, when the classification model M0 according to the first embodiment was used, the rate at which normal data was correctly classified was 93.3%, and the rate at which adversarial samples were correctly classified was 58.7%.

[0051] The data size of the model obtained by the method described in Non-Patent Document 1 is 32×P×K, whereas the data size of the classification model M0 according to the first embodiment is 32×P. P indicates the number of parameters of one model, and K indicates the number of models. This is because one parameter is represented by a 32-bit float. In other words, the classification model M0 according to the first embodiment can reduce the data size to 1 / K of the method described in Non-Patent Document 1, and can achieve the same level of accuracy as the method described in Non-Patent Document 1. In particular, the rate at which adversarial samples were correctly classified was higher than that of the method described in Non-Patent Document 1.

[0052] Other Embodiments Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes are possible. That is, in other embodiments, the order of the above-mentioned processes may be changed as appropriate. Also, some of the processes may be executed in parallel.

[0053] Although the learning device 10 and the authentication device 20 according to the above-described embodiment are configured as separate computers, in other embodiments they may be realized as a single computer.

[0054] The authentication device 20 according to the above-described embodiment performs authentication processing based on the similarity of features extracted by the feature extraction model, but is not limited to this. For example, the authentication device 20 according to another embodiment may perform authentication processing based on the output of the classification model M0. In the above-described embodiment, the arithmetic device using the trained model is applied to the authentication device 20 that performs biometric authentication, but is not limited to this. For example, the arithmetic device according to another embodiment may be a device that performs processing other than biometric authentication, such as a device that inputs data other than biometric data, such as an image of a product, and determines whether the product is genuine or not.

[0055] The authentication device 20 according to the embodiment described above extracts features based on one temporary model obtained by pruning a feature extraction model, but is not limited to this. For example, the authentication device 20 according to another embodiment may generate multiple temporary models from one feature extraction model and extract features by ensemble processing of the multiple temporary models.

[0056] The learning device 10 according to the embodiment described above uses the diversity criterion function shown in formula (3) to update the parameters of the classification model M0 so that the difference between the calculation results for each pattern of the reaping position is increased, but is not limited to this. For example, the learning device 10 according to another embodiment may use a loss function that increases the distance between vectors in which the correct label is masked, for example, among the calculation results using the temporary model. In other words, the learning device 10 may update the parameters of the classification model M0 without calculating the perturbation vector.

[0057] <Basic configuration> FIG. 5 is a schematic block diagram showing the basic configuration of the learning device 010. As shown in FIG. In the above embodiment, the learning device 10 shown in FIG. 1 has been described as one embodiment of the learning device 010, and the basic configuration of the learning device 010 is as shown in FIG. That is, the learning device 010 basically comprises a reaping position determination unit 011 and an update unit 012 .

[0058] The reaping position determination unit 011 determines a pattern of the reaping positions of the edges of the neural network model. The reaping position determination unit 011 corresponds to the reaping position determination unit 13. The update unit 012 updates the parameters of the neural network model based on the learning data, which is a set of an input sample and an output sample. Specifically, the update unit 012 updates the parameters so that an output sample is output when an input sample is input to an edge of the neural network model in which the weight of an edge related to a cutting position is set to zero. The update unit 012 corresponds to the update unit 16.

[0059] This enables the learning device 010 to increase robustness against adversarial samples and suppress an increase in memory capacity.

[0060] FIG. 6 is a schematic block diagram showing the basic configuration of the arithmetic device 020. As shown in FIG. In the above embodiment, the authentication device 20 shown in FIG. 1 has been described as one embodiment of the arithmetic device 020, but the basic configuration of the arithmetic device 020 is as shown in FIG. That is, the calculation device 020 basically includes a trained model acquisition unit 021, a reaping position determination unit 022, and a calculation unit 023.

[0061] The trained model acquisition unit 021 acquires the trained model generated by the learning device 010. The trained model acquisition unit 021 corresponds to the model acquisition unit 22. The reaping position determination unit 022 determines a pattern of the reaping position of the edge of the trained model. The reaping position determination unit 022 corresponds to the reaping position determination unit 25. The calculation unit 023 obtains output data by inputting input data to edges of the trained model in which the weight of the edge related to the cutting position is set to zero. The calculation unit 023 corresponds to the feature extraction unit 26.

[0062] This enables the calculation device 020 to increase robustness against adversarial samples and suppress an increase in memory capacity.

[0063] Computer Configuration FIG. 7 is a schematic block diagram illustrating a computer configuration according to at least one embodiment. The computer 90 comprises a processor 91, a main memory 93, a storage 95, and an interface 97. The learning device 10 and authentication device 20 described above are implemented in a computer 90. The operations of each of the above-mentioned processing units are stored in the form of a program in a storage 95. The processor 91 reads the program from the storage 95, loads it in the main memory 93, and executes the above-mentioned processing according to the program. The processor 91 also secures storage areas in the main memory 93 corresponding to each of the above-mentioned storage units according to the program. Examples of the processor 91 include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a microprocessor.

[0064] The program may be for realizing a part of the functions to be performed by the computer 90. For example, the program may be for realizing the functions by combining with other programs already stored in the storage or by combining with other programs implemented in other devices. In another embodiment, the computer 90 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to the above configuration or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions to be realized by the processor 91 may be realized by the integrated circuit. Such an integrated circuit is also included in an example of a processor.

[0065] Examples of the storage 95 include a magnetic disk, a magneto-optical disk, an optical disk, a semiconductor memory, etc. The storage 95 may be an internal medium directly connected to the bus of the computer 90, or an external medium connected to the computer 90 via the interface 97 or a communication line. In addition, when the program is distributed to the computer 90 via a communication line, the computer 90 that receives the program may load the program into the main memory 93 and execute the above-mentioned processing. In at least one embodiment, the storage 95 is a non-transitory tangible storage medium.

[0066] The program may be for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that realizes the above-mentioned functions in combination with another program already stored in the storage 95. [Explanation of symbols]

[0067] 1...Authentication system 10...Learning device 11...Classification model storage unit 12...Data set acquisition unit 13...Harvesting position determination unit 14...Calculation unit 15...Perturbation generation unit 16...Update unit 17...Model extraction unit 18...Output unit 20...Authentication device 21...User data storage unit 22...Model acquisition unit 23...Extraction model storage unit 24...Biometric data acquisition unit 25...Harvesting position determination unit 26...Feature extraction unit 27...Similarity calculation unit 28...Authentication unit 90...Computer 91...Processor 93...Main memory 95...Storage 97...Interface

Claims

1. a pruning position determination unit that determines a plurality of patterns of pruning positions of edges of the neural network model; an update unit that updates parameters of the neural network model based on learning data that is a set of an input sample and an output sample, so that the output sample is output when the input sample is input to an edge of the neural network model in which a weight of an edge related to the cutting position is set to zero; Equipped with The update unit obtains a perturbation vector for the neural network model in which a weight of an edge related to a reaping position is set to zero for each of the patterns, and updates parameters of the neural network model so that a difference between the perturbation vectors for each of the patterns becomes large. Learning device.

2. The learning device according to claim 1 , wherein the output sample is a one-hot vector representing a correct label of the input sample.

3. The update unit updates parameters of the neural network model such that when the input sample is input to an edge of the neural network model in which a weight of an edge related to the cutting position is set to zero, the output sample is output and a difference between the perturbation vectors for each pattern becomes large. The learning device according to claim 2 .

4. an output unit that outputs a partial model obtained by removing a subsequent layer from the neural network model in which the parameters have been updated as a feature extraction model that extracts features of input data; The learning device according to claim 2 or 3, comprising:

5. On the computer, determining a plurality of patterns of edge pruning locations of the neural network model; updating parameters of the neural network model based on learning data which is a set of an input sample and an output sample, so that the output sample is output when the input sample is input to an edge of the neural network model in which a weight of an edge related to the cutting position is set to zero; Run the command, The step of updating the parameters includes determining a perturbation vector for the neural network model in which the weight of an edge related to a reaping position is set to zero for each of the patterns, and updating the parameters of the neural network model so that the difference between the perturbation vectors for each of the patterns becomes large. program.

6. determining a plurality of patterns of edge pruning locations of the neural network model; updating parameters of the neural network model based on learning data which is a set of an input sample and an output sample, so that the output sample is output when the input sample is input to an edge of the neural network model having a weight of zero for an edge related to the cutting position; outputting a trained model, which is the neural network model with the updated parameters; Equipped with The step of updating the parameters includes determining a perturbation vector for the neural network model in which the weight of an edge related to a reaping position is set to zero for each of the patterns, and updating the parameters of the neural network model so that the difference between the perturbation vectors for each of the patterns becomes large. How to generate a trained model.

7. The step of outputting the trained model includes outputting, as the trained model, a partial model obtained by removing a subsequent layer from the neural network model in which the parameters have been updated. A method for generating a trained model according to claim 6.

8. A trained model acquisition unit that acquires a trained model generated by a trained model generation method comprising the steps of: determining multiple patterns of edge pruning positions of a neural network model; updating parameters of the neural network model based on learning data that is a pair of input samples and output samples so that the output sample is output when the input sample is input to an edge of the neural network model in which the weight of the edge related to the pruning position is set to zero; and outputting a trained model that is the neural network model with the updated parameters; A pruning position determination unit that randomly determines a pattern of pruning positions of edges of the trained model; a calculation unit that obtains output data by inputting input data to edges of the trained model in which a weight of the edge related to the reaping position is set to zero; A computing device comprising:

9. On the computer, A step of acquiring a trained model generated by a trained model generation method including the steps of: determining a plurality of patterns of edge pruning positions of a neural network model; updating parameters of the neural network model based on learning data that is a pair of an input sample and an output sample, so that the output sample is output when the input sample is input to an edge of the neural network model that has a weight of zero for the edge related to the pruning position; and outputting the trained model that is the neural network model with the updated parameters; Randomly determining a pattern of edge pruning positions of the trained model; A step of obtaining output data by inputting input data to edges of the trained model in which a weight of an edge related to the reaping position is set to zero; A program for executing the above.

Images