LEARNING DATA GENERATION DEVICE, LEARNING DATA GENERATION METHOD, PROGRAM, DETECTION MODEL GENERATION METHOD, AND AUTHENTICATION SYSTEM

Abstract

In the present invention, a sample acquisition unit acquires a first sample, which is individual data pertaining to a first individual. An AX generation unit applies a perturbation to the first sample to generate an adversarial example. A feature amount acquisition unit acquires: a first feature amount, which is a feature amount of the adversarial example calculated by a learned model that calculates the feature amount by performing, on individual data that has been input, a calculation that uses a learned parameter; a second feature amount, which is a feature amount of the individual data pertaining to the first individual; and a third feature amount, which is a feature amount of individual data pertaining to a second individual different from the first individual. An output unit outputs an adversarial example which has a degree of similarity between the first feature amount and the second feature amount greater than or equal to a threshold value and a degree of similarity between the first feature amount and the third feature amount greater than or equal to a threshold value.

Description

[Technical field]

[0001] The present invention relates to a learning data generating device, a learning data generating method, a program, a detection model generating method, and an authentication system. [Background technology]

[0002] In machine learning, there is a known attacking technique called Adversarial Examples, which induces misjudgment by adding a certain amount of noise to input data. In this technique, noise is calculated based on the tendency of the trained model's calculations so that the features of the data that is the subject of misjudgment are close to each other. Non-Patent Document 1 discloses a learning method for a detection model that detects adversarial samples from input data. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Zhitao Gong, Wenlu Wang, Wei-Shinn Ku, “Adversarial and Clean Data Are Not Twins”, [online], [Retrieved May 28, 2021], Internet〈URL: https: / / arxiv.org / abs / 1704.04960〉 Summary of the Invention [Problem to be solved by the invention]

[0004] Adversarial samples are a threat to systems that perform learning-based authentication processes, such as person authentication systems. In order to improve the detection accuracy of adversarial samples that pose a threat to authentication systems using a detection model, it is necessary to provide the detection model with appropriate data as learning data, rather than providing randomly generated adversarial samples. In order to solve the above-mentioned problems, the object of the present invention is to provide a training data generation device, a training data generation method, a program, a detection model generation method, and an authentication system that can improve the detection accuracy of hostile samples that pose a threat to authentication systems. [Means for solving the problem]

[0005] According to a first aspect of the present invention, a training data generation device includes a sample acquisition unit that acquires a first sample, which is individual data related to a first individual; an AX generation unit that generates an adversarial sample by applying a perturbation to the first sample; a feature acquisition unit that acquires a first feature that is a feature of the adversarial sample calculated by a trained model that calculates features by performing a calculation using trained parameters on input individual data, a second feature that is a feature of individual data related to the first individual, and a third feature that is a feature of individual data related to a second individual different from the first individual; and an output unit that outputs the adversarial sample in which a similarity between the first feature and the second feature is equal to or greater than a threshold and a similarity between the first feature and the third feature is equal to or greater than a threshold.

[0006] According to a second aspect of the present invention, a method for generating training data includes the steps of acquiring a first sample, which is individual data relating to a first individual; generating an adversarial sample by applying a perturbation to the first sample; acquiring a first feature, which is a feature of the adversarial sample, calculated by a trained model that calculates features by performing a calculation using trained parameters on input individual data, a second feature, which is a feature of individual data relating to the first individual, and a third feature, which is a feature of individual data relating to a second individual different from the first individual; and outputting the adversarial sample, in which a similarity between the first feature and the second feature is equal to or greater than a threshold and a similarity between the first feature and the third feature is equal to or greater than a threshold, as training data.

[0007] According to a third aspect of the present invention, a program causes a computer to execute the steps of acquiring a first sample which is individual data relating to a first individual; generating an adversarial sample by applying a perturbation to the first sample; acquiring a first feature which is a feature of the adversarial sample calculated by a trained model which calculates features by performing a calculation using trained parameters on input individual data, a second feature which is a feature of individual data relating to the first individual, and a third feature which is a feature of individual data relating to a second individual different from the first individual; and outputting the adversarial sample in which a similarity between the first feature and the second feature is equal to or greater than a threshold and a similarity between the first feature and the third feature is equal to or greater than a threshold.

[0008] According to a fourth aspect of the present invention, a method for generating a detection model includes the steps of generating a plurality of adversarial samples by the method for generating learning data as shown in claim 2, and using the generated plurality of adversarial samples and a plurality of normal individual data, learning parameters of a detection model which is a machine learning model that takes individual data as input and outputs a determination result as to whether the individual data is normal individual data or an adversarial sample.

[0009] According to a fifth aspect of the present invention, an authentication system includes a storage medium for storing biometric data used for authenticating a person, a registration device for registering authentication biometric data and account data in the storage medium, an authentication device for authenticating the person based on the storage medium, and a determination device for determining whether the biometric data is an adversarial sample. The registration device includes an acquisition unit for acquiring first biometric data submitted by a first person, a generation unit for generating second biometric data from the first person, and a feature extraction model, which is a trained model that calculates features by performing calculations using trained parameters on input individual data, and a determination unit for determining a similarity between the first biometric data and the second biometric data. and a recording unit that records the first biometric data in the storage medium if the similarity calculated by the first calculation unit is equal to or greater than a threshold. The authentication device comprises a generation unit that generates third biometric data from a second person, a second calculation unit that calculates features of each of the first biometric data stored in the storage medium and features of the third data using the feature extraction model, and a determination unit that determines whether the similarity calculated by the second calculation unit is equal to or greater than a threshold. The determination device comprises a determination unit that determines whether the first biometric data is an adversarial sample by inputting the first biometric data into a trained detection model generated by the detection model generation method related to the above aspect. Effect of the Invention

[0010] According to at least one of the above aspects, it is possible to provide learning data that can improve the accuracy of detecting adversarial samples that pose a threat in an authentication system. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram showing a configuration of an authentication system according to a first embodiment. [Diagram 2] 2 is a schematic diagram showing a computer configuration of the issuing device according to the first embodiment. FIG. [Diagram 3] 5 is a flowchart showing the operation of the issuing device according to the first embodiment. [Figure 4] 1 is a schematic diagram showing a configuration of an authentication device according to a first embodiment. [Diagram 5] 4 is a flowchart showing the operation of the authentication device according to the first embodiment. [Figure 6] 1 is a schematic block diagram showing a configuration of a learning device according to a first embodiment. [Figure 7] 4 is a flowchart showing a feature extraction model learning method performed by the learning device according to the first embodiment. [Figure 8] 4 is a first flowchart showing a method for learning a detection model by the learning device according to the first embodiment. [Figure 9] 5 is a second flowchart showing the method for learning a detection model by the learning device according to the first embodiment. [Figure 10] FIG. 1 is a schematic block diagram showing a basic configuration of a learning data generating device. [Figure 11] 1 is a schematic block diagram showing a basic configuration of an authentication system. [Figure 12] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] First embodiment <Authentication system configuration> Hereinafter, the embodiments will be described in detail with reference to the drawings. FIG. 1 is a schematic diagram showing the configuration of an authentication system 1 according to the first embodiment. The authentication system 1 according to the first embodiment authenticates a person by comparing a facial image obtained by capturing an image of the person with a facial image recorded in a registration certificate R. The facial image of the person is an example of biometric data. The facial image is also an example of individual data.

[0013] The registration certificate R is a recording medium on which an authentication face image used to authenticate a person and the account data of the person are recorded in association with each other. The account data is data that can identify a person, such as the person's name or ID. For example, the registration certificate R may be an IC card or a magnetic stripe card that electronically or magnetically stores the authentication face image and the account data. The registration certificate R may also be a card on which the authentication face image and the account data are printed.

[0014] The authentication system 1 includes an issuing device 11, an authentication device 13, and a learning device 15.

[0015] The issuing device 11 is provided, for example, in an issuing office of the registration certificate R. The issuing device 11 accepts the submission of an authentication face image from a person who wishes to receive the issuance of the registration certificate R, and issues the registration certificate R on which the submitted authentication face image and account data are recorded. The issuing device 11 compares the submitted authentication face image with the face image of the person, and issues the registration certificate R only when the comparison is successful.

[0016] The authentication device 13 is provided at the entrance of a facility that permits entry to those who have a legitimate registration card R. The authentication device 13 captures a facial image of a person attempting to enter the facility and compares it with an authentication facial image recorded on the registration card R. If the comparison is successful, the authentication device 13 determines whether or not the authentication facial image on the registration card R is an adversarial sample. If the authentication device 13 succeeds in comparing the facial image with the authentication facial image and the authentication facial image is not an adversarial sample, it permits the person to enter the facility. The authentication device 13 has a function as a determination device that determines whether or not biometric data is an adversarial sample.

[0017] The learning device 15 learns parameters of a feature extraction model used for matching face images in the issuing device 11 and the authentication device 13. The issuing device 11 and the authentication device 13 match face images by comparing features extracted from face images by the feature extraction model. The learning device 15 also learns parameters of a detection model used in the authentication device 13 to determine whether an image is an adversarial sample or not.

[0018] Configuration of issuing device 11 The issuing device 11 according to the first embodiment includes a computer 111, an imaging device 112, and an issuing machine 113. The imaging device 112 is installed so as to capture an image of a person positioned in front of a counter in an issuing office. The imaging device 112 is provided, for example, on a counter in the issuing office. The issuing machine 113 outputs a registration certificate R on which predetermined information is recorded. The issuing machine 113 is, for example, an IC card writer or a printer.

[0019] FIG. 2 is a schematic diagram showing a computer configuration of the issuing device 11 according to the first embodiment. The computer 111 includes a submitted information acquisition unit 121 , an imaging unit 122 , a model storage unit 123 , a feature amount calculation unit 124 , a matching unit 125 , and an issuing unit 126 .

[0020] The submitted information acquisition unit 121 acquires account data and a facial image submitted by a person who wishes to receive the issuance of the registration certificate R. Hereinafter, the facial image submitted by the person will be referred to as a submitted facial image. The submitted information acquisition unit 121 is an example of an acquisition unit that acquires biometric data submitted by a person.

[0021] The imaging unit 122 captures a face image of a person who wishes to register account data, using the imaging device 112 connected to the issuing device 11. Hereinafter, the face image captured by the imaging unit 122 of the issuing device 11 will be referred to as a first captured face image. The imaging unit 122 is an example of a generating unit that generates biometric data from a person.

[0022] The model storage unit 123 stores a feature extraction model used for calculation by the feature calculation unit 124. The feature extraction model receives face image data as an input and outputs features of the face image data. The feature extraction model is a trained model in which parameters have been trained by the learning device 15.

[0023] The feature amount calculation unit 124 calculates the feature amount of the submitted facial image and the feature amount of the first captured facial image. The feature amount calculation unit 124 calculates the feature amount of the submitted facial image by inputting the submitted facial image into the feature amount extraction model stored in the model storage unit 123. The feature amount calculation unit 124 calculates the feature amount of the first captured facial image by inputting the first captured facial image into the feature amount extraction model stored in the model storage unit 123.

[0024] The matching unit 125 calculates the similarity between the feature amount of the submitted face image calculated by the feature amount calculation unit 124 and the feature amount of the first captured face image, and judges whether or not the person appearing in the two face images is the same person. Specifically, the matching unit 125 calculates the distance (e.g., L2 distance) between the feature amount of the submitted face image and the feature amount of the first captured face image, and judges whether or not the person appearing in the two face images is the same person if the distance is equal to or less than a predetermined threshold. Note that the matching unit 125 may calculate a similarity (e.g., cosine similarity) instead of the distance. Judging that the distance is equal to or less than a predetermined threshold is synonymous with determining that the similarity is equal to or more than a predetermined threshold.

[0025] When the comparison unit 125 determines that the person appearing in the submitted face image and the person appearing in the first captured face image are the same person, the issuing unit 126 causes the issuing machine 113 to issue a registration certificate R on which the account data and face image accepted by the submitted information acquisition unit 121 are recorded. In other words, the issuing unit 126 records the account data and the face image in the registration certificate R.

[0026] <<Operation of issuing device 11>> FIG. 3 is a flowchart showing the operation of the issuing device 11 according to the first embodiment. A person who wishes to issue a registration certificate R submits their account data and a facial image they have brought with them (submitted facial image) to a receptionist at the issuing office of the registration certificate R. The receptionist inputs the submitted account data and submitted facial image into the issuing device 11. The submitted facial image may be submitted as data or as a printout. If the submitted facial image is submitted as a printout, the receptionist digitizes the submitted facial image, for example, using a scanner.

[0027] The submitted information acquisition unit 121 of the issuing device 11 reads the input account data and the submitted face image (step S1). Next, the imaging unit 122 captures a face image of the issuing requester using the imaging device 112, and generates a first captured face image (step S2).

[0028] The feature amount calculation unit 124 inputs the submitted face image acquired in step S1 into the feature amount extraction model stored in the model storage unit 123, and calculates the feature amount of the submitted face image (step S3). The feature amount calculation unit 124 also inputs the first captured face image acquired in step S2 into the feature amount extraction model, and calculates the feature amount of the first captured face image (step S4). The matching unit 125 calculates the distance between the feature amount of the submitted face image calculated in step S3 and the feature amount of the first captured face image calculated in step S4 (step S5). The matching unit 125 determines whether the calculated distance is equal to or smaller than a predetermined threshold value (step S6).

[0029] If the distance is less than or equal to the threshold (step S6: YES), the issuing unit 126 outputs the account data and the submitted facial image acquired in step S1 to the issuing machine 113, and causes the issuing machine 113 to issue a registration certificate R in which the account data and the submitted facial image are recorded (step S7). If the distance exceeds the threshold value (step S6: NO), the matching unit 125 outputs a message on the display indicating that matching of the face image has failed, and does not issue the registration certificate R (step S8).

[0030] Thereby, the issuing device 11 issues the registration certificate R when it is confirmed that the submitted face image is the face image of the person who desires to issue the certificate.

[0031] Configuration of authentication device 13 FIG. 4 is a schematic diagram showing the configuration of the authentication device 13 according to the first embodiment. The authentication device 13 according to the first embodiment includes a computer 131, an imaging device 132, a reading device 133, and a gate 134. The imaging device 132 is installed so as to capture an image of a person positioned in front of the counter of the issuing office. The reading device 133 reads data recorded on the registration certificate R. The reading device 133 may be, for example, an IC card reader or a camera. The reading device 133 is provided on the entrance side of the gate 134. The gate 134 opens when authentication by the computer 131 is successful, and closes when authentication is unsuccessful.

[0032] The computer 131 includes a registration certificate reading unit 141 , an imaging unit 142 , a model storage unit 143 , a feature amount calculation unit 144 , a matching unit 145 , an AX determination unit 146 , and a gate control unit 147 .

[0033] The certificate reading unit 141 acquires the account data and the facial image recorded in the certificate R via the reading device 133. The facial image recorded in the certificate R is called an authentication facial image. The imaging unit 142 captures a face image of the person who presents the registration certificate R by the imaging device 132 connected to the authentication device 13. Hereinafter, the face image captured by the imaging unit 142 of the authentication device 13 will be referred to as a second captured face image. The imaging unit 142 is an example of a generation unit that generates biometric data from a person.

[0034] The model storage unit 143 stores a feature extraction model used in calculations by the feature calculation unit 144 and a detection model used in calculations by the AX determination unit 146. The feature extraction model is the same as the feature extraction model stored in the issuing device 11. The detection model receives face image data as input, and outputs a determination result indicating the probability that the face image data is an adversarial sample. The detection model receives face image data as input, and outputs the probability that the face image data is an adversarial sample. The detection model is a trained model whose parameters have been trained by the learning device 15.

[0035] The feature amount calculation unit 144 calculates the feature amount of the authentication face image and the feature amount of the second captured face image. The feature amount calculation unit 144 calculates the feature amount of the authentication face image by inputting the authentication face image to the feature amount extraction model stored in the model storage unit 143. The feature amount calculation unit 144 calculates the feature amount of the second captured face image by inputting the second captured face image to the feature amount extraction model stored in the model storage unit 143.

[0036] The matching unit 145 calculates the similarity between the feature amount of the authentication face image calculated by the feature amount calculation unit 144 and the feature amount of the second captured face image, and determines whether the person appearing in the two face images is the same person.

[0037] The AX determination unit 146 determines whether or not the authentication face image is an adversarial sample by inputting the authentication face image to the detection model stored in the model storage unit 143. Specifically, the AX determination unit 146 determines that the submitted face image is an adversarial sample when the probability that the submitted face image is an adversarial sample calculated by the detection model is equal to or greater than a predetermined threshold.

[0038] The gate control unit 147 outputs a control signal to open the gate 134 when the matching unit 145 determines that the person appearing in the authentication face image and the second captured face image is the same person and the AX determination unit 146 determines that the authentication face image is not a hostile sample.

[0039] <<Operation of the authentication device 13>> FIG. 5 is a flowchart showing the operation of the authentication device 13 according to the first embodiment. A person wishing to enter the facility holds the registration certificate R issued by the issuing device 11 over the reading device 133. The registration certificate reading unit 141 of the authentication device 13 acquires an authentication face image recorded in the registration certificate R via the reading device 133 (step S21). Next, the imaging unit 142 captures a face image of the person wishing to enter the facility using the imaging device 132, and generates a second captured face image (step S22).

[0040] The feature amount calculation unit 144 inputs the authentication face image acquired in step S21 into the feature amount extraction model stored in the model storage unit 143, and calculates the feature amount of the authentication face image (step S23). The feature amount calculation unit 144 also inputs the second captured face image acquired in step S22 into the feature amount extraction model, and calculates the feature amount of the second captured face image (step S24). The matching unit 145 calculates the distance between the feature amount of the authentication face image calculated in step S23 and the feature amount of the second captured face image calculated in step S24 (step S25). The matching unit 145 determines whether the calculated distance is equal to or smaller than a predetermined threshold value (step S26).

[0041] If the distance is equal to or less than the threshold (step S26: YES), the AX determination unit 146 inputs the authentication face image acquired in step S21 into the detection model stored in the model storage unit 143, and calculates the probability that the authentication face image is an adversarial sample (step S27). The AX determination unit 146 determines whether the probability that the authentication face image is an adversarial sample is equal to or less than the threshold (step S28). If the probability is equal to or less than the threshold (step S28: YES), the gate control unit 147 outputs a control signal to the gate 134 to instruct it to be in an open state (step S29).

[0042] On the other hand, if the distance exceeds the threshold in step S25 (step S26: NO), and if the probability exceeds the threshold in step S28 (step S28: NO), the gate control unit 147 outputs a control signal to instruct the gate 134 to maintain the closed state (step S30).

[0043] This allows the authentication device 13 to permit entry to the facility when the person appearing in the authentication face image is the same person as the person seeking entry and when the authentication face image is not a hostile sample.

[0044] Configuration of learning device 15 FIG. 6 is a schematic block diagram showing the configuration of a learning device 15 according to the first embodiment. The learning device 15 includes a model memory unit 152, a dataset acquisition unit 151, a classification processing unit 153, a first update unit 154, a model extraction unit 155, an AX generation unit 156, an AX verification unit 157, a judgment processing unit 158, a second update unit 159, and an output unit 160.

[0045] The model storage unit 152 stores a classification model configured by a neural network and a detection model.

[0046] The classification model has an input layer, an intermediate layer, and an output layer. The input layer outputs input face image data as a vector to the intermediate layer. The intermediate layer is composed of a neural network with two or more layers. The intermediate layer may be composed of a CNN (Convolutional Neural Network). The intermediate layer converts the vector input from the input layer into a low-dimensional feature vector and outputs it to the output layer. The feature vector is a vector that represents a feature amount. The output layer converts the feature vector input from the intermediate layer into a P-dimensional vector that indicates the probability of the person represented by the feature vector.

[0047] The detection model includes an input layer, an intermediate layer, and an output layer. The input layer outputs input face image data as a vector to the intermediate layer. The intermediate layer is composed of a neural network with two or more layers. The intermediate layer may be composed of a CNN. The intermediate layer converts the vector input from the input layer into a low-dimensional feature vector and outputs it to the output layer. The output layer converts the feature vector input from the intermediate layer into a two-dimensional vector indicating the probability that the image represented by the feature vector is an adversarial sample or not.

[0048] The dataset acquisition unit 151 acquires a learning dataset in which face image data, which is an input sample, is associated with person labels, which are output samples. The person labels are represented by a P-dimensional one-hot vector, where P is the number of people in the dataset.

[0049] The classification processing unit 153 inputs input data to a classification model to calculate an output vector.

[0050] The first update unit 154 uses the learning dataset acquired by the dataset acquisition unit 151 to train the parameters of the classification model so that when face image data is input to the classification model, a P-dimensional vector indicating the probability that the person corresponds to the face image data is output.

[0051] The model extraction unit 155 extracts a partial model consisting of an input layer and an intermediate layer from the classification model trained by the first update unit 154 as a feature extraction model, and records it in the model storage unit 152.

[0052] The AX generating unit 156 calculates a perturbation vector for generating an adversarial sample based on the classification model learned by the first update unit 154. The perturbation vector is a vector for making the element having the largest value among the vectors output by the classification model different by adding it to the input data. The method for generating the perturbation vector can be, for example, the IFTGSM method shown in Reference 1 (Yaoyao Zhong, Weihong Deng, “Adversarial Learning with Margin-based Triplet Embedding Regularization”, ICCV 2019) or the PGD Attack method shown in Reference 2 (Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu, “Towards Deep Learning Models Resistant to Adversarial Attacks”, ICLR 2018). The AX generating unit 156 generates an adversarial sample by adding a perturbation to the input sample.

[0053] The AX verification unit 157 verifies whether the adversarial sample generated by the AX generation unit 156 is similar to both the input sample and other face image data. That is, the AX verification unit 157 outputs, from among the generated adversarial samples, those that can pass both the issuing device 11 and the authentication device 13. Specifically, the AX verification unit 157 calculates features for the adversarial sample and multiple input samples related to the learning dataset using a feature extraction model, and calculates the similarity between the adversarial sample and each input sample.

[0054] The determination processing unit 158 ​​inputs input data to the detection model, and calculates an output vector.

[0055] The second update unit 159 uses the input vectors of the learning dataset acquired by the dataset acquisition unit 151 and the adversarial samples output by the AX verification unit 157 to learn the parameters of the detection model so that when facial image data is input into the detection model, a two-dimensional vector indicating the probability of whether the facial image data is an adversarial sample or not is output. The output unit 160 outputs the feature extraction model extracted by the model extraction unit 155 to the issuing device 11. The output unit 160 also outputs the feature extraction model extracted by the model extraction unit 155 and the detection model learned by the second update unit 159 to the authentication device 13. The output may be performed by communication via a network or the like, or may be performed via a portable storage device.

[0056] 《Learning Method》 FIG. 7 is a flowchart showing a feature extraction model learning method performed by the learning device 15 according to the first embodiment. The dataset acquisition unit 151 of the learning device 15 acquires a dataset prepared in advance from a database (not shown) (step S41). The classification processing unit 153 inputs a plurality of input samples included in the acquired dataset to the classification model stored in the model storage unit 152, respectively, and calculates an output vector (step S42). The first update unit 154 calculates a loss function indicating a cross-entropy error based on the output sample corresponding to the input sample and the output vector calculated in step S42 (step S43). The first update unit 154 updates the parameters of the classification model stored in the model storage unit 152 so that the calculated loss function L becomes smaller (step S44).

[0057] The first update unit 154 determines whether the parameter update process of the classification model has been performed a predetermined number of epochs (step S45). If the number of times the parameter update process of the classification model has been performed is less than the predetermined number of epochs (step S45: NO), the learning device 15 returns the process to step S42 and repeats the parameter update process. On the other hand, if the number of times the parameter update process of the classification model has been executed reaches the predetermined number of epochs (step S45: YES), the model extraction unit 155 extracts a partial model consisting of an input layer and an intermediate layer from the classification model stored in the model storage unit 152 as a feature extraction model (step S46). The model extraction unit 155 records the extracted feature extraction model in the model storage unit 152 (step S47). Then, the learning device 15 ends the learning process of the feature extraction model.

[0058] Fig. 8 is a first flowchart showing a method for learning a detection model by the learning device 15 according to the first embodiment. Fig. 9 is a second flowchart showing a method for learning a detection model by the learning device 15 according to the first embodiment. The data set acquisition unit 151 of the learning device 15 acquires a data set prepared in advance from a database (not shown) (step S61). The AX verification unit 157 inputs a plurality of input samples related to the acquired data set into a feature extraction model stored in the model storage unit 152, thereby calculating the feature of each of the plurality of input samples (step S62).

[0059] The AX generation unit 156 generates a plurality of adversarial samples based on a plurality of input samples related to the data set acquired in step S61 (step S63). The AX verification unit 157 selects the plurality of adversarial samples generated in step S63 one by one (step S64) and executes the following processes from step S65 to step S70.

[0060] The AX verification unit 157 calculates the features of the adversarial sample (step S65) by inputting the adversarial sample selected in step S64 into the feature extraction model stored in the model storage unit 152. The AX verification unit 157 calculates the distance between the feature of the adversarial sample calculated in step S65 and each of the features of the multiple input samples calculated in step S62 (step S66).

[0061] Based on the calculation result of step S66, the AX verification unit 157 determines whether the distance between the feature of the adversarial sample selected in step S64 and the feature of the input sample used to generate the adversarial sample is equal to or less than a predetermined threshold (step S67). If the distance between the feature of the selected adversarial sample and the feature of the input sample used to generate the adversarial sample is equal to or less than the threshold (step S67: YES), the AX verification unit 157 extracts, from among the multiple input samples, multiple stranger samples that are input samples associated with labels different from the input sample used to generate the adversarial sample selected in step S64 (step S68). The AX verification unit 157 determines whether any of the multiple stranger samples has a distance between the feature of the adversarial sample selected in step S64 and the feature of the input sample that is equal to or less than a predetermined threshold (step S69). If there is at least one stranger sample whose distance from the feature of the selected adversarial sample is less than or equal to the threshold (step S69: YES), the AX verification unit 157 outputs the adversarial sample selected in step S64 as learning data for the detection model (step S70).

[0062] On the other hand, if the distance between the features of the selected adversarial sample and the features of the input sample used for generation exceeds the threshold (step S67: NO), or if there is no stranger sample whose distance from the features of the selected adversarial sample is less than or equal to the threshold (step S69: NO), the AX verification unit 157 decides not to use the adversarial sample selected in step S64 as training data for the detection model.

[0063] Next, the classification processing unit 153 inputs the multiple input samples included in the data set acquired in step S61 and the adversarial sample output in step S70 into the detection model stored in the model storage unit 152, and calculates an output vector (step S71). The second update unit 159 calculates a loss function indicating a cross-entropy error based on the output vector calculated in step S2 and a one-hot vector indicating whether the input data is an adversarial sample or not (step S72). The second update unit 159 updates the parameters of the detection model stored in the model storage unit 152 so that the calculated loss function becomes smaller (step S73).

[0064] The second update unit 159 determines whether the parameter update process of the detection model has been performed a predetermined number of epochs (step S74). If the number of times the parameter update process of the detection model has been performed is less than the predetermined number of epochs (step S74: NO), the learning device 15 returns the process to step S63 and repeats the parameter update process. On the other hand, if the number of times the detection model parameter update process has been performed reaches the predetermined number of epochs (step S74: YES), the learning device 15 ends the detection model learning process.

[0065] Then, the output unit 160 outputs the feature extraction model stored in the model storage unit 152 to the issuing device 11 and the authentication device 13. The output unit 160 also outputs the detection model stored in the model storage unit 152 to the authentication device 13.

[0066] Actions and Effects In this way, the authentication system 1 according to the first embodiment extracts and uses data similar to both the person related to the image data from which the adversarial sample was generated and other people as learning data for the detection model. The extracted adversarial samples can pass both the matching when issuing the registration certificate R and the matching when authenticating the person, and are in line with the operation of the authentication system 1. As a result, the authentication system 1 according to the first embodiment can generate a detection model that accurately detects adversarial samples during the operation of the authentication system 1.

[0067] Other Embodiments Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes are possible. That is, in other embodiments, the order of the above-mentioned processes may be changed as appropriate. Also, some of the processes may be executed in parallel.

[0068] In the above-described embodiment, the issuing device 11 and the authentication device 13 each extract and compare features, but this is not limited to the above. For example, in the authentication system 1 according to another embodiment, the feature extraction and comparison may be realized by the same server device. For example, the issuing device 11 and the authentication device 13 may transmit two pieces of image data to the authentication system 1 and receive the comparison result calculated by the authentication system 1.

[0069] In the above-described embodiment, the learning device 15 performs the learning process of both the feature extraction model and the detection model, but this is not limited thereto. For example, the authentication system 1 according to another embodiment may perform authentication using a feature extraction model provided by a cloud service. In other words, the configuration of the feature extraction model is a black box for the learning device 15. In this case, the AX generation unit 156 of the learning device 15 generates an adversarial sample using a simulated feature extraction model that simulates the feature extraction model provided by the cloud service. The simulated feature extraction model is generated by, for example, a distillation process using the input and output of the feature extraction model. Then, the AX verification unit 157 of the learning device 15 verifies the adversarial sample using the feature extraction model provided by the cloud service.

[0070] In the above embodiment, the AX verification unit 157 performs matching between the adversarial sample and the stranger sample in step S69, but is not limited to this. For example, if it is known that the adversarial sample generated by the AX generation unit 156 is always similar to the input sample associated with another label, the AX verification unit 157 may omit the judgment in step S69.

[0071] In the above-described embodiment, the learning device 15 performs both extraction of adversarial samples for learning and learning using the adversarial samples, but is not limited to this. For example, the authentication system 1 according to another embodiment may include a learning device 15 and a learning data generation device separately, the learning data generation device generates adversarial samples for learning, and the learning device 15 performs a learning process using the adversarial samples generated by the learning data generation device.

[0072] In addition, in the authentication system 1 according to the above-described embodiment, the issuing device 11 does not determine whether or not a sample is an adversarial sample, and the authentication device 13 determines whether or not a sample is an adversarial sample, but this is not limited to the above. For example, in the authentication system 1 according to another embodiment, the issuing device 11 may determine whether or not a sample is an adversarial sample. In this case, the authentication device 13 does not need to determine whether or not a sample is an adversarial sample. Note that the configuration according to the first embodiment can be applied when the issuing device 11 and the authentication device 13 are operated by different operators and the operator of the authentication device 13 wishes to increase robustness against adversarial samples.

[0073] <Basic configuration> FIG. 10 is a schematic block diagram showing the basic configuration of the learning data generating device 01. As shown in FIG. In the above embodiment, the configuration of the learning device 15 shown in FIG. 6 has been described as one embodiment of the learning data generating device 01, but the basic configuration of the learning data generating device 01 is as shown in FIG. That is, the learning data generating device 01 basically includes a sample acquiring unit 011, an AX generating unit 012, a feature acquiring unit 013, and an output unit 014.

[0074] The sample acquisition unit 011 acquires a first sample, which is individual data relating to a first individual. The sample acquisition unit 011 corresponds to the data set acquisition unit 151. The AX generation unit 012 generates an adversarial sample by applying a perturbation to the first sample. The AX generation unit 012 corresponds to the AX generation unit 156. The feature acquisition unit 013 acquires a first feature which is a feature of an adversarial sample calculated by a trained model that calculates features by performing a calculation using trained parameters on input individual data, a second feature which is a feature of individual data related to a first individual, and a third feature which is a feature of individual data related to a second individual different from the first individual. The feature acquisition unit 013 corresponds to the AX verification unit 157. The output unit 014 outputs an adversarial sample in which the similarity between the first feature amount and the second feature amount is equal to or greater than a threshold value and the similarity between the first feature amount and the third feature amount is equal to or greater than a threshold value. The output unit 014 corresponds to the AX verification unit 157.

[0075] This allows the learning data generation device 01 to supply learning data that can improve the accuracy of detecting hostile samples that pose a threat to the authentication system.

[0076] FIG. 11 is a schematic block diagram showing the basic configuration of the authentication system 02. As shown in FIG. In the above embodiment, the configuration of the authentication system 1 shown in FIG. 1 has been described as one embodiment of the authentication system 02, but the basic configuration of the authentication system 02 is as shown in FIG. That is, the authentication system 02 basically comprises a recording device 021, an authentication device 022, and a determination device 023.

[0077] The recording device 021 records the authentication biometric data and the account data in a storage medium. The recording device 021 corresponds to the issuing device 11. The recording device 021 includes an acquiring unit 0211, a first generating unit 0212, a first calculating unit 0213, and a recording unit 0214.

[0078] The acquiring unit 0211 acquires the first biometric data submitted by the first person. The acquiring unit 0211 corresponds to the submitted information acquiring unit 121. The first generation unit 0212 generates second biometric data from the first person. The first generation unit 0212 corresponds to the imaging unit 122. The first calculation unit 0213 calculates the feature amounts of the first biometric data and the second biometric data by a feature amount extraction model, which is a trained model that calculates feature amounts by performing a calculation using trained parameters on the input individual data. The first calculation unit 0213 corresponds to the feature amount calculation unit 124. The recording unit 0214 records the first biometric data in a storage medium when the calculated similarity between the feature amount of the first biometric data and the feature amount of the second biometric data is equal to or greater than a threshold.

[0079] The authentication device 022 performs person authentication based on the storage medium. The authentication device 022 corresponds to the authentication device 13. The authentication device 022 includes a second generation unit 0221 , a second calculation unit 0222 , and a matching unit 0223 . The second generating unit 0221 generates third biometric data from the second person. The second generating unit 0221 corresponds to the imaging unit 142. The second calculation unit 0222 calculates the feature amount of the first biometric data stored in the storage medium and the feature amount of the third data by using the feature amount extraction model. The second calculation unit 0222 corresponds to the feature amount calculation unit 144. The collation unit 0223 determines whether the calculated similarity is equal to or greater than a threshold value. The collation unit 0223 corresponds to the collation unit 145.

[0080] The determination device 023 determines whether or not the biometric data is an adversarial sample. The determination device 023 corresponds to the authentication device 13. The determination device 023 includes a determination unit 0231. The determination unit 0231 inputs the first biometric data to a trained detection model generated by the learning data generation device 01, thereby determining whether or not the first biometric data is an adversarial sample.

[0081] This enables the authentication system 02 to improve the accuracy of detecting threatening adversarial samples.

[0082] Computer Configuration FIG. 12 is a schematic block diagram illustrating a computer configuration according to at least one embodiment. The computer comprises a processor 91 , a main memory 93 , a storage 95 , and an interface 97 . The learning device 15 described above is implemented in a computer. The operations of each of the above-mentioned processing units are stored in the storage 95 in the form of a program. The processor 91 reads the program from the storage 95, loads it in the main memory 93, and executes the above-mentioned processing in accordance with the program. The processor 91 also secures storage areas in the main memory 93 corresponding to each of the above-mentioned storage units in accordance with the program. Examples of the processor 91 include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a microprocessor.

[0083] The program may be for realizing a part of the functions to be performed by the computer. For example, the program may be for realizing the functions by combining with other programs already stored in the storage or by combining with other programs implemented in other devices. In another embodiment, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to the above configuration or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor 91 may be realized by the integrated circuit. Such an integrated circuit is also included in an example of a processor.

[0084] Examples of storage 95 include a magnetic disk, a magneto-optical disk, an optical disk, a semiconductor memory, etc. Storage 95 may be an internal medium directly connected to a bus of the computer, or an external medium connected to the computer via an interface 97 or a communication line. In addition, when the program is distributed to a computer via a communication line, the computer receiving the program may load the program into main memory 93 and execute the above-mentioned processing. In at least one embodiment, storage 95 is a non-transitory tangible storage medium.

[0085] The program may be for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that realizes the above-mentioned functions in combination with another program already stored in the storage 95. [Explanation of symbols]

[0086] 1...Authentication system 11...Issuing device 111...Computer 112...Imaging device 113...Issuing machine 121...Submission information acquisition unit 122...Imaging unit 123...Model storage unit 124...Feature amount calculation unit 125...Matching unit 126...Issuing unit 13...Authentication device 131...Computer 132...Imaging device 133...Reading device 134...Gate 141...Registration certificate reading unit 142...Imaging unit 143...Model storage unit 144...Feature amount calculation unit 145...Matching unit 146...AX judgment unit 147...Gate control unit 15...Learning device 151...Data set acquisition unit 152...Model storage unit 153...Classification processing unit 154...First update unit 155...Model extraction unit 156...AX generation unit 157...AX verification unit 158...Judgment processing unit 159...Second update unit 160...output unit 91...processor 93...main memory 95...storage 97...interface R...registration certificate

Claims

1. a sample acquisition unit that acquires a first sample, which is individual data relating to a first individual; an AX generation unit that generates an adversarial sample by perturbing the first sample; a feature acquisition unit that acquires a first feature that is a feature of the adversarial sample calculated by a trained model that calculates features by performing a calculation using trained parameters on input individual data, a second feature that is a feature of individual data related to the first individual, and a third feature that is a feature of individual data related to a second individual different from the first individual; an output unit that outputs the adversarial sample in which the similarity between the first feature amount and the second feature amount is equal to or greater than a threshold and the similarity between the first feature amount and the third feature amount is equal to or greater than a threshold; A learning data generating device comprising:

2. The computer Obtaining a first sample which is individual data relating to a first individual; perturbing the first sample to generate an adversarial sample; acquiring a first feature amount, which is a feature amount of the adversarial sample, a second feature amount, which is a feature amount of individual data related to the first individual, and a third feature amount, which is a feature amount of individual data related to a second individual different from the first individual, all of which are calculated by a trained model that calculates features by performing a calculation using trained parameters on input individual data; The hostile sample in which the similarity between the first feature amount and the second feature amount is equal to or greater than a threshold and the similarity between the first feature amount and the third feature amount is equal to or greater than a threshold is output as learning data. How to generate training data.

3. On the computer, obtaining a first sample of individual data relating to a first individual; perturbing the first sample to generate an adversarial sample; a step of acquiring a first feature amount, which is a feature amount of the adversarial sample, a second feature amount, which is a feature amount of individual data related to the first individual, and a third feature amount, which is a feature amount of individual data related to a second individual different from the first individual, calculated by a trained model that calculates features by performing a calculation using trained parameters on input individual data; and a step of outputting the adversarial sample in which the similarity between the first feature amount and the second feature amount is equal to or greater than a threshold value and the similarity between the first feature amount and the third feature amount is equal to or greater than a threshold value. A program for executing.

4. The computer A plurality of adversarial samples are generated as training data by the training data generation method according to claim 2; Using the generated plurality of adversarial samples and a plurality of normal individual data, parameters of a detection model, which is a machine learning model that receives individual data as input and outputs a determination result of whether the individual data is normal individual data or an adversarial sample, are learned. How to generate a detection model.

5. a recording device for recording the authentication biometric data and the account data in a storage medium; an authentication device that performs person authentication based on the storage medium; A determination device for determining whether biometric data is an adversarial sample; An authentication system comprising: The recording device includes: an acquisition unit that acquires first biometric data submitted by a first person; a first generating unit configured to generate second biometric data from the first person; a first calculation unit that calculates features of the first biometric data and features of the second biometric data using a feature extraction model that is a trained model that calculates features by performing a calculation using trained parameters on input individual data; a recording unit that records the first biometric data in the storage medium when a degree of similarity between the calculated feature amount of the first biometric data and the calculated feature amount of the second biometric data is equal to or greater than a threshold value; Equipped with The authentication device includes: a second generation unit that generates third biometric data from the second person; a second calculation unit that calculates a feature amount of the first biometric data stored in the storage medium and a feature amount of the third biometric data by the feature amount extraction model; a matching unit that judges whether the calculated similarity is equal to or greater than a threshold; Equipped with The determination device includes a determination unit that determines whether the first biometric data is an adversarial sample by inputting the first biometric data to a trained detection model generated by the detection model generation method according to claim 4. Authentication system.

Images