Feature transformation learning device, authentication device, feature transformation learning method, authentication method, and program

Abstract

To provide a feature amount conversion learning device, an authentication device, a feature amount conversion learning method, an authentication method, and a recording medium, which perform feature amount super resolution with higher accuracy.SOLUTION: A feature amount conversion learning device 200 includes: an image patch acquisition unit 201 that acquires a first image; an image reduction unit 202 that reduces the first image to a second image having a resolution lower than that of the first image; an image enlargement unit 103 that enlarges the second image to a third image having the same resolution as that of the first image; a feature amount extraction unit 104 that extracts a first feature amount as a feature amount of the first image and a second feature amount as a feature amount of the third image; a feature amount conversion unit 105 that converts the second feature amount into the third feature amount; and a learning control unit 208 that allows the feature amount conversion unit 105 to learn a feature amount conversion method on the basis of a comparison result between the first feature amount and the third feature amount.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] This disclosure relates to a feature quantity conversion learning device, an authentication device, a feature quantity conversion learning method, an authentication method, and program related thereto.

Background Art

[0002] The technology of converting an image into an image with a higher resolution than the original image is called super-resolution (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When performing iris authentication using a face image, for example, the resolution of the image of the authentication target may be insufficient. In this case, it is conceivable to convert the image of the authentication target into an image with a higher resolution by super-resolution and use it for authentication. However, since it is necessary to generate an image with a high resolution, there is a possibility that the super-resolution may require a high computational cost.

[0005] An example of the object of this disclosure is to provide a feature quantity conversion learning device, an authentication device, a feature quantity conversion learning method, an authentication method, and program related thereto that can solve the above-described problems.

Means for Solving the Problems

[0006] According to a first aspect of this disclosure, the feature transformation learning device includes: an image acquisition means for acquiring a first image; an image reduction means for reducing the first image to a second image with a lower resolution than the first image; an image enlargement means for enlarging the second image to a third image with the same resolution as the first image; a feature extraction means for extracting a first feature, which is a feature of the first image, and a second feature, which is a feature of the third image; and a feature transformation means for converting the second feature into a third feature. Using a loss function that minimizes the loss when the class determined by the classification based on the third feature matches the correct class, The system includes a learning control means that causes the feature transformation means to learn a feature transformation method based on the comparison result between the first feature and the third feature. The feature extraction means includes an attention processing means that performs weighting to emphasize features corresponding to specific parts of the image from which features are to be extracted.

[0007] According to a second aspect of this disclosure, the authentication device includes an authentication target image acquisition means for acquiring an authentication target image, an image enlargement means for enlarging the authentication target image, a feature extraction means for extracting the feature quantities of the enlarged image of the authentication target image in the same way as the feature extraction method for a comparison image with a higher resolution than the authentication target image, and a feature of a training image. Quantity and , the feature quantities of the degraded image obtained by scaling down the training image and then enlarging it Features after conversion from The feature extraction means includes a feature transformation means that transforms the features of an enlarged image of the image to be authenticated based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference from the original image decreases, and a feature comparison means that compares the features after transformation by the feature transformation means with the features of the comparison image, and the feature extraction means includes an attention processing means that weights features that correspond to a specific part of the image from which features are to be extracted to emphasize them.

[0008] According to a third aspect of this disclosure, the feature transformation learning method includes acquiring a first image, reducing the first image to a second image with lower resolution than the first image, enlarging the second image to a third image with the same resolution as the first image, extracting a first feature which is a feature of the first image and a second feature which is a feature of the third image, and transforming the second feature into a third feature. Using a loss function that minimizes the loss when the class determined by the classification based on the third feature matches the correct class,This includes learning a feature transformation method that transforms the second feature into the third feature based on the comparison result between the first feature and the third feature. Furthermore, extracting the first and third features includes weighting to emphasize features corresponding to specific parts of the image from which features are to be extracted.

[0009] According to a fourth aspect of this disclosure, the authentication method includes acquiring an image to be authenticated, enlarging the image to be authenticated, extracting the features of the enlarged image of the image to be authenticated using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated, and the features of the training image. Quantity and , the feature quantities of the degraded image obtained by scaling down the training image and then enlarging it Features after conversion from The method includes transforming the features of an enlarged image of the image to be authenticated based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference from the original image decreases, and comparing the transformed features from the enlarged image of the image to be authenticated with the features of the comparison image, wherein the extraction of features from the enlarged image includes weighting to emphasize features corresponding to a specific part of the image from which features are to be extracted.

[0010] According to a fifth aspect of this disclosure, the program causes the computer to perform the following actions: acquire a first image; reduce the first image to a second image with a lower resolution than the first image; enlarge the second image to a third image with the same resolution as the first image; extract a first feature, which is a feature of the first image, and a second feature, which is a feature of the third image; convert the second feature to a third feature; and learn a feature conversion method that converts the second feature to a third feature based on the comparison result between the first and third features, using a loss function that reduces the loss when the class determined by classification based on the third feature matches the correct class; and, in extracting the first and third features, perform weighting to emphasize features corresponding to a specific part of the image from which features are to be extracted. of It is a program.

[0011] According to a sixth aspect of this disclosure, the program acquires an image to be authenticated, enlarges the image to be authenticated, extracts the features of the enlarged image of the image to be authenticated in the same way as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated, and the features of the training image Quantity and , the feature quantities of the degraded image obtained by scaling down the training image and then enlarging it Features after conversion from Based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference from the original decreases, the following steps are performed: transform the features of the enlarged image of the image to be authenticated, and compare the transformed features from the enlarged image of the image to be authenticated with the features of the comparison image. In extracting features from the enlarged image, in order to perform weighting to emphasize features corresponding to a specific part of the image from which features are to be extracted, of It is a program. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example configuration of an authentication device according to the first embodiment. [Figure 2] This flowchart shows an example of the processing procedure for iris authentication performed by the authentication device according to the first embodiment. [Figure 3] This figure shows an example configuration of a feature transformation learning device according to the second embodiment. [Figure 4] This figure shows an example of a processing procedure in which the feature transformation learning device according to the second embodiment performs learning of the feature transformation unit. [Figure 5] This figure shows an example of a processing procedure for calculating loss by the feature transformation learning device according to the second embodiment. [Figure 6] This figure shows an example of the configuration of the feature extraction unit according to the third embodiment. [Figure 7] This figure shows an example of the configuration of the attention processing unit according to the third embodiment. [Figure 8] This figure shows an example of the configuration of an attention block 302 according to a first modified example of the attention processing unit according to the third embodiment. [Figure 9]It is a diagram showing a configuration example of an attention processing unit 303 according to a second modification of the attention processing unit according to the third embodiment. [Figure 10] It is a diagram showing a configuration example of an attention processing unit 304 according to a second modification of the attention processing unit according to the third embodiment. [Figure 11] It is a diagram showing a configuration example of a feature amount extraction unit according to a modification of the feature amount extraction unit in the fourth embodiment. [Figure 12] It is a diagram showing an example of a hierarchical structure of a neural network constituting a feature amount extraction unit according to the fourth embodiment. [Figure 13] It is a diagram showing a configuration example of an authentication device according to the fifth embodiment. [Figure 14] It is a diagram showing a configuration example of a feature amount conversion learning device according to the sixth embodiment. [Figure 15] It is a diagram showing a configuration example of an authentication device according to the seventh embodiment. [Figure 16] It is a diagram showing a configuration example of a feature amount conversion learning device according to the eighth embodiment. [Figure 17] It is a diagram showing a configuration example of an authentication device according to the ninth embodiment. [Figure 18] It is a diagram showing a configuration example of a feature amount conversion learning device according to the tenth embodiment. [Figure 19] It is a diagram showing a configuration example of an authentication device according to the eleventh embodiment. [Figure 20] It is a diagram showing a configuration example of a feature amount conversion learning device according to the twelfth embodiment. [Figure 21] It is a diagram showing a configuration example of a feature amount conversion learning device according to the thirteenth embodiment. [Figure 22] It is a diagram showing a configuration example of an authentication device according to the fourteenth embodiment. [Figure 23] It is a flowchart showing an example of a processing procedure in a feature amount conversion learning method according to the fifteenth embodiment. [Figure 24] It is a flowchart showing an example of a processing procedure in an authentication method according to the sixteenth embodiment. [Figure 25]This is a schematic block diagram showing the configuration of a computer according to at least one embodiment. [Modes for carrying out the invention]

[0013] <First Embodiment> Figure 1 is a diagram showing an example of the configuration of an authentication device according to the first embodiment. In the configuration shown in Figure 1, the authentication device 100 includes an image acquisition unit 101 for acquiring an image to be authenticated, an image acquisition unit 102 for comparison, an image enlargement unit 103, a feature extraction unit 104, a feature conversion unit 105, and a feature comparison unit 106.

[0014] The authentication device 100 extracts feature quantities from an image and performs authentication using the extracted feature quantities. The following explanation uses the case where the authentication device 100 performs iris authentication using a low-resolution image as an example. However, the authentication performed by the authentication device 100 is not limited to iris authentication. The resolution comparison between the two images should be performed with the images of the same size.

[0015] The authentication target image acquisition unit 101 acquires an image of the eyes of the person to be authenticated, which is the image to be authenticated. The authentication target image acquisition unit 101 is an example of an authentication target image acquisition means. The comparison image acquisition unit 102 acquires a comparison image to be used for comparison with the image to be authenticated. The comparison image may be registered in advance in the authentication device 100.

[0016] Here, we assume that the image to be authenticated has a lower resolution than the comparison image. For example, the image to be authenticated may be an image of the eyes from a full-face image of the person being authenticated. On the other hand, the comparison image may be an image obtained by taking a photograph of only the eyes for iris authentication. In this way, if iris authentication can be performed using an image of the eyes from a full-face image, it becomes possible to improve authentication accuracy by performing both facial authentication and iris authentication.

[0017] The image enlargement unit 103 enlarges the low-resolution image of the image to be authenticated. The image enlargement unit 103 is an example of an image enlargement means. The enlarged image of the image to be authenticated is also referred to as the enlarged image of the image to be authenticated, or simply the enlarged image. The feature extraction unit 104 extracts the features of the image to be authenticated using the same method as the feature extraction method for the comparison image.

[0018] The feature extraction unit 104 extracts the features of the enlarged image to be authenticated and the comparison image. The feature extraction unit 104 applies the same algorithm to the enlarged image to be authenticated and the comparison image to extract their respective features. The authentication device 100 may be equipped with two identical feature extraction units 104. Alternatively, one feature extraction unit 104 may extract the features of the enlarged image to be authenticated and the comparison image using, for example, time-division processing. Furthermore, the features of the comparison image may be registered in advance in the authentication device 100, for example, by having the feature extraction unit 104 extract the features of the comparison image beforehand. The feature extraction unit 104 is an example of a feature extraction means.

[0019] The feature transformation unit 105 transforms the features of the enlarged image to be authenticated. The feature transformation unit 105 performs feature transformation so that the difference between the features of the enlarged image to be authenticated and the high-resolution features is minimized. The feature transformation unit 105 is an example of a feature transformation means.

[0020] The transformation of features by the feature transformation unit 105 is also called feature super-resolution. The features after transformation by the feature transformation unit 105 are also called super-resolution features. The feature transformation unit 105 learns a method for transforming features by learning using a loss function that decreases as the difference between the features of the training image and the features of the degraded image (which is obtained by scaling down the training image and then scaling it up) decreases.

[0021] The feature comparison unit 106 compares the feature quantities converted by the feature conversion unit 105 with the feature quantities of the comparison image to determine, for example, whether the person in the authentication target image and the person in the comparison image are the same person. The feature comparison unit 106 is an example of a feature comparison means.

[0022] Figure 2 is a flowchart showing an example of the processing procedure by which the authentication device 100 performs iris authentication. In the process shown in Figure 2, the authentication target image acquisition unit 101 acquires the authentication target image (step S111).

[0023] Next, the image enlargement unit 103 enlarges the image to be authenticated by interpolating the pixels of the image to be authenticated, which is a low-resolution image, so that it has the same resolution as the comparison image, which is a high-resolution image (step S112). The algorithm used by the image enlargement unit 103 to enlarge the image is not limited to a specific algorithm. For example, the image enlargement unit 103 may use the bicubic method or the bilinear method to enlarge the image, but is not limited to these. The image enlarged by the image enlargement unit 103 is also called the enlarged image.

[0024] Next, the feature extraction unit 104 extracts features from the enlarged image (step S113). The feature extraction unit 104 may, but is not limited to, a feature extractor trained for authentication using high-resolution images. The feature extraction unit 104 may be configured using a trained model of a deep neural network (DNN) such as VGG or ResNet (Residual Network). The feature extraction unit 104 may output features in the form of a one-dimensional vector. Alternatively, the feature extraction unit 104 may output features in the form of a two-dimensional or three-dimensional tensor that preserves the image information.

[0025] Next, the feature transformation unit 105 transforms the features of the enlarged image (step S114). The feature transformation unit 105 transforms the features of the enlarged image so that they can be compared with the features of the high-resolution image.

[0026] Furthermore, the comparison image acquisition unit 102 acquires a comparison image (step S121). Then, the feature extraction unit 104 extracts features from the comparison image (step S122). As described above, the feature extraction unit 104 extracts features from the comparison image using the same algorithm as when extracting features from the enlarged image. Steps S111 to S114 and steps S121 to S122 may be executed in parallel. Alternatively, the processing from steps S121 to S122 may be performed in advance, and the authentication device 100 may store the feature quantities of the comparison image.

[0027] After steps S114 and S122, the feature comparison unit 106 compares the features of the enlarged image with the features of the comparison image (step S131). For example, the feature comparison unit 106 calculates the similarity between the features of the enlarged image and the features of the comparison image, and determines that the person in the authentication target image and the person in the comparison image are the same person if the similarity is greater than or equal to a predetermined threshold. The method used by the feature comparison unit 106 to calculate the similarity can be any of the methods that can calculate the similarity of vectors or tensors, and is not limited to a specific method. For example, the feature comparison unit 106 may calculate the L2 distance or cosine similarity between the features of the enlarged image and the features of the comparison image. After step S131, the authentication device 100 terminates the process shown in Figure 2.

[0028] As described above, the authentication target image acquisition unit 101 acquires the authentication target image. The image enlargement unit 103 enlarges the authentication target image. The feature extraction unit 104 extracts the features of the enlarged image of the authentication target image using the same method as the feature extraction method for a comparison image with a higher resolution than the authentication target image. The feature transformation unit 105 transforms the features of the enlarged image of the authentication target image based on learning using a loss function that becomes smaller the smaller the difference between the features of the training image and the features of the degraded image obtained by enlarging the training image after it has been reduced. The feature comparison unit 106 compares the features after transformation by the feature transformation unit 105 with the features of the comparison image.

[0029] According to the authentication device 100, by enlarging a low-resolution image, extracting features, and generating super-resolution features, it is possible to obtain features for the low-resolution image to be authenticated that are comparable to the features of the high-resolution image used for comparison. In this respect, the authentication device 100 can perform authentication even if the input image is a low-resolution image.

[0030] Furthermore, it is expected that the feature transformation performed by the feature transformation unit 105 will bring the features of the enlarged image to be authenticated closer to those of the image to be authenticated if it were captured at a higher resolution. In this respect, the authentication device 100 is expected to be able to perform authentication with high accuracy. Furthermore, the authentication device 100 can perform iris authentication using low-resolution images. This means that the authentication device 100 can perform both facial authentication and iris authentication using a single facial image without requiring a high-resolution camera, and is expected to improve authentication accuracy.

[0031] <Second Embodiment> Figure 3 shows an example of the configuration of a feature transformation learning device according to the second embodiment. In the configuration shown in Figure 3, the feature transformation learning device 200 comprises an image batch acquisition unit 201, an image reduction unit 202, an image enlargement unit 103, a feature extraction unit 104, a feature transformation unit 105, a loss function calculation unit 203, and a learning control unit 208. The loss function calculation unit 203 comprises a reconstruction loss calculation unit 204, a feature discrimination unit 205, an adversarial loss calculation unit 206, and a similarity loss calculation unit 207. In Figure 3, parts that have the same function as the parts in Figure 1 are denoted by the same reference numerals (103, 104, 105).

[0032] The feature transformation learning device 200 performs training on the feature transformation unit 105. The trained feature transformation unit 105 can then be used in the authentication device 100. A model that has parameters to be adjusted through learning is called a machine learning model. The parameters of a machine learning model are called machine learning model parameters, and the values ​​of these parameters are called machine learning model parameter values. The adjustment of machine learning model parameter values ​​corresponds to learning. The following explanation will use the case where the machine learning model of the feature transformation unit 105 is configured using a neural network as an example. However, the configuration of the machine learning model of the feature transformation unit 105 is not limited to a specific one.

[0033] Furthermore, the following explanation will use the case where the feature transformation learning device 200 uses a loss function for learning as an example. In this case, the feature transformation learning device 200 performs learning in such a way that the loss function value becomes small. However, the feature transformation learning device 200 may also be configured to perform learning using an evaluation function in which the function value increases as the evaluation level increases. In this case, the feature transformation learning device 200 will perform learning in such a way that the evaluation function value increases.

[0034] The image batch acquisition unit 201 acquires a training dataset used for training the feature transformation unit 105. The training data acquired by the image batch acquisition unit 201 includes multiple combinations of a captured image of a human eye and a class label indicating the correct class in the classification of that image. The training dataset is also simply referred to as training data. Each combination of one image and one class label included in the training dataset is also referred to as a labeled image. In the second embodiment, the images acquired by the image batch acquisition unit 201 are also referred to as high-resolution images. The image batch acquisition unit 201 is an example of an image acquisition means. The high-resolution image acquired by the image batch acquisition unit 201 is an example of the first image.

[0035] The image reduction unit 202 reduces the resolution of the high-resolution image. The reduction performed by the image reduction unit 202 results in the removal of pixels, resulting in a lower resolution image than the original high-resolution image. The image reduced by the image reduction unit 202 is referred to as the reduced image. The image reduction unit 202 may also use random numbers to randomly determine the size or reduction ratio of the reduced image and perform image reduction. The image reduction unit 202 is an example of an image reduction means. The reduced image is an example of a second image.

[0036] In the feature transformation learning device 200, the image enlargement unit 103 enlarges the image by the same reduction ratio as the image reduction unit 202. The image enlargement unit 103 enlarges the reduced image by interpolating the pixels of the reduced image so that the enlarged image of the reduced image, which is a low-resolution image, has the same resolution as the high-resolution image. The image obtained by the image enlargement unit 103 to enlarge the reduced image is also called a degraded image. As described above, the image enlargement unit 103 is an example of an image enlargement means. The degraded image is an example of a third image.

[0037] In the feature transformation learning device 200, the feature extraction unit 104 extracts features from high-resolution images and features from degraded images using the same algorithm. As described above, the feature extraction unit 104 is an example of a feature extraction means. In the feature transformation learning device 200, the feature transformation unit 105 transforms the features of the degraded image. The feature transformation unit 105 transforms the features of the degraded image to approximate the features of the high-resolution image. As in the first embodiment, the features transformed by the feature transformation unit 105 are also called super-resolution features. As described above, the feature transformation unit 105 is an example of a feature transformation means.

[0038] The loss function calculation unit 203 calculates the loss function for training the feature transformation unit 105. The loss function calculation unit 203 is an example of a loss function calculation means. The reconstruction loss calculation unit 204 calculates the reconstruction loss. The reconstruction loss is a loss whose value decreases as the similarity between the high-resolution image features and the super-resolution features as vectors or tensors increases. The reconstruction loss corresponds to an example of an index value of the similarity between the high-resolution image features and the super-resolution features, according to the loss function calculation unit 203. The total loss function, which uses the reconstruction loss as one of its variables, corresponds to an example of a loss function whose loss decreases as the similarity between the high-resolution image features and the super-resolution features increases.

[0039] The feature discrimination unit 205 distinguishes between high-resolution image features and super-resolution features. Specifically, the feature discrimination unit 205 receives feature input and determines whether the input feature is a high-resolution image feature or a super-resolution feature. The feature discrimination unit 205 is also configured to be trainable. For example, the machine learning model of the feature discrimination unit 205 may be configured using a neural network. The feature discrimination unit 205 is an example of a feature discrimination means.

[0040] The adversarial loss calculation unit 206 calculates the learning loss for the feature transformation unit 105 and the learning loss for the feature transformation unit 205 based on the discrimination result (determination result) of the feature discrimination unit 205. The learning loss for the feature transformation unit 105 is also referred to as the loss of the feature transformation unit 105. The learning loss for the feature discrimination unit 205 is also referred to as the loss of the feature discrimination unit 205. The learning of the feature transformation unit 105 and the learning of the feature discrimination unit 205 are performed alternately in a single batch of learning, according to the control of the learning control unit 208.

[0041] During the training of the feature transformation unit 105, the adversarial loss calculation unit 206 calculates the adversarial loss. The adversarial loss here is a loss whose value decreases when the feature discrimination unit 205 misclassifies super-resolution features as features of a high-resolution image. A total loss function that uses the adversarial loss as one of its variables is an example of a loss function in which the loss decreases when the feature discrimination unit 205 cannot distinguish between features transformed by the feature transformation unit 105 and other features. On the other hand, during the training of the feature discrimination unit 205, the adversarial loss calculation unit 206 calculates a loss that becomes smaller when the feature discrimination unit 205 correctly distinguishes between super-resolution features and high-resolution image features.

[0042] The similarity loss calculation unit 207 calculates the similarity loss. The similarity loss is a loss that decreases in class classification using super-resolution features when the class of the classification result matches the ground truth class. The total loss function that uses the similarity loss as one of its variables is an example of a loss function in which the loss decreases when the class determined by class classification based on super-resolution features matches the ground truth class.

[0043] The learning control unit 208 updates the machine learning model parameter values ​​of the feature transformation unit 105 and the feature discrimination unit 205, respectively, using the loss calculated by the loss function calculation unit 203. The learning control unit 208 is an example of a learning control means. The learning control unit 208 adjusts the machine learning model parameter values ​​for feature transformation by the feature transformation unit 105 using the loss calculated by the loss function calculation unit 203. In this respect, the learning control unit 208 causes the feature transformation unit 105 to learn the feature transformation method based on the loss function. The update of the machine learning model parameter values ​​of the feature transformation unit 105 corresponds to the learning of the feature transformation unit 105. The update of the machine learning model parameter values ​​of the feature discrimination unit 205 corresponds to the learning of the feature discrimination unit 205.

[0044] Figure 4 shows an example of the processing procedure in which the feature transformation learning device 200 performs learning on the feature transformation unit 105. In the process shown in Figure 4, the image batch acquisition unit 201 acquires a training dataset containing multiple combinations of high-resolution images, which are images of a human eye, and class labels that indicate the correct class of that high-resolution image (step S211). As described above, each combination of a single high-resolution image and a single class label included in the training dataset is also referred to as a labeled image. Next, the image batch acquisition unit 201 acquires batch data from the training dataset (step S212). For example, the image batch acquisition unit 201 randomly selects a predetermined batch size number of labeled images from among the labeled images included in the training dataset. The batch data acquired by the image batch acquisition unit 201 is also referred to as image batch data. The batch size of the image batch data acquired by the image batch acquisition unit 201 is not limited to a specific size. For example, the image batch acquisition unit 201 may acquire a training dataset containing 128 labeled images, but is not limited to this.

[0045] Next, the feature transformation learning device 200 starts a loop L11 that processes each labeled image included in the image batch data (step S213). The feature transformation learning device 200 may execute the processing of loop L11 in parallel or sequentially.

[0046] In the loop L11, the image reduction unit 202 reduces the resolution of the high-resolution labeled image (step S221). The image reduction unit 202 may also randomly determine the size of the reduced image and perform the image reduction. The image reduction unit 202 reduces the image by downsampling pixels. Therefore, the image resolution decreases due to the image reduction by the image reduction unit 202. The reduced image corresponds to a low-resolution image.

[0047] Next, the image enlargement unit 103 enlarges the reduced image (step S222). The image obtained by the image enlargement unit 103 enlarges the reduced image is also called a degraded image. As described above, the image enlargement unit 103 enlarges the reduced image by interpolating the pixels of the reduced image so that the degraded image has the same resolution as the high-resolution image.

[0048] Next, the feature extraction unit 104 extracts the features of the degraded image (step S223). Next, the feature transformation unit 105 transforms the features of the degraded image (step S224). As described above, the features transformed by the feature transformation unit 105 are also called super-resolution features.

[0049] Furthermore, the feature extraction unit 104 extracts features from the high-resolution image of the labeled image (step S231). Steps S221 to S224 and step S231 may be executed in parallel.

[0050] After steps S224 and S231, the feature transformation learning device 200 performs termination processing of loop L11 (step S241). Specifically, the feature transformation learning device 200 waits for the processing of loop L11 to be completed for all labeled images included in the learning batch. When the feature transformation learning device 200 detects that the processing of loop L11 has been completed for all labeled images included in the learning batch, it terminates loop L11.

[0051] After loop L11 is completed, the feature transformation learning device 200 calculates the loss for learning the feature transformation unit 105 (step S242). Based on the results of processing loop L11 for all labeled images included in the image batch data, the feature transformation learning device 200 calculates a single total loss function value. Figure 5 shows an example of the processing procedure by which the feature transformation learning device 200 calculates the loss. The feature transformation learning device 200 performs the processing shown in Figure 5 in step S242 of Figure 4.

[0052] In the process shown in Figure 5, the feature discrimination unit 205 outputs a vector for determining (discriminating) whether the input feature is a super-resolution feature or a high-resolution image feature (step S251). For example, the feature discrimination unit 205 may calculate the probability that the input feature is a high-resolution image feature and output a vector that includes that probability as an element. This probability takes a real value between 0 and 1, and the type of input image can be determined, for example, by looking closer to 1, indicating a higher probability that it is a high-resolution image feature, and by looking closer to 0, indicating a higher probability that it is a super-resolution feature.

[0053] Next, the adversarial loss calculation unit 206 calculates the loss using the vector output by the feature discrimination unit 205 (step S252). The feature transformation unit 105 and the feature discrimination unit 205 have the structure of a Generative Adversarial Network (GAN), and the adversarial loss calculation unit 206 calculates the adversarial loss using the vector output by the feature discrimination unit 205. The adversarial loss here is the loss whose value decreases when the discriminator makes a wrong judgment.

[0054] During the training of the feature transformation unit 105, the adversarial loss calculation unit 206 calculates the adversarial loss using a loss function that minimizes the adversarial loss when the feature discrimination unit 205 misclassifies the super-resolution features as features of a high-resolution image. The learning control unit 208 adjusts the machine learning model parameter values ​​of the feature transformation unit 105 so that the adversarial loss calculated by the adversarial loss calculation unit 206 is minimized. As a result, the feature transformation unit 105 generates super-resolution features in a way that deceives the feature discrimination unit 205. For example, the feature transformation unit 105 may be trained using cross-entropy loss so that the adversarial loss calculation unit 206 outputs 1 when a super-resolution feature is input.

[0055] Meanwhile, during the training of the feature discrimination unit 205, which is performed alternately with the training of the feature transformation unit 105, the adversarial loss calculation unit 206 calculates the loss using a loss function that minimizes the loss when the feature discrimination unit 205 correctly distinguishes between super-resolution features and high-resolution image features. The learning control unit 208 adjusts the machine learning model parameter values ​​of the feature discrimination unit 205 so that the loss calculated by the adversarial loss calculation unit 206 is minimized. Therefore, the feature discrimination unit 205 is adjusted so that it can correctly distinguish between super-resolution features and high-resolution image features.

[0056] For example, the feature discrimination unit 205 may output the discrimination result as a real number with a value range from 0 to 1. As described above, the feature discrimination unit 205 may output the probability that the input feature is a feature of a high-resolution image. Furthermore, the adversarial loss calculation unit 206 may calculate the adversarial loss using binary cross-entropy, outputting 1 when a feature of a high-resolution image is input and 0 when a super-resolution feature is input.

[0057] If the output of the feature discrimination unit 205 is "0" representing a super-resolution feature and "1" representing a high-resolution feature, the learning control unit 208 adjusts the machine learning model parameter values ​​of the feature transformation unit 105 so that the output of the feature discrimination unit 205 approaches 1 when the input to the feature discrimination unit 205 is a super-resolution feature. As a result, the feature transformation unit 105 generates super-resolution features in a way that deceives the feature discrimination unit 205.

[0058] On the other hand, the learning control unit 208 adjusts the machine learning model parameter values ​​of the feature discrimination unit 205 so that when the input to the feature discrimination unit 205 is a super-resolution feature, the output of the feature discrimination unit 205 approaches 0, and when the input to the feature discrimination unit 205 is a high-resolution image feature, the output of the feature discrimination unit 205 approaches 1. In other words, the feature discrimination unit 205 is adjusted to increase the likelihood of correctly distinguishing between super-resolution features and high-resolution image features.

[0059] Furthermore, the reconstruction loss calculation unit 204 calculates a reconstruction loss that decreases as the similarity between the super-resolution features and the high-resolution image features as vectors or tensors increases (step S261). The learning control unit 208 adjusts the machine learning model parameter values ​​of the feature transformation unit 105 so that the reconstruction loss is reduced. In this way, the learning control unit 208 adjusts the machine learning model parameter values ​​of the feature transformation unit 105 so that the super-resolution features are closer to the features of the high-resolution image.

[0060] The index value representing the similarity between the super-resolution features and the high-resolution image features as a vector or tensor, calculated by the reconstruction loss calculation unit 204, is not limited to a specific value. For example, the reconstruction loss calculation unit 204 may calculate the reconstruction loss in such a way that the smaller the L2 distance between the super-resolution features and the high-resolution image features, the smaller the reconstruction loss. Alternatively, the reconstruction loss calculation unit 204 may calculate the reconstruction loss using the L1 distance instead of the L2 distance.

[0061] Furthermore, the similarity loss calculation unit 207 calculates a similarity loss using class labels. The similarity loss calculated by the similarity loss calculation unit 207 can be any of the following types of losses, and is not limited to any particular one, where the value decreases when the classification result using super-resolution features matches the correct answer.

[0062] For example, a single trainable linear layer may be added after the output layer of the super-resolution features of the feature transformation unit 105, and the output of the linear layer may be one-hot vectors equal to the number of classes. The learning control unit 208 trains the linear layer so that it performs class estimation based on the super-resolution features. As a result, the one-hot vectors output by the linear layer indicate the estimated class.

[0063] Alternatively, the similarity loss calculation unit 207 may input each element of the one-hot vector output by the linear layer into a softmax function and calculate the similarity loss by cross-entropy using the class labels.

[0064] However, the similarity loss used by the similarity loss calculation unit 207 is not limited to a specific one. For example, the similarity loss calculation unit 207 may use L2 softmax loss, cosine loss, ArcFace, CosFace, SphereFace, or AdaCos as the similarity loss.

[0065] Alternatively, instead of adding the single linear layer described above, the similarity loss calculation unit 207 may use a triplet loss, center loss, or contrastive loss as the similarity loss.

[0066] Steps S251 to S252, S261, and step S271 may be executed in parallel. After steps S252, S261, and S271, the loss function calculation unit 203 calculates a total loss function value based on adversarial loss, reconstruction loss, and similarity loss (step S261). The loss function calculation unit 203 calculates the loss function value using a loss function in which the loss function value decreases as the value of adversarial loss decreases, the loss function value decreases as the value of reconstruction loss decreases, and the loss function value decreases as the value of similarity loss decreases. For example, the loss function calculation unit 203 multiplies each loss by a coefficient and sums them up to calculate the total loss function. The values ​​of these coefficients are not particularly limited.

[0067] However, the loss function calculation unit 203 may calculate the loss function value using only one or two of the adversarial loss, reconstruction loss, and similarity loss. Alternatively, the loss function calculation unit 203 may calculate the loss function value using other methods without using any of the adversarial loss, reconstruction loss, and similarity loss. After step S261, the feature transformation learning device 200 completes the process shown in Figure 5.

[0068] After the processing shown in Figure 5 is completed, the learning control unit 208 calculates the gradient of the neural network parameters of the feature transformation unit 105 using backpropagation (step S243). Then, the learning control unit 208 updates the parameter values ​​of the feature transformation unit 105 using the calculated gradient (step S244). The updating of parameter values ​​in step S244 corresponds to the learning of the feature transformation unit 105.

[0069] For example, the learning control unit 208 optimizes the parameter values ​​so that the loss function value is minimized. Examples of optimization methods used by the learning control unit 208 include, but are not limited to, stochastic gradient descent (SGD) or Adam.

[0070] Next, the feature transformation learning device 200 calculates the loss for training the feature discrimination unit 205 (step S245). As described above, in training the feature discrimination unit 205, the adversarial loss calculation unit 206 calculates the loss using a loss function that minimizes the loss when the feature discrimination unit 205 correctly distinguishes between super-resolution features and high-resolution image features.

[0071] Next, the learning control unit 208 calculates the gradient of the neural network parameters of the feature discrimination unit 205 using the backpropagation method (step S246). Then, the learning control unit 208 updates the parameter values ​​of the feature discrimination unit 205 using the calculated gradient (step S244). The updating of parameter values ​​in step S244 corresponds to the learning of the feature discrimination unit 205.

[0072] In this manner, the learning control unit 208 alternately optimizes the parameter values ​​of the feature transformation unit 105 and the feature discrimination unit 205. When the learning control unit 208 optimizes the parameter values ​​of the feature transformation unit 105, it fixes the parameter values ​​of the feature discrimination unit 205. Conversely, when the learning control unit 208 optimizes the parameter values ​​of the feature discrimination unit 205, it fixes the parameter values ​​of the feature transformation unit 105.

[0073] After step S244, the learning control unit 208 determines whether the learning termination condition has been met (step S248). The learning termination condition here is not limited to a specific condition. For example, the learning control unit 208 may determine that the termination condition has been met when the number of repetitions of the loop from step S212 to S248 reaches a predetermined number.

[0074] Alternatively, after updating the parameter values ​​in step S247, the feature transformation learning device 200 may have the learning control unit 208 or the loss function calculation unit 203 calculate the matching accuracy using super-resolution features based on the evaluation data obtained during the loop. The learning control unit 208 may then determine that the termination condition has been met when the matching accuracy using super-resolution features reaches or exceeds a predetermined accuracy.

[0075] If the learning control unit 208 determines in step S248 that the termination condition has not been met (step S248: NO), the process returns to step S212. In this case, the feature transformation learning device 200 continues to perform learning on the feature transformation unit 105 and the feature discrimination unit 205. On the other hand, if the learning control unit 208 determines in step S248 that the termination condition is met (step S248: YES), the feature transformation learning device 200 terminates the process shown in Figure 4.

[0076] As described above, the image batch acquisition unit 201 acquires high-resolution images. The image reduction unit 202 reduces the high-resolution images to reduced images with a lower resolution than the high-resolution images. The image enlargement unit 103 enlarges the reduced images to degraded images with the same resolution as the high-resolution images. The feature extraction unit 104 extracts features from the high-resolution images and the degraded images. The feature transformation unit 105 transforms the features from the degraded images to generate super-resolution features. The learning control unit 208 causes the feature transformation unit 105 to learn a feature transformation method based on the comparison result between the features from the high-resolution images and the super-resolution features. Learning the feature transformation method here means adjusting the parameter values ​​of the machine learning model that performs the feature transformation.

[0077] In the feature transformation learning device 200, by training the feature transformation unit 105, the feature transformation unit 105 can bring the features of an enlarged image (a reduced image) closer to the features of a high-resolution image. It is expected that the authentication device 100, using the trained feature transformation unit 105, will be able to perform authentication with relatively high accuracy even when receiving a low-resolution image as input.

[0078] Furthermore, the feature discrimination unit 205 receives feature quantities from a high-resolution image or super-resolution feature quantities as input and determines whether the input feature quantities are super-resolution feature quantities or not. The loss function calculation unit 203 causes the feature transformation unit 105 to learn a feature transformation method based on a loss function that minimizes the loss when the feature discrimination unit 205 cannot distinguish between the feature quantities transformed by the feature transformation unit 105 and other feature quantities.

[0079] According to the feature transformation learning device 200, the feature transformation unit 105 is expected to output super-resolution features that are close to the features of the high-resolution image, so that the feature discrimination unit 205 fails to distinguish between the features of the high-resolution image and the super-resolution features. As a result, the authentication device 100 using the trained feature transformation unit 105 is expected to be able to perform authentication with relatively high accuracy even when a low-resolution image is input.

[0080] Furthermore, the loss function calculation unit 203 calculates an index value of the similarity between the high-resolution image features and the super-resolution features. The learning control unit 208 causes the feature transformation unit 105 to learn a feature transformation method based on a loss function in which the loss decreases as the high-resolution image features and the super-resolution features become more similar.

[0081] According to the feature transformation learning device 200, the feature transformation unit 105 is expected to output super-resolution features similar to the features of high-resolution images. As a result, the authentication device 100 using the trained feature transformation unit 105 is expected to be able to perform authentication with relatively high accuracy even when a low-resolution image is input.

[0082] Furthermore, the learning control unit 208 causes the feature transformation unit 105 to learn a feature transformation method based on a loss function that minimizes the loss when the class determined by the super-resolution feature classification matches the correct class.

[0083] As a result, it is expected that the authentication device 100, which uses the trained feature transformation unit 105, will be able to perform authentication with relatively high accuracy even when it receives low-resolution images as input.

[0084] Furthermore, the image reduction unit 202 reduces the high-resolution image to a size determined using random numbers to generate a reduced image. According to the feature transformation learning device 200, the feature transformation unit 105 is expected to be able to output super-resolution features that are close to the features of high-resolution images for enlarged images obtained from images of various resolutions. As a result, the authentication device 100 using the trained feature transformation unit 105 is expected to be able to perform authentication with high accuracy for input images of various resolutions.

[0085] <Third Embodiment> In the third embodiment, an example of the configuration of the feature extraction unit 104 shown in Figures 1 and 3 will be described. It is preferable that the authentication device 100 in Figure 1 and the feature transformation learning device 200 in Figure 3 are equipped with the same feature extraction unit 104, but either the authentication device 100 or the feature transformation learning device 200 may be equipped with the feature extraction unit according to the third embodiment.

[0086] Figure 6 shows an example of the configuration of the feature extraction unit 104 according to the third embodiment. In the configuration shown in Figure 6, the feature extraction unit 104 includes an attention processing unit 301. In the configuration shown in Figure 6, the attention processing units 301 are connected in series. The first attention processing unit 301 receives image data as input, and the last attention processing unit 301 outputs the features. The number of attention processing units 301 included in the feature extraction unit 104 can be one or more.

[0087] In this context, attention refers to emphasizing a part of an image or its corresponding feature. The emphasized feature is more likely to be reflected in the authentication result. For example, in iris authentication, authentication may be performed based on the features of the iris in particular from the image of the eye. Therefore, the attention processing unit 301 may be configured to emphasize the feature corresponding to the iris from the feature extracted by the feature extraction unit 104 from the image of the eye. By emphasizing the feature corresponding to the characteristic part of the image in this way, the authentication accuracy of the authentication device 100 is expected to improve. The attention processing unit 301 is an example of an attention processing means. As described above, the feature extraction unit 104 in Figure 6 corresponds to the example of the feature extraction unit 104 in Figures 1 and 3.

[0088] In the authentication device 100 or the feature transformation learning device 200, the feature extraction unit 104 extracts features from both the high-resolution image and the enlarged low-resolution image. It is preferable to train the feature extraction unit 104 in advance using high-resolution data.

[0089] Figure 6 shows an example where the feature extraction unit 104 is configured using a neural network, with each of the attention processing units 301 corresponding to a layer of the neural network. Attention processing may be performed in each layer of the neural network. Alternatively, attention processing may be performed in only some of the layers of the neural network. Alternatively, attention processing may be performed as a pre-processing or post-processing step for feature extraction.

[0090] Figure 7 shows an example of the configuration of the attention processing unit 301. In the configuration shown in Figure 7, the attention processing unit 301 comprises a two-dimensional convolution operation unit 311a and 311b, a function calculation unit 312, and a multiplication unit 313. The two-dimensional convolution operation units 311a and 311b are collectively referred to as the two-dimensional convolution operation unit 311.

[0091] The 2D convolution unit 311 performs a convolution operation on data in the form of a 2D matrix. The 2D convolution unit 311a performs a convolution operation for feature extraction. The 2D convolution unit 311b performs a convolution operation as a filter for attention to the features extracted by the 2D convolution unit 311a.

[0092] The function calculation unit 312 applies a predetermined function to the output of the two-dimensional convolution unit 311b. For example, the function calculation unit 312 replaces the values ​​of each element of the data in the form of a two-dimensional matrix representing the calculation result of the two-dimensional convolution unit 311b with values ​​obtained by applying the Softmax function. This allows the function calculation unit 312 to calculate weight coefficients for emphasizing the portion detected by the two-dimensional convolution unit 311b from the output data of the two-dimensional convolution unit 311a. Furthermore, the application of a function by the function calculation unit 312 also corresponds to the calculation of the activation function of the neural network.

[0093] The multiplication unit 313 multiplies the output of the 2D convolution unit 311a and the output of the function calculation unit 312 element by element of the data in the form of a 2D matrix. As a result, the multiplication unit 313 weights the features extracted by the 2D convolution unit 311a using the weight coefficients calculated by the function calculation unit 312. The attention processing unit 301 may learn how to detect the area of ​​interest. Specifically, the attention processing unit 301 may learn the filter of the two-dimensional convolution operation unit 311b to reduce the loss calculated by the learning control unit 208.

[0094] Figure 8 shows an example of the configuration of an attention block 302 according to a first modified example of the attention processing unit 301. The feature extraction unit 104 shown in Figure 6 may be equipped with an attention block 302 instead of an attention processing unit 301. In the configuration shown in Figure 8, the attention block 302 comprises a channel separation unit 321, an attention processing unit 301, and a channel joining unit 322.

[0095] In the configuration shown in Figure 8, multiple attention processing units 301 are arranged in parallel. Each attention processing unit 301 receives data input from the channel separation unit 321 and outputs data to the channel coupling unit 322. However, the number of attention processing units 301 in the attention block 302 only needs to be one or more.

[0096] The channel separation unit 321 acquires the input data to the attention block 302 as input data to the channel separation unit 321, and outputs the input data or a portion of the data to each of the attention processing units 301. The processing performed by the channel separation unit 321 corresponds to the process of separating the channels of the neural network into multiple blocks.

[0097] The method by which the channel separation unit 321 generates data for each attention processing unit 301 from the input data is not limited to a specific method. For example, the channel separation unit 321 may output the input data as is to each of the attention processing units 301. Alternatively, the channel separation unit 321 may divide the input data based on its position in the image, such as by splitting it into data corresponding to the left, center, and right of the original image, and output each piece of data to the attention processing unit 301. Furthermore, the channel separation unit 321 may divide the input data based on its position in the channel, such as by splitting the original number of channels into multiple channels, and output each piece of data to the attention processing unit 301.

[0098] Each attention processing unit 301 extracts features from the input data to itself, and outputs a portion of the extracted features with emphasis. The attention processing unit 301 shown in Figure 7 can be used as the attention processing unit 301 of the attention block 302.

[0099] The channel merging unit 322 combines the data output by each of the attention processing units 301 into a single data. The method by which the channel merging unit 322 combines the data can be various methods corresponding to the method by which the channel separation unit 321 generates data for each attention processing unit 301 from a single data.

[0100] For example, if the channel separation unit 321 outputs the input data directly to each of the attention processing units 301, the channel joining unit 322 may combine the data in the form of a two-dimensional matrix output from each of the attention processing units 301 into a single data set by averaging or summing each element.

[0101] Furthermore, if the channel separation unit 321 divides the input data based on its position in the image, the channel joining unit 322 may combine the data from each of the attention processing units 301 into a single data set by arranging and joining them according to their positions in the image.

[0102] In the images used for iris recognition, many features for identifying an individual are present, particularly in the iris and the area around the eye. Thus, it is conceivable that features for identifying an individual are present in multiple parts of the image being authenticated. Therefore, the attention block 302 separates the neural network channels into multiple blocks, performs attention processing on each block, and combines the outputs of each block. This allows the attention block 302 to focus on and extract features from each part of the image that contains many features for identifying an individual, such as the iris, eyelids, and eyebrows.

[0103] The attention block 302 may also highlight some channels of the feature vector instead of the image region. Figure 9 shows an example of the configuration of an attention processing unit 303 according to a second modified example of the attention processing unit 301. The attention block 302 shown in Figure 8 may be configured to include an attention processing unit 303 instead of an attention processing unit 301.

[0104] In the configuration shown in Figure 9, the attention processing unit 303 comprises an image reduction unit 331, a linear calculation unit 332, a function calculation unit 312, and a multiplication unit 313. Comparing the attention processing unit 303 with the attention processing unit 301, the two-dimensional convolution operation unit 311b of the attention processing unit 301 is replaced in the attention processing unit 303 by an image reduction unit 331 and a linear calculation unit 332. In other words, in the attention processing unit 303, the two-dimensional convolution operation unit 311 is divided into an image reduction unit 331 and a linear calculation unit 332. In all other respects, the attention processing unit 303 is the same as the attention processing unit 301.

[0105] When distinguishing between the two image reduction units 331, the image reduction unit 331 corresponding to the two-dimensional convolution unit 311a is denoted as image reduction unit 331a, and the image reduction unit 331 corresponding to the two-dimensional convolution unit 311b is denoted as image reduction unit 331b. When distinguishing between the two linear calculation units 332, the linear calculation unit 332 corresponding to the two-dimensional convolution unit 311a is denoted as linear calculation unit 332a, and the linear calculation unit 332 corresponding to the two-dimensional convolution unit 311b is denoted as linear calculation unit 332b.

[0106] The image reduction unit 331 reduces the input image to a size of 1x1. That is, the image reduction unit 331 reduces a tensor with channel C, height H, and width W to a tensor with channel C, height 1, and width 1. The method by which the image reduction unit 331 reduces the image is not limited to a specific method. For example, the image reduction unit 331 may reduce the image using an averaging method, but is not limited to this.

[0107] The linear operation unit 332 performs linear operations on the channels. The input tensor and output tensor of the linear operation unit 332 each have a size of (C,1,1). The function calculation unit 312 applies the softmax function to the output of the linear calculation unit 332b to calculate weights for emphasizing the channel.

[0108] Figure 10 shows an example of the configuration of an attention processing unit 304, which is a third modified example of the attention processing unit 301. The feature extraction unit 104 shown in Figure 6 may be equipped with an attention processing unit 304 instead of an attention processing unit 301. As shown in Figure 10, the attention processing unit 304 comprises a two-dimensional convolution unit 311a, a multiplication unit 313, and an attention map acquisition unit 341. The two-dimensional convolution unit 311a is also referred to as the two-dimensional convolution unit 311.

[0109] The processing performed by the 2D convolution unit 311a is the same as that performed by the 2D convolution unit 311a of the attention processing unit 301. The processing performed by the multiplication unit 313 is the same as that performed by the multiplication unit 313 of the attention processing unit 301. The attention map acquisition unit 341 acquires an attention map. The attention map is data that indicates the parts of the input data to the attention processing unit 304 that the attention processing unit 304 should emphasize. When the attention processing unit 304 uses an attention map, the attention map is, so to speak, the correct attention data and is included in the training data.

[0110] In the attention processing unit 304, the multiplication unit 313 weights the output from the two-dimensional convolution unit 311a using an attention map. Furthermore, the attention processing unit 304 may learn not only the feature extraction method in the two-dimensional convolution unit 311a, but also the method of emphasizing those features using attention. For example, the attention processing unit 304 may multiply the attention map input from the training data by a temperature coefficient S before inputting it into the softmax function, and the resulting function value may be multiplied by the result of the convolution unit 311 in the multiplication unit 313. In this case, the temperature coefficient S may be a parameter that is adjusted through learning.

[0111] As described above, the attention processing unit 301 performs weighting to emphasize features corresponding to specific parts of the image or channels from which feature extraction is performed. As a result, the authentication device 100 is expected to be able to perform authentication with relatively high accuracy by using feature quantities that emphasize the parts containing many features for identifying an individual.

[0112] <Fourth Embodiment> Figure 11 shows an example of the configuration of a feature extraction unit 114, which is a modified version of the feature extraction unit 104. Both or either of the authentication device 100 in Figure 1 and the feature transformation learning device 200 in Figure 3 may be equipped with a feature extraction unit 114 instead of the feature extraction unit 104.

[0113] In the configuration shown in Figure 11, the feature extraction unit 114 comprises a first feature extraction unit 114a and a second feature extraction unit 114b. The output of the first feature extraction unit 114a is input to the feature conversion unit 105, and the output of the feature conversion unit 105 is input to the second feature extraction unit 114b. The first feature extraction unit 114a and the second feature extraction unit 114b are each composed of a subset network obtained by dividing the neural network constituting the feature extraction unit 104 into two parts between layers.

[0114] Figure 12 shows an example of the hierarchical structure of the neural network that constitutes the feature extraction unit 104. In the example shown in Figure 12, the feature extraction unit 104 is constructed using a neural network with multiple layers. The layers of the neural network constituting the feature extraction unit 104 are divided into two groups: one or more layers from the front and one or more layers from the back. Here, the front refers to the input layers, and the back refers to the output layers.

[0115] The first feature extraction unit 114a is configured using a neural network corresponding to one or more layers from the front of these two groups. The second feature extraction unit 114b is configured using a neural network corresponding to one or more layers from the back of these two groups.

[0116] The output of the first feature extraction unit 114a corresponds to the intermediate features in the feature extraction unit 104. When a low-resolution image is input to the first feature extraction unit 114a as image data, the output of the first feature extraction unit 114a is also referred to as the low-resolution intermediate features. In the configuration shown in Figure 11, the feature transformation unit 105 receives low-resolution intermediate features as input and transforms the input low-resolution intermediate features. The output of the feature transformation unit 105 in this case is also called super-resolution intermediate features. The second feature extraction unit 114b receives the super-resolution intermediate features as input and performs further feature extraction. The output of the second feature extraction unit 114b corresponds to the super-resolution features output by the feature transformation unit 105 in the configurations shown in Figure 1 and Figure 3.

[0117] Thus, the feature transformation unit 105 may perform feature super-resolution using features in the intermediate layers, not limited to features output by the neural network constituting the feature extraction unit 104. For example, VGG16 consists of 13 convolutional layers and 3 fully connected layers. Among these layers, the feature transformation unit 105 may perform feature super-resolution on the intermediate features output by the second layer from the front of the convolutional layer, and the super-resolution intermediate features output by the feature transformation unit 105 may be input to the third layer from the front of the convolutional layer. Alternatively, the feature transformation unit 105 may perform feature super-resolution on the intermediate features output by the fourth layer from the front of the convolutional layer, and the super-resolution intermediate features output by the feature transformation unit 105 may be input to the fifth layer from the front of the convolutional layer.

[0118] As described above, the feature transformation unit 105 performs feature transformation on the intermediate features generated by the feature extraction unit. As a result, the authentication device 100 can obtain an effect between image super-resolution, which super-resolutions the input image to the feature extractor, and feature super-resolution, which super-resolutions the output features. In other words, by not limiting the super-resolution portion to the input or output to the feature transformation unit 105, feature super-resolution can be performed using intermediate features that maximize authentication performance at the desired magnification. As a result, feature super-resolution can be performed with even higher accuracy.

[0119] <Fifth Embodiment> Figure 13 shows an example of the configuration of an authentication device according to the fifth embodiment. In the configuration shown in Figure 13, the authentication device 411 includes an image acquisition unit 101 for acquiring an image to be authenticated, an image acquisition unit 102 for comparison, a feature extraction unit 104, a feature conversion unit 105, and a feature comparison unit 106.

[0120] Of the parts in Figure 13, those that have the same function as the parts in Figure 1 are denoted by the same reference numerals (101, 12, 104, 105, 106), and detailed explanations are omitted here. The authentication device 100 in Figure 1 and the authentication device 411 in Figure 13 use different images for authentication. Authentication device 100 acquires a low-resolution image as the image to be authenticated and a high-resolution image as the comparison image. In contrast, authentication device 411 acquires an image of the iris with glasses on as the image to be authenticated and an image of the iris without glasses, with the same resolution as the image to be authenticated, as the comparison image. Note that the resolution of the authentication image and the comparison image only need to be of a similar resolution; they do not need to be exactly the same.

[0121] Furthermore, the authentication device 411 does not have the image enlargement unit 103 that the authentication device 100 has. In the authentication device 411, the authentication target image acquisition unit 101 outputs the acquired authentication target image to the feature extraction unit 104. In all other respects, the authentication device 411 is the same as the authentication device 100.

[0122] In the authentication device 411 of the fifth embodiment, the feature extraction unit 104 extracts the features of the image to be authenticated using the same method as the feature extraction method for the comparison image, which is an image of the iris of the naked eye. The feature extraction unit 104 may also extract the features of the comparison image during authentication. Alternatively, the authentication device 100 may store the features of the comparison image in advance. In the authentication device 411 of the fifth embodiment, the feature transformation unit 105 receives input of feature quantities from an iris image when wearing glasses and transforms the features so that the difference between the transformed iris image and the unenhanced iris image is reduced. The machine learning parameter values ​​of the feature transformation unit 105 are adjusted by learning using a loss function whose value decreases as the difference between the transformed features from the iris image when wearing glasses and the features from the unenhanced iris image decreases. In this case, the iris image when wearing glasses corresponds to an example of a first training image, and the iris image without glasses corresponds to an example of a second training image. The feature transformation learning device 412 according to the sixth embodiment performs this learning.

[0123] As described above, the authentication target image acquisition unit 101 acquires an image of the iris while wearing glasses as the authentication target image. The feature extraction unit 104 extracts the features of the authentication target image using the same method as the feature extraction method for the comparison image, which is an image of the iris without glasses. The feature transformation unit 105 transforms the features of the authentication target image based on the learning results of the feature transformation method, which uses a loss function whose value decreases as the difference between the transformed features from the features of the first training image (an image of the iris with glasses) and the features of the second training image (an image of the iris without glasses) decreases. The feature comparison unit 106 compares the transformed features from the feature transformation unit 105 with the features of the comparison image.

[0124] According to the authentication device 411 of the fifth embodiment, it is expected that the feature quantities of the iris image when wearing glasses can be made closer to the feature quantities of the iris image when not wearing glasses. In this respect, according to the authentication device 411 of the fifth embodiment, it is expected that iris authentication can be performed with relatively high accuracy even when the person to be authenticated is wearing glasses.

[0125] <Sixth Embodiment> Figure 14 shows an example of the configuration of a feature transformation learning device according to the sixth embodiment. The feature transformation learning device 412 of the sixth embodiment performs training on the feature transformation unit 105 of the authentication device 411 of the fifth embodiment.

[0126] As shown in Figure 14, the feature transformation learning device 412 comprises an image batch acquisition unit 413, a feature extraction unit 104, a feature transformation unit 105, a loss function calculation unit 203, and a learning control unit 208. The loss function calculation unit 203 comprises a reconstruction loss calculation unit 204, a feature discrimination unit 205, an adversarial loss calculation unit 206, and a similarity loss calculation unit 207. Of the parts in Figure 14, those that have the same function as the parts in Figure 3 are denoted by the same reference numerals (104, 105, 203-208), and detailed explanations are omitted here.

[0127] The feature transformation learning device 200 in Figure 3 and the feature transformation learning device 412 in Figure 14 contain different images in their training data. The image batch acquisition unit 201 of the feature transformation learning device 200 acquires training data that includes multiple combinations of high-resolution images and class labels indicating the correct class in the classification of those images.

[0128] In response, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of iris images with glasses on and iris images without glasses, and class labels indicating the correct class in the classification of those images. Images of the same person and with the same resolution are used as the iris images with glasses on and iris images without glasses. The image batch acquisition unit 413 corresponds to an example of an image acquisition unit. The image of the iris of the naked eye included in the training data corresponds to the first example image, and the image of the iris with glasses on corresponds to the second example image.

[0129] Furthermore, the feature transformation learning device 412 does not include the image reduction unit 202 and the image enlargement unit 103 that are present in the feature transformation learning device 200. The image batch acquisition unit 201 outputs images of the iris with glasses on and images of the iris without glasses to the feature extraction unit 104. The feature extraction unit 104 extracts the features from the iris image with glasses on and the iris image without glasses. The features of an iris image without glasses are an example of a first-class feature. The features of an iris image with glasses on are an example of a second-class feature.

[0130] The feature transformation unit 105 receives the feature quantities of the iris image when wearing glasses as input and transforms the feature quantities so that the difference from the feature quantities of the iris image when not wearing glasses is minimized. The feature quantities after transformation from the feature quantities of the iris image when wearing glasses correspond to an example of a third feature. The loss function calculation unit 203 calculates a loss based on the transformed features from the feature transformation unit 105 and the features of the iris image of the naked eye.

[0131] The learning control unit 208 updates the machine learning model parameter values ​​of the feature transformation unit 105 and the feature discrimination unit 205 based on the loss calculated by the loss function calculation unit 203. The learning control unit 208 updates the machine learning model parameter values ​​so that the difference between the features transformed by the feature transformation unit 105 and the features of the iris image of the naked eye is minimized.

[0132] As described above, the image batch acquisition unit 413 acquires a first image, which is an image of the iris without glasses, and a second image, which is an image of the iris with glasses on. The feature extraction unit 104 extracts a first feature, which is a feature of the first image, and a second feature, which is a feature of the second image. The feature conversion unit 105 converts the second feature into a third feature. The learning control unit 208 causes the feature conversion unit 105 to learn a feature conversion method based on the comparison result between the first feature and the third feature.

[0133] According to the feature transformation learning device 412 of the sixth embodiment, the feature transformation unit 105 can be trained to transform the features of the iris image when wearing glasses to be closer to the features of the iris image when not wearing glasses. When the authentication device 411 performs authentication using this feature transformation unit 105, it is expected that iris authentication can be performed with relatively high accuracy even when the person to be authenticated is wearing glasses.

[0134] <Seventh Embodiment> Figure 15 shows an example of the configuration of an authentication device according to the seventh embodiment. In the configuration shown in Figure 15, the authentication device 411 includes an image acquisition unit 101 for acquiring an image to be authenticated, an image acquisition unit 102 for comparison, a feature extraction unit 104, a feature conversion unit 105, and a feature comparison unit 106.

[0135] The configuration of the authentication device 411 according to the seventh embodiment is the same as that of the authentication device 411 according to the fifth embodiment. In Figure 15, parts that have the same function as the parts in Figure 13 are denoted by the same reference numerals (101, 102, 104, 105, 106, 411), and detailed explanations are omitted here.

[0136] In the seventh embodiment, the image used for authentication by the authentication device 411 differs from that in the fifth embodiment. In the fifth embodiment, the authentication device 411 acquires an image of the iris while wearing glasses as the image to be authenticated, and acquires an image of the iris without glasses, with the same resolution as the image to be authenticated, as a comparison image. In contrast, in the seventh embodiment, the authentication device 411 acquires an image of the iris taken from an oblique angle as the image to be authenticated, and acquires an image of the iris taken from the front, with the same resolution as the image to be authenticated, as a comparison image.

[0137] In the authentication device 411 of the seventh embodiment, the feature extraction unit 104 extracts the features of the image to be authenticated using the same method as the feature extraction method for a comparison image, which is an image of the iris taken from the front. The feature extraction unit 104 may also extract the features of the comparison image during authentication. Alternatively, the authentication device 100 may store the features of the comparison image in advance. In the authentication device 411 of the seventh embodiment, the feature transformation unit 105 receives the feature quantities of an iris image taken from an oblique angle and transforms the feature quantities so that the difference between them and the feature quantities of an iris image taken from the front is reduced. The machine learning parameter values ​​of the feature transformation unit 105 are adjusted by learning using a loss function whose value decreases as the difference between the transformed feature quantities from the iris image taken from an oblique angle and the feature quantities of the iris image taken from the front becomes smaller. In this case, the iris image taken from an oblique angle corresponds to an example of a first training image, and the iris image taken from the front corresponds to an example of a second training image. The feature transformation learning device 412 of the eighth embodiment performs this learning.

[0138] As described above, the authentication target image acquisition unit 101 acquires an image of the iris taken from an oblique angle as the authentication target image. The feature extraction unit 104 extracts the features of the authentication target image using the same method as the feature extraction method for the comparison image, which is an image of the iris taken from the front. The feature transformation unit 105 transforms the features of the authentication target image based on the learning results of the feature transformation method, which uses a loss function whose value decreases as the difference between the transformed features of the first training image (an image of the iris taken from an oblique angle) and the features of the second training image (an image of the iris taken from the front) decreases. The feature comparison unit 106 compares the transformed features by the feature transformation unit 105 with the features of the comparison image.

[0139] According to the authentication device 411 of the seventh embodiment, it is expected that the image features of the iris captured from an oblique angle can be made to approximate the feature features of the iris captured from the front. In this respect, according to the authentication device 411 of the seventh embodiment, it is expected that iris authentication can be performed with relatively high accuracy even when an image of the iris of the person to be authenticated captured from an oblique angle is used as the image to be authenticated.

[0140] <Eighth Embodiment> Figure 16 shows an example of the configuration of the feature transformation learning device according to the eighth embodiment. The feature transformation learning device 412 of the eighth embodiment performs training on the feature transformation unit 105 of the authentication device 411 of the seventh embodiment.

[0141] As shown in Figure 16, the feature transformation learning device 412 comprises an image batch acquisition unit 413, a feature extraction unit 104, a feature transformation unit 105, a loss function calculation unit 203, and a learning control unit 208. The loss function calculation unit 203 comprises a reconstruction loss calculation unit 204, a feature discrimination unit 205, an adversarial loss calculation unit 206, and a similarity loss calculation unit 207.

[0142] The configuration of the feature transformation learning device 412 according to the eighth embodiment is the same as that of the feature transformation learning device 412 according to the sixth embodiment. Parts in Figure 16 that have the same function as those in Figure 14 are denoted by the same reference numerals (104, 105, 203-208, 312, 412, 413), and detailed explanations are omitted here.

[0143] In the eighth embodiment, the images included in the training data acquired by the feature transformation learning device 412 are different from those in the sixth embodiment. In the sixth embodiment, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of iris images with glasses on and iris images without glasses, and class labels indicating the correct class in the classification of those images.

[0144] In contrast, in the eighth embodiment, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of iris images taken from the front and iris images taken from an oblique angle, and class labels indicating the correct class in the classification of those images. Images of the same person and with the same resolution are used as the iris images taken from the front and iris images taken from an oblique angle. The image batch acquisition unit 413 corresponds to an example of an image acquisition unit. The iris image taken from the front included in the training data corresponds to the first example image, and the iris image taken from an oblique angle corresponds to the second example image.

[0145] In the eighth embodiment, the image batch acquisition unit 201 outputs an image of the iris taken from an oblique angle and an image of the iris taken from the front to the feature extraction unit 104, respectively. The feature extraction unit 104 extracts the features from the image of the iris taken from an oblique angle and the features from the image of the iris taken from the front. The features of an iris image taken from the front are an example of a first-order feature. The features of an iris image taken from an oblique angle are an example of a second-order feature.

[0146] The feature transformation unit 105 receives the feature quantities from an iris image taken from an oblique angle and transforms the feature quantities so that the difference between them and the feature quantities from an iris image taken from the front is minimized. The transformed feature quantities from the iris image taken from an oblique angle correspond to an example of a third feature quantity. The loss function calculation unit 203 calculates a loss based on the transformed feature quantities from the feature transformation unit 105 and the feature quantities from the iris image taken from the front.

[0147] The learning control unit 208 updates the machine learning model parameter values ​​of the feature transformation unit 105 and the feature discrimination unit 205 based on the loss calculated by the loss function calculation unit 203. The learning control unit 208 updates the machine learning model parameter values ​​so that the difference between the features transformed by the feature transformation unit 105 and the features of the iris image taken from the front is minimized.

[0148] As described above, the image batch acquisition unit 413 acquires a first image, which is an image of the iris taken from the front, and a second image, which is an image of the iris taken from an oblique angle. The feature extraction unit 104 extracts a first feature, which is a feature of the first image, and a second feature, which is a feature of the second image. The feature conversion unit 105 converts the second feature into a third feature. The learning control unit 208 causes the feature conversion unit 105 to learn a feature conversion method based on the comparison result between the first feature and the third feature.

[0149] According to the feature transformation learning device 412 of the eighth embodiment, the feature transformation unit 105 can be trained to transform the features of an iris image taken from an oblique angle to be closer to the features of an iris image taken from the front. When the authentication device 411 performs authentication using this feature transformation unit 105, it is expected that iris authentication can be performed with relatively high accuracy even when an iris image taken from an oblique angle is used as the image to be authenticated.

[0150] <Ninth Embodiment> Figure 17 shows an example of the configuration of an authentication device according to the ninth embodiment. In the configuration shown in Figure 17, the authentication device 411 includes an image acquisition unit 101 for acquiring an image to be authenticated, an image acquisition unit 102 for comparison, a feature extraction unit 104, a feature conversion unit 105, and a feature comparison unit 106.

[0151] The configuration of the authentication device 411 according to the ninth embodiment is the same as that of the authentication device 411 according to the fifth embodiment. Parts in Figure 17 that have the same function as those in Figure 13 are denoted by the same reference numerals (101, 102, 104, 105, 106, 411), and detailed explanations are omitted here.

[0152] In the ninth embodiment, the image used for authentication by the authentication device 411 differs from that in the fifth embodiment. In the fifth embodiment, the authentication device 411 acquires an image of the iris while wearing glasses as the image to be authenticated, and acquires an image of the iris without glasses, with the same resolution as the image to be authenticated, as a comparison image. In contrast, the authentication device 411 in the ninth embodiment acquires an image of the iris captured in visible light as the image to be authenticated, and acquires an image of the iris with the same resolution as the image to be authenticated, but captured in light other than visible light, such as near-infrared light, as a comparison image. Note that the wavelength of the image for authentication is not limited to visible light, and may be an image captured in other wavelengths such as ultraviolet or near-infrared light. Also, there is no particular limit to the number of channels representing the color of the input image.

[0153] In the authentication device 411 of the ninth embodiment, the feature extraction unit 104 extracts the features of the image to be authenticated using the same method as the feature extraction method for a comparison image, which is an image of the iris taken with light other than visible light. The feature extraction unit 104 may also extract the features of the comparison image during authentication. Alternatively, the authentication device 100 may store the features of the comparison image in advance. In the authentication device 411 of the ninth embodiment, the feature transformation unit 105 receives feature quantities from an iris image captured in visible light and transforms the features so that the difference between the transformed features and those from an iris image captured in light other than visible light is reduced. The machine learning parameter values ​​of the feature transformation unit 105 are adjusted by learning using a loss function whose value decreases as the difference between the transformed features from the iris image captured in visible light and the features from an iris image captured in light other than visible light decreases. In this case, the iris image captured in visible light corresponds to an example of a first training image, and the iris image captured in light other than visible light corresponds to an example of a second training image. The feature transformation learning device 412 of the tenth embodiment performs this learning.

[0154] As described above, the authentication target image acquisition unit 101 acquires an image of the iris captured in visible light as the authentication target image. The feature extraction unit 104 extracts the features of the authentication target image using the same method as the feature extraction method for the comparison image, which is an image of the iris captured in light other than visible light. The feature transformation unit 105 transforms the features of the authentication target image based on the learning results of the feature transformation method, which uses a loss function whose value decreases as the difference between the transformed features from the features of the first training image (an image of the iris captured in visible light) and the features of the second training image (an image of the iris captured in light other than visible light) decreases. The feature comparison unit 106 compares the transformed features from the feature transformation unit 105 with the features of the comparison image.

[0155] According to the authentication device 411 of the ninth embodiment, it is expected that the image features of the iris captured in visible light can be made to approximate the image features of the iris captured in light other than visible light. According to the authentication device 411 of the seventh embodiment, in this respect, it is expected that even when the comparison image is captured in light other than visible light, iris authentication can be performed with relatively high accuracy using the image captured in visible light.

[0156] <Tenth Embodiment> Figure 18 shows an example of the configuration of a feature transformation learning device according to the 10th embodiment. The feature transformation learning device 412 of the 10th embodiment performs learning of the feature transformation unit 105 of the authentication device 411 of the 9th embodiment. In the tenth embodiment, the wavelength of the image used for authentication is not limited to visible light, but may be an image captured at other wavelengths such as ultraviolet or near-infrared light. Furthermore, the number of channels representing the color of the input image is not particularly limited.

[0157] As shown in Figure 18, the feature transformation learning device 412 comprises an image batch acquisition unit 413, a feature extraction unit 104, a feature transformation unit 105, a loss function calculation unit 203, and a learning control unit 208. The loss function calculation unit 203 comprises a reconstruction loss calculation unit 204, a feature discrimination unit 205, an adversarial loss calculation unit 206, and a similarity loss calculation unit 207.

[0158] The configuration of the feature transformation learning device 412 according to the 10th embodiment is the same as that of the feature transformation learning device 412 according to the 6th embodiment. In Figure 18, parts that have the same function as the parts in Figure 14 are denoted by the same reference numerals (104, 105, 203-208, 312, 412, 413), and detailed explanations are omitted here.

[0159] In the tenth embodiment, the images included in the training data acquired by the feature transformation learning device 412 are different from those in the sixth embodiment. In the sixth embodiment, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of iris images with glasses on and iris images without glasses, and class labels indicating the correct class in the classification of those images.

[0160] In contrast, in the tenth embodiment, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of iris images taken with light other than visible light and iris images taken with visible light, and class labels indicating the correct class in the classification of those images. Images of the same person and with the same resolution are used as the iris images taken with light other than visible light and the iris images taken with visible light. The image batch acquisition unit 413 corresponds to an example of an image acquisition unit. Images of irises captured with light other than visible light included in the training data correspond to the first example image, and images of irises captured with visible light correspond to the second example image.

[0161] In the tenth embodiment, the image batch acquisition unit 201 outputs images of the iris captured with visible light and images of the iris captured with light other than visible light to the feature extraction unit 104. The feature extraction unit 104 extracts the feature quantities from the iris images captured with visible light and the feature quantities from the iris images captured with light other than visible light. Features of iris images captured with light other than visible light are examples of first-order features. Features of iris images captured with visible light are examples of second-order features.

[0162] The feature transformation unit 105 receives the feature quantities of an iris image captured in visible light as input and transforms the feature quantities so that the difference between them and the feature quantities of an iris image captured in light other than visible light is minimized. The transformed feature quantities from the iris image captured in visible light correspond to an example of a third feature quantity. The loss function calculation unit 203 calculates a loss based on the transformed feature quantities from the feature transformation unit 105 and the feature quantities of an iris image captured in light other than visible light.

[0163] The learning control unit 208 updates the machine learning model parameter values ​​of the feature transformation unit 105 and the feature discrimination unit 205 based on the loss calculated by the loss function calculation unit 203. The learning control unit 208 updates the machine learning model parameter values ​​so that the difference between the features transformed by the feature transformation unit 105 and the features of the iris image taken with light other than visible light is minimized.

[0164] As described above, the image batch acquisition unit 413 acquires a first image, which is an image of the iris taken with light other than visible light, and a second image, which is an image of the iris taken with visible light. The feature extraction unit 104 extracts a first feature, which is the feature of the first image, and a second feature, which is the feature of the second image. The feature conversion unit 105 converts the second feature into a third feature. The learning control unit 208 causes the feature conversion unit 105 to learn a feature conversion method based on the comparison result between the first feature and the third feature.

[0165] According to the feature transformation learning device 412 of the 10th embodiment, the feature transformation unit 105 can be trained to transform the features of an iris image captured in visible light to be closer to the features of an iris image captured in light other than visible light. When the authentication device 411 performs authentication using this feature transformation unit 105, it is expected that iris authentication can be performed with relatively high accuracy using an image captured in visible light, even when the comparison image is captured in light other than visible light.

[0166] <Embodiment 11> Figure 19 is a diagram showing an example of the configuration of an authentication device according to the 11th embodiment. In the configuration shown in Figure 19, the authentication device 411 comprises an image acquisition unit 101 for acquiring an image to be authenticated, an image acquisition unit 102 for comparison, a feature extraction unit 104, a feature conversion unit 105, and a feature comparison unit 106.

[0167] The configuration of the authentication device 411 according to the 11th embodiment is the same as that of the authentication device 411 according to the 5th embodiment. In Figure 19, parts that have the same function as the parts in Figure 13 are denoted by the same reference numerals (101, 102, 104, 105, 106, 411), and detailed explanations are omitted here.

[0168] In the 11th embodiment, the image used for authentication by the authentication device 411 differs from that in the 5th embodiment. In the 5th embodiment, the authentication device 411 acquires an image of the iris with glasses on as the image to be authenticated, and acquires an image of the iris without glasses with the same resolution as the image to be authenticated as a comparison image. In contrast, in the 11th embodiment, the authentication device 411 acquires an image of the iris that is out of focus as the image to be authenticated, and acquires an image of the iris that is in focus with the same resolution as the image to be authenticated as a comparison image. When an image is in focus, it is also called the point of focus. When an image is out of focus, it is also called the point of focus.

[0169] In the authentication device 411 of the 11th embodiment, the feature extraction unit 104 extracts the features of the image to be authenticated using the same method as the feature extraction method for the comparison image, which is an image of the iris in focus. The feature extraction unit 104 may also extract the features of the comparison image during authentication. Alternatively, the authentication device 100 may store the features of the comparison image in advance. In the authentication device 411 of the ninth embodiment, the feature transformation unit 105 receives the feature quantities of an iris image that is out of focus as input and transforms the feature quantities so that the difference between the transformed feature quantities and the feature quantities of an iris image that is in focus is reduced. The machine learning parameter values ​​of the feature transformation unit 105 are adjusted by learning using a loss function whose value decreases as the difference between the transformed feature quantities from the iris image that is out of focus and the feature quantities of the iris image that is in focus decreases. In this case, the iris image that is out of focus corresponds to an example of a first training image, and the iris image that is in focus corresponds to an example of a second training image. The feature transformation learning device 412 of the twelfth embodiment performs this learning.

[0170] As described above, the authentication target image acquisition unit 101 acquires an image of the iris that is out of focus as the authentication target image. The feature extraction unit 104 extracts the features of the authentication target image using the same method as the feature extraction method for the comparison image, which is an image of the iris that is in focus. The feature transformation unit 105 transforms the features of the authentication target image based on the learning results of the feature transformation method, which uses a loss function whose value decreases as the difference between the transformed features from the features of the first training image (an image of the iris that is out of focus) and the features of the second training image (an image of the iris that is in focus) decreases. The feature comparison unit 106 compares the transformed features from the feature transformation unit 105 with the features of the comparison image.

[0171] According to the authentication device 411 of the 11th embodiment, it is expected that the feature quantities of an iris image that is out of focus can be made closer to the feature quantities of an iris image that is in focus. In this respect, according to the authentication device 411 of the 11th embodiment, it is expected that iris authentication can be performed with relatively high accuracy even when an iris image that is out of focus is used as the image to be authenticated.

[0172] <Twelfth Embodiment> Figure 20 shows an example of the configuration of the feature transformation learning device according to the 12th embodiment. The feature transformation learning device 412 of the 12th embodiment performs training on the feature transformation unit 105 of the authentication device 411 of the 11th embodiment.

[0173] As shown in Figure 20, the feature transformation learning device 412 comprises an image batch acquisition unit 413, a feature extraction unit 104, a feature transformation unit 105, a loss function calculation unit 203, and a learning control unit 208. The loss function calculation unit 203 comprises a reconstruction loss calculation unit 204, a feature discrimination unit 205, an adversarial loss calculation unit 206, and a similarity loss calculation unit 207.

[0174] The configuration of the feature transformation learning device 412 according to the 12th embodiment is the same as that of the feature transformation learning device 412 according to the 6th embodiment. Parts in Figure 20 that have the same function as the parts in Figure 14 are denoted by the same reference numerals (104, 105, 203-208, 312, 412, 413), and detailed explanations are omitted here.

[0175] In the twelfth embodiment, the images included in the training data acquired by the feature transformation learning device 412 are different from those in the sixth embodiment. In the sixth embodiment, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of iris images with glasses on and iris images without glasses, and class labels indicating the correct class in the classification of those images.

[0176] In contrast, in the 12th embodiment, the image batch acquisition unit 413 of the feature transformation learning device 412 acquires training data that includes multiple combinations of images of the iris that are out of focus and images of the iris that are in focus, and class labels that indicate the correct class in the classification of those images. Images of the same person and with the same resolution are used as the images of the iris that are out of focus and images of the iris that are in focus. The image batch acquisition unit 413 corresponds to an example of an image acquisition unit. Images of the iris that are in focus in the training data correspond to the first example image, and images of the iris that are out of focus correspond to the second example image.

[0177] In the 12th embodiment, the image batch acquisition unit 201 outputs images of the iris that are out of focus and images of the iris that are in focus to the feature extraction unit 104. The feature extraction unit 104 extracts the features from the images of the iris that are out of focus and the features from the images of the iris that are in focus. The feature of an iris image that is in focus corresponds to an example of a first feature. The feature of an iris image that is out of focus corresponds to an example of a second feature.

[0178] The feature transformation unit 105 receives the feature quantities of the iris image, which is out of focus, as input and transforms the feature quantities so that the difference between them and the feature quantities of the iris image, which is in focus, is minimized. The transformed feature quantities from the iris image, which is out of focus, correspond to an example of a third feature. The loss function calculation unit 203 calculates a loss based on the transformed feature quantities from the feature transformation unit 105 and the feature quantities of the iris image, which is in focus.

[0179] The learning control unit 208 updates the machine learning model parameter values ​​of the feature transformation unit 105 and the feature discrimination unit 205 based on the loss calculated by the loss function calculation unit 203. The learning control unit 208 updates the machine learning model parameter values ​​so that the difference between the features transformed by the feature transformation unit 105 and the features of the iris image in focus is minimized.

[0180] As described above, the image batch acquisition unit 413 acquires a first image, which is an image of the iris in focus, and a second image, which is an image of the iris that is out of focus. The feature extraction unit 104 extracts a first feature, which is a feature of the first image, and a second feature, which is a feature of the second image. The feature conversion unit 105 converts the second feature into a third feature. The learning control unit 208 causes the feature conversion unit 105 to learn a feature conversion method based on the comparison result between the first feature and the third feature.

[0181] According to the feature transformation learning device 412 of the 12th embodiment, the feature transformation unit 105 can be trained to transform the features of an iris image that is out of focus to be closer to the features of an iris image that is in focus. When the authentication device 411 performs authentication using this feature transformation unit 105, it is expected that iris authentication can be performed with relatively high accuracy even when an image that is out of focus is used as the image to be authenticated.

[0182] <13th Embodiment> Figure 21 is a diagram showing an example configuration of a feature transformation learning device according to the 13th embodiment. In the configuration shown in Figure 21, the feature transformation learning device 610 comprises an image acquisition unit 611, an image reduction unit 612, an image enlargement unit 613, a feature extraction unit 614, a feature transformation unit 615, and a learning control unit 616.

[0183] In this configuration, the image acquisition unit 611 acquires the first image. The image reduction unit 612 reduces the first image to a second image with a lower resolution than the first image. The image enlargement unit 613 enlarges the second image to a third image with the same resolution as the first image. The feature extraction unit 614 extracts the first feature, which is the feature of the first image, and the second feature, which is the feature of the third image. The feature conversion unit 615 converts the second feature to the third feature. The learning control unit 616 causes the feature conversion unit 615 to learn a feature conversion method based on the comparison result between the first feature and the third feature.

[0184] The image acquisition unit 611 is an example of an image acquisition means. The image reduction unit 612 is an example of an image reduction means. The image enlargement unit 613 is an example of an image enlargement means. The feature extraction unit 614 is an example of a feature extraction means. The feature conversion unit 615 is an example of a feature conversion means. The learning control unit 616 is an example of a learning control means.

[0185] In the feature transformation learning device 610, by training the feature transformation unit 615, the feature transformation unit 615 can bring the features of an enlarged image (a reduced image) closer to the features of a high-resolution image. It is expected that an authentication device using the trained feature transformation unit 615 will be able to perform authentication with relatively high accuracy even when receiving a low-resolution image as input.

[0186] <14th Embodiment> Figure 22 is a diagram showing an example of the configuration of an authentication device according to the 14th embodiment. In the configuration shown in Figure 22, the authentication device 620 includes an image acquisition unit 621, an image enlargement unit 622, a feature extraction unit 623, a feature conversion unit 624, and a feature comparison unit 625.

[0187] In this configuration, the authentication target image acquisition unit 621 acquires the authentication target image. The image enlargement unit 622 enlarges the authentication target image. The feature extraction unit 623 extracts the features of the enlarged image of the authentication target image using the same method as the feature extraction method for a comparison image with a higher resolution than the authentication target image. The feature transformation unit 624 transforms the features of the enlarged image of the authentication target image based on the learning results of a feature transformation method, which uses a loss function whose value decreases as the difference between the transformed features from the training image features and the features of the degraded image (which is obtained by scaling down the training image) decreases. The feature comparison unit 625 compares the transformed features from the feature transformation unit 624 with the features of the comparison image.

[0188] The authentication target image acquisition unit 621 is an example of an authentication target image acquisition means. The image enlargement unit 622 is an example of an image enlargement means. The feature extraction unit 623 is an example of a feature extraction means. The feature conversion unit 624 is an example of a feature conversion means. The feature comparison unit 625 is an example of a feature comparison means.

[0189] According to the authentication device 620, by enlarging a low-resolution image to extract features and transforming the extracted features, features comparable to those of a high-resolution comparison image can be obtained for the low-resolution image of the image to be authenticated. In this respect, the authentication device 620 can perform authentication even if the input image is a low-resolution image.

[0190] Furthermore, it is expected that the feature transformation performed by the feature transformation unit 624 will bring the features of the enlarged image to be authenticated closer to those of the image to be authenticated if it were captured at a higher resolution. In this respect, the authentication device 620 is expected to be able to perform authentication with high accuracy. Furthermore, the authentication device 620 can perform iris authentication using low-resolution images. This means that the authentication device 620 can perform both facial authentication and iris authentication using a single facial image without requiring a high-resolution camera, and is expected to improve authentication accuracy.

[0191] <15th Embodiment> Figure 23 is a flowchart showing an example of the processing steps in the feature transformation learning method according to the 15th embodiment. The feature transformation learning method shown in Figure 23 includes acquiring an image (step S611), reducing the size of the image (step S612), enlarging the image (step S613), extracting features (step S614), transforming the features (step S615), and controlling the learning (step S616).

[0192] In step S611, the first image is acquired. In step S612, the first image is reduced to a second image with a lower resolution than the first image. In step S613, the second image is enlarged to a third image with the same resolution as the first image. In step S614, the first feature, which is the feature of the first image, and the second feature, which is the feature of the third image, are extracted. In step S615, the second feature is converted to a third feature. In step S616, the learning is controlled, and a feature conversion method is learned that converts the second feature to a third feature based on the comparison result between the first and third features.

[0193] According to the feature transformation learning method shown in Figure 23, by training the feature transformation method, the features of an enlarged image obtained from a reduced image can be made to approximate the features of a high-resolution image through feature transformation. In authentication using the trained feature transformation method, it is expected that authentication can be performed with relatively high accuracy even when a low-resolution image is received as input.

[0194] <Embodiment 16> Figure 24 is a flowchart showing an example of the processing steps in the authentication method according to the 16th embodiment. The authentication method shown in Figure 24 includes acquiring an image to be authenticated (step S621), enlarging the image (step S622), extracting features (step S623), transforming the features (step S624), and comparing the features (step S625).

[0195] In step S621, the image to be authenticated is acquired. In step S622, the image to be authenticated is enlarged. In step S623, the features of the enlarged image of the image to be authenticated are extracted using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated. In step S624, the features of the enlarged image of the image to be authenticated are transformed based on the learning results of a feature transformation method, which uses a loss function whose value decreases as the difference between the transformed features from the features of the training image and the features of the degraded image (which is obtained by scaling down the training image and then enlarging it) decreases. In step S625, the features of the enlarged image of the image to be authenticated are compared with the features of the comparison image.

[0196] According to the authentication method shown in Figure 24, by enlarging a low-resolution image, extracting features, and transforming the extracted features, it is possible to obtain features for the low-resolution image to be authenticated that are comparable to the features of the high-resolution image used for comparison. In this respect, the authentication method shown in Figure 24 can perform authentication even if the input image is a low-resolution image.

[0197] Furthermore, through feature transformation, it is expected that the features of the enlarged image to be authenticated will approach those of an image captured at a higher resolution. In this respect, the authentication method shown in Figure 24 is expected to enable highly accurate authentication. Furthermore, according to the authentication method shown in Figure 24, iris authentication can be performed using a low-resolution image. As a result, the authentication device 100 can perform both facial authentication and iris authentication using a single facial image without requiring a high-resolution camera, and an improvement in authentication accuracy is expected.

[0198] <Embodiment 17> The configuration of the authentication device 620 according to the 17th embodiment is the same as the configuration of the authentication device 620 according to the 14th embodiment. The authentication device 620 according to the 17th embodiment further includes a feature discrimination unit that receives high-resolution image feature quantities or super-resolution feature quantities as input and determines whether the input feature quantities are super-resolution feature quantities or not. In this configuration, the feature discrimination unit causes the feature transformation unit 615 to learn a feature transformation method based on a loss function that minimizes the loss when the feature transformation unit 615 cannot distinguish between the transformed feature quantities and other feature quantities.

[0199] <Embodiment 18> The configuration of the authentication device 620 according to the 18th embodiment is the same as the configuration of the authentication device 620 according to the 14th or 17th embodiment. In the authentication device 620 according to the 18th embodiment, the learning control unit 616 causes the feature transformation unit 615 to learn a feature transformation method based on a loss function in which the loss decreases as the features of the high-resolution image and the super-resolution features become more similar.

[0200] <19th Embodiment> The configuration of the authentication device 620 according to the 19th embodiment is the same as the configuration of the authentication device 620 according to the 14th embodiment, the 17th embodiment, or the 18th embodiment. In the authentication device 620 according to the 19th embodiment, the learning control unit 616 causes the feature transformation unit 615 to learn a feature transformation method based on a loss function that minimizes the loss when the class determined by the classification based on super-resolution features matches the correct class.

[0201] <20th Embodiment> The configuration of the authentication device 620 according to the 20th embodiment is the same as the configuration of the authentication device 620 according to the 14th embodiment, the 17th embodiment, the 18th embodiment, or the 19th embodiment. In the authentication device 620 according to the 20th embodiment, the image reduction unit 612 reduces a high-resolution image to a size determined using random numbers to generate a reduced image.

[0202] Figure 25 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. In the configuration shown in Figure 25, the computer 700 includes a CPU (Central Processing Unit) 710, a main memory 720, an auxiliary memory 730, and an interface 740.

[0203] One or more of the above authentication device 100, feature transformation learning device 200, authentication device 411, feature transformation learning device 412, feature transformation learning device 610, and authentication device 620, or a part thereof, may be implemented in the computer 700. In that case, the operation of each processing unit described above is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory 720, and executes the above processing according to the program. The CPU 710 also allocates memory areas in the main memory 720 corresponding to each of the above-mentioned memory units according to the program. Communication between each device and other devices is performed by the interface 740 having a communication function and communicating according to the control of the CPU 710.

[0204] When the authentication device 100 is implemented in the computer 700, the operations of the authentication target image acquisition unit 101, the comparison image acquisition unit 102, the image enlargement unit 103, the feature extraction unit 104, the feature conversion unit 105, and the feature comparison unit 106 are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory device 720, and executes the above processes according to the program.

[0205] Furthermore, the CPU 710 reserves memory in the main memory 720 for processing the authentication device 100 according to the program. Communication between the authentication device 100 and other devices is performed by the interface 740 having a communication function and communicating according to the control of the CPU 710. Interaction between the authentication device 100 and the user is performed by the interface 740 equipped with a display device and an input device, displaying various images according to the control of the CPU 710 and accepting user operations.

[0206] When the feature transformation learning device 200 is implemented in the computer 700, the image batch acquisition unit 201, image reduction unit 202, image enlargement unit 103, feature extraction unit 104, feature transformation unit 105, loss function calculation unit 203, learning control unit 208, and the operation of each of these units are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main memory device 720, and executes the above processing according to the program.

[0207] Furthermore, the CPU 710 allocates memory in the main memory 720 for processing by the feature transformation learning device 200 according to the program. Communication between the feature transformation learning device 200 and other devices is performed by the interface 740, which has a communication function and communicates according to the control of the CPU 710. Interaction between the feature transformation learning device 200 and the user is performed by the interface 740, which is equipped with a display device and an input device, and displays various images according to the control of the CPU 710 and accepts user operations.

[0208] When the authentication device 411 is implemented in the computer 700, the operations of the authentication target image acquisition unit 101, the comparison image acquisition unit 102, the feature extraction unit 104, the feature conversion unit 105, and the feature comparison unit 106 are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory device 720, and executes the above processes according to the program.

[0209] Furthermore, the CPU 710 reserves memory in the main memory 720 for processing by the authentication device 411 according to the program. Communication between the authentication device 411 and other devices is performed by the interface 740 having a communication function and communicating under the control of the CPU 710. Interaction between the authentication device 411 and the user is performed by the interface 740 equipped with a display device and an input device, displaying various images under the control of the CPU 710 and accepting user operations.

[0210] When the feature transformation learning device 412 is implemented in the computer 700, the image batch acquisition unit 413, the feature extraction unit 104, the feature transformation unit 105, the loss function calculation unit 203, the learning control unit 208, and the operation of each of these units are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory device 720, and executes the above processing according to the program.

[0211] Furthermore, the CPU 710 allocates memory in the main memory 720 for processing by the feature transformation learning device 412 according to the program. Communication between the feature transformation learning device 412 and other devices is performed by the interface 740, which has a communication function and communicates according to the control of the CPU 710. Interaction between the feature transformation learning device 412 and the user is performed by the interface 740, which is equipped with a display device and an input device, and displays various images according to the control of the CPU 710 and accepts user operations.

[0212] When the feature transformation learning device 610 is implemented in the computer 700, the operations of the image acquisition unit 611, image reduction unit 612, image enlargement unit 613, feature extraction unit 614, feature transformation unit 615, and learning control unit 616 are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory device 720, and executes the above processes according to the program.

[0213] Furthermore, the CPU 710 allocates memory in the main memory 720 for processing by the feature transformation learning device 610 according to the program. Communication between the feature transformation learning device 610 and other devices is performed by the interface 740, which has a communication function and communicates according to the control of the CPU 710. Interaction between the feature transformation learning device 610 and the user is performed by the interface 740, which is equipped with a display device and an input device, and displays various images according to the control of the CPU 710 and accepts user operations.

[0214] When the authentication device 620 is implemented in the computer 700, the operations of the authentication target image acquisition unit 621, the image enlargement unit 622, the feature extraction unit 623, the feature conversion unit 624, and the feature comparison unit 625 are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory device 720, and executes the above processes according to the program.

[0215] Furthermore, the CPU 710 allocates memory space in the main memory 720 for processing the authentication device 620 according to the program. Communication between the authentication device 620 and other devices is performed by the interface 740 having a communication function and communicating under the control of the CPU 710. Interaction between the authentication device 620 and the user is performed by the interface 740 equipped with a display device and an input device, displaying various images under the control of the CPU 710 and accepting user operations.

[0216] Alternatively, a program for executing all or part of the processing performed by the authentication device 100, the feature transformation learning device 200, the authentication device 411, the feature transformation learning device 412, the feature transformation learning device 610, and the authentication device 620 may be recorded on a computer-readable recording medium, and the processing of each part may be performed by having the computer system read and execute the program recorded on this recording medium. The term "computer system" here includes hardware such as the OS (Operating System) and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs (Read Only Memory), CD-ROMs (Compact Disc Read Only Memory), and storage devices such as hard disks built into computer systems. The above-mentioned program may be intended to implement only a part of the functions described above, and may also be able to implement the above-mentioned functions in combination with programs already recorded in the computer system.

[0217] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0218] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0219] (Note 1) An image acquisition means for acquiring a first image, Image reduction means for reducing the first image to a second image with lower resolution than the first image, Image enlargement means for enlarging the second image to a third image with the same resolution as the first image, A feature extraction means for extracting a first feature, which is a feature of the first image, and a second feature, which is a feature of the third image, A feature transformation means that transforms the two aforementioned features into a third feature, A learning control means that causes the feature transformation means to learn a feature transformation method based on the comparison result of the first feature and the third feature, A feature transformation learning device equipped with the following features. (Note 2) The feature discrimination means, upon receiving the aforementioned first feature or the aforementioned third feature as input, determines whether the input feature has been transformed by the feature transformation means. The learning control means causes the feature transformation means to learn a feature transformation method based on a loss function that minimizes the loss when the feature discrimination means cannot distinguish between features transformed by the feature transformation means and other features. The feature transformation learning device described in Appendix 1. (Note 3) The learning control means causes the feature transformation means to learn a feature transformation method based on a loss function in which the loss decreases as the first feature and the third feature become more similar. A feature transformation learning device as described in Appendix 1 or Appendix 2. (Note 4) The learning control means causes the feature transformation means to learn a feature transformation method based on a loss function that minimizes the loss when the class determined by the classification based on the third feature matches the correct class. A feature transformation learning device described in any one of the appendices 1 to 3. (Note 5) The image reduction means reduces the first image to a size determined using random numbers to generate the second image. A feature transformation learning device described in any one of the appendices 1 to 4. (Note 6) The feature extraction means described above is: Attention processing means that weights features corresponding to specific parts of an image from which features are to be extracted. A feature transformation learning device comprising any one of the features described in Appendix 1 to 5. (Note 7) The feature transformation means performs feature transformation on the intermediate features generated by the feature extraction means. A feature transformation learning device described in any one of the appendices 1 to 6. (Note 8) A means for obtaining an image to be authenticated, Image enlargement means for enlarging the image to be authenticated, A feature extraction means that extracts the feature quantities of an enlarged image of the image to be authenticated using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated, A feature transformation means that transforms the features of the enlarged image of the image to be authenticated based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference between the transformed features from the features of the training image and the features of the degraded image obtained by scaling down and then enlarging the training image is small. A feature comparison means for comparing the feature quantities converted by the feature conversion means with the feature quantities of the comparison image, An authentication device equipped with the following features. (Note 9) Image acquisition means for acquiring a first image, which is an image of the iris of the naked eye, and a second image, which is an image of the iris when wearing glasses. Feature quantity extraction means for extracting a first feature quantity that is a feature quantity of the first image and a second feature quantity that is a feature quantity of the second image; Feature quantity conversion means for converting the second feature quantity into a third feature quantity; Learning control means for causing the feature quantity conversion means to learn a feature quantity conversion method based on a comparison result between the first feature quantity and the third feature quantity; A feature quantity conversion learning device comprising: (Appendix 10) Authentication target image acquisition means for acquiring an image of the iris when wearing glasses as an authentication target image; Feature quantity extraction means for extracting the feature quantity of the authentication target image by the same method as the feature quantity extraction method for a comparison image that is an image of the iris without glasses; Feature quantity conversion means for converting the feature quantity of the authentication target image based on the learning result of the feature quantity conversion method using a loss function in which the value becomes smaller as the difference between the converted feature quantity from the feature quantity of the first learning image that is an image of the iris when wearing glasses and the feature quantity of the second learning image that is an image of the iris without glasses becomes smaller; Feature quantity comparison means for comparing the feature quantity after conversion by the feature quantity conversion means with the feature quantity of the comparison image; An authentication device comprising: (Appendix 11) Image acquisition means for acquiring a first image that is an image of the iris taken from the front and a second image that is an image of the iris taken obliquely; Feature quantity extraction means for extracting a first feature quantity that is a feature quantity of the first image and a second feature quantity that is a feature quantity of the second image; Feature quantity conversion means for converting the second feature quantity into a third feature quantity; Learning control means for causing the feature quantity conversion means to learn a feature quantity conversion method based on a comparison result between the first feature quantity and the third feature quantity; A feature quantity conversion learning device comprising: (Appendix 12) Authentication target image acquisition means for acquiring an image of the iris taken obliquely as an authentication target image; Feature quantity extraction means for extracting the feature quantity of the authentication target image by the same method as the feature quantity extraction method for a comparison image which is an image of the iris photographed from the front; Feature quantity conversion means for converting the feature quantity of the authentication target image based on the learning result of the feature quantity conversion method, using a loss function such that the smaller the difference between the feature quantity after conversion from the feature quantity of the first learning image which is an image of the iris photographed obliquely and the feature quantity of the second learning image which is an image of the iris photographed from the front, the smaller the value; Feature quantity comparison means for comparing the feature quantity after conversion by the feature quantity conversion means with the feature quantity of the comparison image; An authentication device comprising the above. (Appendix 13) Image acquisition means for acquiring a first image which is an image of the iris photographed with light other than visible light and a second image which is an image of the iris photographed with visible light; Feature quantity extraction means for extracting a first feature quantity which is the feature quantity of the first image and a second feature quantity which is the feature quantity of the second image; Feature quantity conversion means for converting the second feature quantity into a third feature quantity; Learning control means for causing the feature quantity conversion means to learn the feature quantity conversion method based on the comparison result between the first feature quantity and the third feature quantity; A feature quantity conversion learning device comprising the above. (Appendix 14) Authentication target image acquisition means for acquiring an image of the iris photographed with visible light as the authentication target image; Feature quantity extraction means for extracting the feature quantity of the authentication target image by the same method as the feature quantity extraction method for a comparison image which is an image of the iris photographed with light other than visible light; Feature quantity conversion means for converting the feature quantity of the authentication target image based on the learning result of the feature quantity conversion method, using a loss function such that the smaller the difference between the feature quantity after conversion from the feature quantity of the first learning image which is an image of the iris photographed with visible light and the feature quantity of the second learning image which is an image of the iris photographed with light other than visible light, the smaller the value; Feature quantity comparison means for comparing the feature quantity after conversion by the feature quantity conversion means with the feature quantity of the comparison image; An authentication device comprising the above. (Note 15) An image acquisition means that acquires a first image, which is an image of the iris taken in focus, and a second image, which is an image of the iris taken out of focus. A feature extraction means for extracting a first feature, which is a feature of the first image, and a second feature, which is a feature of the second image, A feature transformation means that transforms the two aforementioned features into a third feature, A learning control means that causes the feature transformation means to learn a feature transformation method based on the comparison result of the first feature and the third feature, A feature transformation learning device equipped with the following features. (Note 16) A means for acquiring an image to be authenticated, which acquires an image of the iris taken in a blurred state as the image to be authenticated, A feature extraction means that extracts the features of the aforementioned image to be authenticated using the same method as the feature extraction method for a comparison image, which is an image of the iris taken in focus. A feature transformation means transforms the features of the image to be authenticated based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference between the transformed features from a first training image (an image of the iris taken in focus) and the features from a second training image (an image of the iris taken in focus) is smaller. A feature comparison means for comparing the feature quantities converted by the feature conversion means with the feature quantities of the comparison image, An authentication device equipped with the following features. (Note 17) To obtain the first image, The first image is reduced to a second image with a lower resolution than the first image, Enlarging the second image to a third image with the same resolution as the first image, Extracting the first feature, which is a feature of the first image, and the second feature, which is a feature of the third image, Converting the aforementioned two features into a third feature, Based on the comparison results between the aforementioned first feature and the aforementioned third feature, a feature transformation method is learned to transform the aforementioned second feature into the aforementioned third feature. A feature transformation learning method that includes this. (Note 18) Obtaining the image to be authenticated, Enlarging the aforementioned image to be authenticated, The feature quantities of the enlarged image of the image to be authenticated are extracted using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated. Based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference between the transformed features from the training image and the features of the degraded image (which is obtained by scaling down the training image) decreases, the features of the enlarged image of the image to be authenticated are transformed. The process involves comparing the feature quantities obtained by converting the feature quantities from the enlarged image of the image to be authenticated with the feature quantities of the comparison image, Authentication methods including those mentioned above. (Note 19) On the computer, To obtain the first image, The first image is reduced to a second image with a lower resolution than the first image, Enlarging the second image to a third image with the same resolution as the first image, Extracting the first feature, which is a feature of the first image, and the second feature, which is a feature of the third image, Converting the aforementioned two features into a third feature, Based on the comparison results between the aforementioned first feature and the aforementioned third feature, a feature transformation method is learned to transform the aforementioned second feature into the aforementioned third feature. A recording medium that stores a program in order to execute it. (Note 20) On the computer, Obtaining the image to be authenticated, Enlarging the aforementioned image to be authenticated, Extracting the feature amount of the enlarged image of the authentication target image by the same method as the feature amount extraction method for a comparison image having a higher resolution than the authentication target image; Converting the feature amount of the enlarged image of the authentication target image based on the learning result of the feature amount conversion method using a loss function such that the smaller the difference between the converted feature amount from the feature amount of the learning image and the feature amount of the deteriorated image obtained by reducing and then enlarging the learning image, the smaller the value; Comparing the converted feature amount from the feature amount of the enlarged image of the authentication target image with the feature amount of the comparison image; A recording medium for recording a program for executing the above.

Industrial Applicability

[0220] This disclosure may be applied to a feature amount conversion learning device, an authentication device, a feature amount conversion learning method, an authentication method, and a recording medium.

Explanation of Signs

[0221] 100, 411, 620 Authentication device 101 Authentication target image acquisition unit 102 Comparison image acquisition unit 103, 613, 622 Image enlargement unit 104, 114, 614, 623 Feature amount extraction unit 105, 615, 624 Feature amount conversion unit 106, 625 Feature amount comparison unit 114a First feature amount extraction unit 114b Second feature amount extraction unit 200, 412, 610 Feature amount conversion learning device 201, 413 Image batch acquisition unit 202, 612 Image reduction unit 203 Loss function calculation unit 204 Reconstruction loss calculation unit 205 Feature discrimination unit 206 Adversarial loss calculation unit 207 Similarity loss calculation unit 208, 616 Learning control unit 301, 303, 304 Attention Processing Units 302 Attention Block 311, 311a, 311b 2D convolution operation unit 312 Function Calculation Section 313 Multiplication part 321 Channel separation unit 322 Channel coupling section 341 Attention Map Acquisition Unit 611 Image acquisition unit

Claims

1. Image acquisition means for acquiring a first image, Image reduction means for reducing the first image to a second image with lower resolution than the first image, Image enlargement means for enlarging the second image to a third image with the same resolution as the first image, A feature extraction means for extracting a first feature, which is a feature of the first image, and a second feature, which is a feature of the third image, A feature transformation means that transforms the two aforementioned features into a third feature, A learning control means that causes the feature transformation means to learn a feature transformation method based on the comparison result between the first feature and the third feature, using a loss function that minimizes the loss when the class determined by the classification based on the third feature matches the correct class. Equipped with, The feature extraction means includes an attention processing means that performs weighting to emphasize features corresponding to a specific part of the image from which features are to be extracted. Feature transformation learning device.

2. The feature transformation means performs feature transformation on the intermediate features generated by the feature extraction means. The feature transformation learning device according to claim 1.

3. A means for obtaining an image to be authenticated, Image enlargement means for enlarging the image to be authenticated, A feature extraction means that extracts the feature quantities of an enlarged image of the image to be authenticated using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated, A feature transformation means that transforms the features of the enlarged image of the image to be authenticated based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference between the features of the training image and the transformed features of the degraded image obtained by enlarging the training image after it has been reduced in size decreases. A feature comparison means for comparing the feature quantities converted by the feature conversion means with the feature quantities of the comparison image, Equipped with, The feature extraction means includes an attention processing means that performs weighting to emphasize features corresponding to a specific part of the image from which features are to be extracted. Authentication device.

4. To obtain the first image, The first image is reduced to a second image with a lower resolution than the first image, Enlarging the second image to a third image with the same resolution as the first image, Extracting the first feature, which is a feature of the first image, and the second feature, which is a feature of the third image, Converting the two aforementioned features into a third feature, The method involves learning a feature transformation method that transforms the second feature into the third feature based on the comparison result between the first and third features, using a loss function that minimizes the loss when the class determined by the classification based on the third feature matches the correct class. Includes, Extracting the first and third features includes weighting to emphasize features corresponding to specific parts of the image from which features are to be extracted. Feature transformation learning method.

5. Obtaining the image to be authenticated, Enlarging the aforementioned image to be authenticated, The feature quantities of the enlarged image of the image to be authenticated are extracted using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated. Based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference between the features of the training image and the features of the degraded image (which is obtained by scaling down the training image and then scaling it up) decreases, the features of the enlarged image of the image to be authenticated are transformed. The process involves comparing the feature quantities obtained by converting the feature quantities from the enlarged image of the image to be authenticated with the feature quantities of the comparison image, Includes, Extracting features from the enlarged image includes weighting to emphasize features corresponding to specific parts of the image from which features are to be extracted. Authentication method.

6. On the computer, To obtain the first image, The first image is reduced to a second image with a lower resolution than the first image, Enlarging the second image to a third image with the same resolution as the first image, Extracting the first feature, which is a feature of the first image, and the second feature, which is a feature of the third image, Converting the two aforementioned features into a third feature, The method involves learning a feature transformation method that transforms the second feature into the third feature based on the comparison result between the first and third features, using a loss function that minimizes the loss when the class determined by the classification based on the third feature matches the correct class. Make it run, In extracting the first and third features, the system performs weighting to emphasize features corresponding to specific parts of the image from which features are to be extracted. A program for that purpose.

7. On the computer, Obtaining the image to be authenticated, Enlarging the aforementioned image to be authenticated, The feature quantities of the enlarged image of the image to be authenticated are extracted using the same method as the feature extraction method for a comparison image with a higher resolution than the image to be authenticated. Based on the learning results of a feature transformation method that uses a loss function whose value decreases as the difference between the features of the training image and the features of the degraded image (which is obtained by scaling down the training image and then scaling it up) decreases, the features of the enlarged image of the image to be authenticated are transformed. The process involves comparing the feature quantities obtained by converting the feature quantities from the enlarged image of the image to be authenticated with the feature quantities of the comparison image, Make it run, In extracting features from the enlarged image, the system performs weighting to emphasize features corresponding to specific parts of the image from which features are to be extracted. A program for that purpose.

Images