Personal authentication device, control method, and program

Abstract

To suppress a decrease in the authentication accuracy due to exposure and reflection of external light on an eyeball.SOLUTION: A personal authentication apparatus comprises: first imaging means that captures an eyeball image; first image processing means that acquires brightness information of the eyeball image captured by the first imaging means and generates divided eyeball images obtained by dividing a region of the eyeball image on the basis of the brightness information of the eyeball image; first storage means that stores brightness information of the divided eyeball images and position information of the divided eyeball images in relation to the eyeball image in association with each other; and learning means that executes different types of learning processing respectively for the divided eyeball images in accordance with the brightness information and the position information of the divided eyeball images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a personal authentication technique using an eyeball image. [Background technology]

[0002] Iris authentication is known as a personal authentication technology that uses eyeball images. Iris authentication is intended for use indoors where there is little external light, but it is also installed in electronic devices such as smartphones that are often used outdoors. When iris authentication is performed under ambient light that contains a large amount of near-infrared light, such as sunlight, the near-infrared light emitted by the sun or reflected by objects illuminated by the sun is reflected over a wide area on the eyeball. Therefore, when performing iris authentication under ambient light, the iris code calculated from the eyeball image captured under ambient light may not match the pre-registered iris code, resulting in authentication failure.

[0003] Patent Document 1 proposes a technology that improves the accuracy of iris authentication even when the illuminance changes depending on the environment during iris authentication, which in turn changes the size of the pupil.In Patent Document 1, iris data with changed pupils is registered in advance, and the brightness of the iris image captured by lighting provided in the device during iris authentication is changed to create the same conditions as when the registered iris data was generated, thereby improving the accuracy of iris authentication. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-73369 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method of changing the brightness of the iris image as in Patent Document 1 cannot deal with cases where areas of different brightness appear in the iris image depending on how external light hits it, or where some pixels in the iris image become saturated due to reflection of external light.

[0006] The present invention has been made in view of the above-mentioned problems, and its purpose is to realize a technology that suppresses a decrease in authentication accuracy due to the way external light hits the eyeball or glare. [Means for solving the problem]

[0007] In order to solve the above problems and achieve the object, the personal authentication device of the present invention comprises a first imaging means for capturing an eyeball image, a first image processing means for acquiring brightness information of the eyeball image captured by the first imaging means and generating a divided eyeball image by dividing an area of ​​the eyeball image based on the brightness information of the eyeball image, a first storage means for storing the brightness information of the divided eyeball image in association with position information of the divided eyeball image relative to the eyeball image, and a learning means for performing a different learning process for each divided eyeball image depending on the brightness information and the position information of the divided eyeball image. [Effects of the Invention]

[0008] According to the present invention, it is possible to suppress a decrease in authentication accuracy caused by the way external light strikes the eyeball or by glare. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of the learning device according to the present embodiment. [Figure 2] 2 is a block diagram showing the hardware configuration of the inference device of the present embodiment. [Figure 3] FIG. 2 is a diagram showing the software configuration of the learning device of the present embodiment. [Figure 4] 4 is a flowchart showing a learning process performed by the learning device of the present embodiment. [Figure 5] FIG. 2 is a diagram showing the software configuration of the inference device of the present embodiment. [Figure 6] 10 is a flowchart showing an inference process for personal registration performed by the inference device of this embodiment. [Figure 7] 10 is a flowchart showing an inference process for personal authentication performed by the inference device of this embodiment. [Figure 8] 1A and 1B are diagrams illustrating an example of external light being reflected on the eyeball. [Figure 9] FIG. 10 is a diagram showing an example of dividing an area of ​​an eyeball image. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] In this embodiment, an example will be described in which the personal authentication device is applied to a camera equipped with a viewfinder, but the invention is not limited to this. For example, the invention can also be applied to cases in which biometric authentication using an image of a person's eyeball is used to authenticate whether or not the person is the person when unlocking an electronic device such as a smart device or tablet device, or when logging in to an online service, etc.

[0012] The personal authentication device of this embodiment captures an image of a person's eyeball, performs different learning processes for each area of ​​the eyeball image that has different brightness depending on how external light hits the eyeball and reflections, and performs inference processing using the results of the learning processes to compare the feature amounts of the eyeball image of a registered person with the feature amounts of the eyeball image of the person to be authenticated, thereby preventing a decrease in the accuracy and reliability of personal authentication.

[0013] In this embodiment, the eyeball image is an iris image obtained by capturing an image of a person's iris and pupil. The feature amount of the eyeball image is unique information obtained from the pattern of the iris. The external light refers to light incident on the eyeball from sunlight, lighting equipment, etc. The reflection of external light on the eyeball refers to the formation of bright and dark areas in the iris image due to the external light.

[0014] The personal authentication device of this embodiment includes a learning device 100 and an inference device 200 that perform machine learning such as deep learning using a convolutional neural network (hereinafter referred to as CNN). Note that the machine learning algorithm is not limited to deep learning using a CNN, and may be an algorithm such as linear regression, decision tree, random forest, logistic regression, support vector machine, naive Bayes, k-nearest neighbor algorithm, clustering, or association analysis. The learning device 100 calculates CNN weight information using an eyeball image as input. The inference device 200 performs a personal registration inference process that calculates feature amounts of the eyeball image of a person to be registered using the CNN weight information generated by the learning device 100 and registers the calculated feature amounts, and a personal authentication inference process that performs personal authentication by comparing feature amounts of the registered eyeball image with feature amounts of the eyeball image of a person to be authenticated.

[0015] [Configuration of Learning Device 100] First, with reference to FIG. 1, the configuration and functions of the learning device 100 in the personal authentication device of this embodiment will be described.

[0016] The learning device 100 includes an imaging unit 101 , a storage unit 102 , an image processing unit 103 , and a learning unit 104 .

[0017] The imaging unit 101 includes an eyepiece 10 for observing the image of the eyeball of a person 1 looking into a viewfinder. The eyeball image transmitted through the eyepiece 10 is formed on the imaging surface of an eyeball imaging element 11, which has photoelectric conversion elements such as CMOS arranged two-dimensionally.

[0018] The storage unit 102 is a storage medium such as a memory card or hard disk that stores a machine learning program, eyeball images captured by the imaging unit 101, or eyeball images captured by an external device such as a digital camera.

[0019] The image processing unit 103 is a processor that divides an eyeball image stored in the storage unit 102 into regions based on the brightness of the iris region. Specifically, the image processing unit 103 detects the brightness value of each pixel in the eyeball image, calculates a brightness pattern corresponding to a region in which pixels with equivalent brightness values ​​within a predetermined brightness range are distributed, and generates divided eyeball images by dividing the region of the eyeball image according to the brightness pattern. The image processing unit 103 also calculates positional information of each divided eyeball image relative to the eyeball image and positional information of the iris portion of the divided eyeball image relative to the iris of the eyeball image as the positional relationship between the eyeball image and the divided eyeball images, and stores the calculated information in the storage unit 102. The image processing unit 103 also calculates brightness information of the divided eyeball images, for example, an average brightness value of the divided eyeball images, and stores the calculated information in the storage unit 102. A method for dividing an eyeball image by the image processing unit 103 will be described later.

[0020] The learning unit 104 is a computer including a processor and memory that calculates CNN weight information based on the segmented eyeball images stored in the storage unit 102 and brightness information and position information of the segmented eyeball images. The learning unit 104 learns eyeball images based on the segmented eyeball images corresponding to the brightness pattern and the brightness information and position information of the segmented eyeball images. The brightness information includes the brightness direction (the direction in which external light enters the eyeball) and intensity (the intensity that causes differences in brightness in the image). The brightness direction is obtained by using a GPS or the like to determine the position and direction of the imaging unit 101 relative to the user's eyeball. The CNN weight information is a coefficient for changing the weight of the inference process for each segmented eyeball image in the inference unit 204 of the inference device 200, which will be described later. In this embodiment, the learning device 100 is assumed to be online connected to a CNN or a network such as the Internet. The software configuration and processing of the learning device 100 will be described later.

[0021] [Configuration of the inference device 200] Next, the configuration and functions of the inference device 200 in the personal authentication device of this embodiment will be described with reference to FIG.

[0022] The inference device 200 performs an inference process for personal registration, which registers the features of the eyeball image obtained by inferring the eyeball image of the person to be registered, and an inference process for personal authentication, which compares the features of the eyeball image of the registered person with the features of the eyeball image of the person to be authenticated, and performs personal authentication.

[0023] The inference device 200 includes an imaging unit 201, a storage unit 202, an image processing unit 203, an inference unit 204, and an authentication unit 205. Note that the imaging unit 201, the storage unit 202, and the image processing unit 203 are similar to the imaging unit 101, the storage unit 102, and the image processing unit 103 of the learning device 100, and therefore details thereof will be omitted.

[0024] The inference unit 204 is a computer including a processor and a memory that calculates feature amounts of eyeball images for personal enrollment and personal authentication using CNN weight information calculated by performing different learning processes for each segmented eyeball image in the learning device 100. In the inference process for personal enrollment, the inference unit 204, like the learning device 100, captures an eyeball image of a person to be enrolled using the imaging unit 201 and generates segmented eyeball images by dividing the eyeball image into regions according to brightness information using the image processing unit 203. Then, the inference unit 204 calculates feature amounts of the eyeball image of the person to be enrolled based on the segmented eyeball image and the brightness information and position information of the segmented eyeball image, generates an inference model associated with unique information such as the name and date of birth of the person to be enrolled, and stores the generated inference model in the storage unit 202 (registration of the inference model). In the inference process for personal authentication, like the learning device 100, the inference unit 204 captures an eyeball image of a person to be authenticated using the imaging unit 201 and generates segmented eyeball images by dividing the eyeball image into regions according to brightness information using the image processing unit 203. Then, the inference unit 204 compares the inference model of the person registered by the inference process for personal registration with the features calculated by inference from the eyeball image of the person to be authenticated, and stores the comparison result in the memory unit 202.

[0025] The authentication unit 205 is a computer including a processor and memory that performs personal authentication to determine whether a person to be authenticated is a registered person based on the matching result stored in the storage unit 202 by the personal authentication inference process, and stores the authentication result in the storage unit 202. Personal authentication is performed by determining whether there are a predetermined number or more of segmented eyeball images whose degree of match is a predetermined value or more, or whether the total area of ​​the segmented eyeball images whose degree of match is a predetermined value or more is a predetermined value or more. Alternatively, the reliability ratio of the inference result for each segmented eyeball image may be variably set according to the brightness of the segmented eyeball image (the ratio of reliable inference results may be set higher), and personal authentication may be performed by prioritizing highly reliable inference results over less reliable inference results.

[0026] The notification unit 206 notifies the person to be authenticated of the authentication result of the authentication unit 205. Examples of notification methods include displaying the result on a display (not shown) provided in the personal authentication device or an audio notification from a speaker, or displaying the result on an external display connected to the personal authentication device via a dedicated cable or an audio notification from an external speaker.

[0027] An example of external light being reflected in an eyeball image will now be described with reference to Fig. 8. In this embodiment, an example of capturing an eyeball image of a user looking into a finder will be described.

[0028] As shown in Figure 8, eye images captured under ambient light are affected by glare over a wide area. The infrared component of sunlight is intense, and not only direct sunlight but also infrared components from objects that reflect sunlight enter the eye from various angles. For example, Figure 8(a) shows an example of glare when light enters the eye at an angle close to directly to the eye as it looks into the viewfinder. In Figure 8(a), the intensity of the incident infrared light is weak, and the iris pattern is not obscured. However, because the eyeball is spherical, eye images are captured in which the brightness of the iris region varies. Furthermore, as shown in Figures 8(a) to 8(d), depending on the direction of incidence of the infrared light, there are areas where the iris pattern cannot be captured, or where bright and dark areas appear in the iris region.

[0029] As described above, the brightness of the iris region and the distribution of light and dark areas differ for each eyeball image depending on the direction of incident external light, which leads to false detections and makes it difficult to obtain accurate personal authentication results with conventional personal authentication methods. Furthermore, as shown in Figures 8(e) to 8(h), when strong infrared light contained in external light is incident on the eyeball, a brightness pattern with saturated brightness values ​​appears in the iris region corresponding to the direction of incident external light. In such cases, part of the iris pattern disappears, making it impossible to obtain eyeball image features, making it difficult to obtain accurate personal authentication results with conventional personal authentication methods.

[0030] 9 shows an example in which the iris region of an eyeball image is divided according to brightness. The image processing units 103 and 203 detect the luminance value of each pixel in the eyeball image, calculate a luminance pattern corresponding to a region in which pixels with equivalent luminance values ​​within a predetermined luminance range are distributed, and divide the region of the eyeball image according to the luminance pattern.

[0031] Figure 9(a) shows an example of an eyeball image when infrared light is incident on the eyeball from the left side as viewed from the eyeball. In Figure 9(a), the brightness of the pixels on the left side of the eyeball (the right side as viewed from the left side in Figure 9(a)) is high, and a bright image is acquired, while a dark image is acquired on the right side of the eyeball (the left side as viewed from the left side in Figure 9(a)). In Figure 9(a), for example, if the iris region is divided into left and right parts centered on the pupil by a brightness threshold, a divided eyeball image is generated by dividing the iris region of the eyeball image into left and right parts.

[0032] When infrared light is incident on the eyeball from diagonally above as in Figure 9(b), one method is to divide the iris region of the eyeball image radially into vertical, horizontal, and vertical regions at 90° intervals around the pupil as the center. Other methods include dividing the iris region of the eyeball image horizontally and vertically as in Figure 9(c), dividing the iris region of the eyeball image into a grid as in Figure 9(d), or dividing the iris region of the eyeball image into smaller angles than the division angle (90°) in Figure 9(b).

[0033] Furthermore, as shown in Figures 8(e) to 8(g), when the intensity of infrared light contained in external light incident on the eyeball is high, some pixel values ​​in the iris region of the eyeball image may exceed a predetermined value and become saturated, resulting in the loss of feature quantities in the iris region of the eyeball image. In this case, as shown in Figure 9(e), for example, edge detection is performed using filter processing, and the luminance values ​​of the edge coordinates or the differential values ​​of the filter processing output are detected to divide the saturated region. Then, regions other than the saturated region are used in the learning process. Alternatively, as shown in Figure 9(f), the eyeball image may be divided vertically from the center, and regions other than the saturated region may be used.

[0034] In this embodiment, brightness information, such as the average brightness value of the divided regions, is calculated and stored for each divided eyeball image of the eyeball image of the person to be trained, the person to be registered, and the person to be authenticated. Position information for each divided eyeball image relative to the eyeball image is also stored. For example, to compare the feature values ​​of the divided eyeball image of the person to be authenticated with the feature values ​​(inference model) of the divided eyeball image of the registered person corresponding to that eyeball image, position information indicating which part of the eyeball image each divided eyeball image corresponds to is stored. For example, when dividing the eyeball image, the location of the pupil is searched for. A method for searching for the pupil is to display an index in the viewfinder through which the person to be registered or the person to be authenticated looks to guide the user's gaze. The guided index is used to limit the eyeball area, and if the pixel at the coordinate with the lowest pixel value within the limited area in the eyeball image has a brightness value below a predetermined threshold, it is considered to be the pupil. If the pupil is successfully detected, the eyeball image is divided based on the detected brightness information, with the pupil as the center. In this way, the positions of the divided eyeball images of the eyeball image centered on the pupil are identified.

[0035] In this way, in the learning process, CNN weight information is calculated based on divided eyeball images obtained by dividing the area of ​​the eyeball image according to brightness information, brightness information for each divided eyeball image, and position information of each divided eyeball image relative to the eyeball image. In the inference process, the CNN weight information calculated in the learning process is used to perform inference process for personal registration and inference process for personal authentication.

[0036] [Software Configuration of Learning Unit 104] Next, the software configuration of the learning unit 104 will be described with reference to FIG.

[0037] The learning image acquisition unit 1040 acquires a divided eyeball image 1042 corresponding to the brightness of the eyeball image generated by the image processing unit 103, and divided image information 1041 including brightness information of the divided eyeball image 1042 and position information of the divided eyeball image 1042 with respect to the eyeball image. The learning unit 104 performs learning processing using the divided eyeball image 1042 as input, calculates CNN weight information 1049, and outputs an output value 1048 corresponding to the divided image information 1041.

[0038] 3, the learning device that performs the learning process is a CNN 1043 that performs the learning process of a neural network. The CNN 1043 includes, for example, a convolution layer 1044, a pooling layer 1045, a fully connected layer 1046, and an output layer 1047.

[0039] The convolution layer 1044 is an example of an input layer, and is a layer that performs a convolution operation on an image. The convolution of an image corresponds to a process of calculating the correlation with a certain filter.

[0040] The pooling layer 1045 is a layer that performs pooling processing. The pooling processing sets the maximum value among the output values ​​of the neurons in the connected convolution layer 1044, and achieves invariance of the response to a small change in the features that appear in the image.

[0041] The fully connected layer 1046 is a layer that connects all neurons between adjacent layers. Each neuron included in the fully connected layer 1046 is connected to all neurons included in the adjacent layer. Although only one fully connected layer 1046 is shown in FIG. 3, the fully connected layer 1046 may be configured with multiple layers.

[0042] The output layer 1047 is the layer arranged on the most output side of the CNN 1043.

[0043] A threshold is set for each neuron, and the output of each neuron is basically determined depending on whether the sum of the products of each input and each weight exceeds the threshold.

[0044] The learning unit 104 constructs a CNN 1043 that, when a segmented eyeball image 1042 is input to a convolutional layer 1044 on the input side of the CNN 1043, outputs an output value 1048 corresponding to segmented image information 1041 from an output layer 1047. In this way, the learning unit 104 acquires information indicating the configuration of the CNN 1043, the connection weights between the neurons, and the thresholds of the neurons, and stores this information in the storage unit 102 as CNN weight information 1049.

[0045] [Learning Process in Learning Device 100] Next, the learning process in the learning device 100 will be described with reference to FIGS.

[0046] The process in Figure 4 is implemented by the computer of learning device 100 executing a program to control each component of learning device 100. The process of each step in Figure 4 may be modified as much as possible. Steps may be omitted, replaced, or added as appropriate depending on the embodiment.

[0047] In step S401, the image capturing unit 101 captures an image of the eyeballs of a person to be studied, and stores the image in the storage unit .

[0048] In step S402, the image processing unit 103 generates a divided eyeball image 1042 according to brightness from the eyeball image acquired in step S401. The image processing unit 103 also generates divided image information 1041 including brightness information of the divided eyeball image 1042 and position information of the divided eyeballs.

[0049] In step S403, the learning unit 104 inputs the divided eyeball image 1042 generated in step S402 and the divided image information 1041 to the learning image acquisition unit 1040.

[0050] In step S404, the learning unit 104 receives the segmented eyeball image 1042 and segmented image information 1041 received in step S403 as input, executes learning processing of the CNN 1043, and calculates CNN weight information 1049 corresponding to the output value 1048.

[0051] Specifically, a CNN 1043 to be subjected to the learning process is prepared. The CNN weight information 1049, which is the configuration of the CNN 1043 prepared here, the initial values ​​of the connection weights between the neurons, and the initial values ​​of the thresholds of the neurons, may be prepared using a template or may be provided by input from an operator. Furthermore, when re-learning is performed, the learning unit 104 may prepare the CNN 1043 based on the CNN weight information 1049 to be re-learned.

[0052] In step S405, the learning unit 104 stores information indicating the configuration of the CNN 1043, the weights of the connections between the neurons, and the threshold value of each neuron, which have been calculated by the learning process in step S404, in the storage unit 102 as CNN weight information 049. As described above, the learning process in the learning device 100 calculates CNN weight information to be used in the inference process, which will be described later, by performing the learning process for each divided eyeball image obtained by dividing the eyeball image according to brightness information, and stores the CNN weight information in the storage unit 202.

[0053] [Software Configuration of Inference Unit 204] Next, the software configuration of the inference unit 204 will be described with reference to FIG.

[0054] The inference image acquisition unit 2040 acquires the divided eyeball image 2042 generated by the image processing unit 203 and divided image information 2041 including brightness information of the divided eyeball image 2042 and position information of the divided eyeball image 2042 relative to the eyeball image. The inference unit 204 performs inference processing for personal registration using the divided eyeball image 2042 of the eyeball image of the person to be registered as input, and outputs eyeball information 2048 corresponding to the divided image information 2041, by using the CNN weight information 1049 calculated by the learning unit 104. The inference unit 204 also performs inference processing for personal authentication using the divided eyeball image 2042 of the eyeball image of the person to be authenticated as input, and outputs eyeball information 2048 corresponding to the divided image information 2041. As shown in FIG. 5, the inference unit that performs inference processing is a CNN 2043 that performs inference processing of a neural network.

[0055] The CNN 2043 has, for example, a convolutional layer 2044, a pooling layer 2045, a fully connected layer 2046, and an output layer 2047, and is configured in the same manner as the CNN 1043 of the learning unit 104. Furthermore, since each of the layers 2044 to 2047 is the same as each of the layers 1044 to 1047 of the CNN 1043 of the learning unit 104, a description thereof will be omitted.

[0056] The configuration of the CNN 2043 used for the personal registration inference process of the person to be registered and the personal authentication inference process of the person to be authenticated, the weight of the connections between each neuron, and the threshold value of each neuron are set by referring to the CNN weight information 1049.

[0057] The inference unit 204 inputs the divided eyeball image 2042 and the divided image information 2041, and uses the CNN weight information 1049 generated by the learning process in the learning device 100 for each divided eyeball image 2042 to calculate the feature amount of the user's eyeball image through the inference process of the CNN 2043, and stores the calculated feature amount in the memory unit 102.

[0058] [Inference Processing for Personal Registration in Inference Device 200] Next, the inference processing for personal registration in the inference device 200 will be described with reference to FIGS.

[0059] The processing in Figure 6 is realized by the computer of inference device 200 executing a program to control each component of inference device 200. The processing of each step in Figure 6 may be changed as much as possible. Steps may be omitted, replaced, or added as appropriate depending on the embodiment. The same applies to Figure 7, which will be described later.

[0060] In step S601, the image capturing unit 201 captures an image of the eyeballs of the person to be registered and stores it in the storage unit 202.

[0061] In step S602, the image processing unit 203 generates a divided eyeball image 2042 according to the brightness from the eyeball image acquired in step S601.

[0062] In step S603, the inference unit 204 receives the segmented eyeball image 2042 generated in step S602.

[0063] In step S604, the inference unit 204 executes inference processing of the CNN 2043 using the segmented eyeball image 2042 input in S603 as an input.

[0064] In step S605, the inference unit 204 acquires eyeball information 2048 corresponding to the feature amounts of the divided eyeball image 2042 of the person to be registered for each divided eyeball image 2042 from the CNN 2043, using the CNN weight information 1049 calculated by the learning process in the learning device 100. The information acquired from the CNN 2043 is, for example, eyeball information 2048 that associates the feature amounts of the divided eyeball image with divided image information 2041 that includes brightness information and position information of the divided eyeball image.

[0065] In step S606, the inference unit 204 generates an inference model from the eyeball information 2048 obtained in step S605 for each segmented eyeball image.

[0066] In step S607, the inference unit 204 stores the inference model generated in step S606 in the memory unit 202.

[0067] As described above, in the inference process for personal registration in the inference device 200, eyeball information is obtained from a divided eyeball image obtained by dividing the eyeball image of the person to be registered according to brightness information through CNN inference process using CNN weight information 1049 calculated by the learning process in the learning device 100, and an inference model of the person to be registered is stored in the memory unit 202.

[0068] [Inference Processing for Personal Authentication in Inference Device 200] Next, the inference processing for personal authentication in the inference device 200 will be described with reference to FIGS.

[0069] The software configuration of the inference unit 204 in the inference process for personal authentication is the same as the software configuration of the inference unit 204 in the inference process for personal registration.

[0070] In step S701, the image capturing unit 201 captures an image of the eyeballs of the person to be authenticated and stores it in the storage unit 202.

[0071] In step S702, the image processing unit 203 generates a divided eyeball image 2042 according to the brightness from the eyeball image acquired in step S701.

[0072] In step S703, the inference unit 204 receives the segmented eyeball image 2042 generated in step S702.

[0073] In step S704, the inference unit 204 executes the inference logic of the CNN 2043 using the segmented eyeball image 2042 input in step S703 as an input.

[0074] In step S705, the inference unit 204 acquires eyeball information 2048 corresponding to the feature amounts of the divided eyeball image of the person to be authenticated for each divided eyeball image from the CNN 2043 by using the CNN weight information 1049 calculated by the learning process in the learning device 100. The eyeball information 2048 acquired from the CNN 2043 is, for example, information that associates the feature amounts of the divided eyeball image with divided image information 2041 that includes brightness information and position information of the divided eyeball image.

[0075] In step S706, the authentication unit 205 reads out the inference model generated in step S606 of Figure 6 from the memory unit 202. The authentication unit 205 also inputs the eyeball information 2048 acquired in step S705. The authentication unit 205 then compares the inference model with the eyeball information 2048.

[0076] In step S707, the authentication unit 205 performs authentication processing to determine whether the person to be authenticated is a person registered in the inference model based on the results of the comparison in step S706.

[0077] In step S708, the notification unit 206 notifies the person to be authenticated of the authentication result determined in step S707.

[0078] As described above, in the inference processing for personal authentication in the inference device 200, eyeball information is obtained from a divided eyeball image obtained by dividing the eyeball image of the person to be authenticated according to brightness information through CNN inference processing using CNN weight information 1049 calculated by the learning processing in the learning device 100, and the authentication result obtained by comparing the eyeball information with a registered inference model is notified to the person to be authenticated.

[0079] As described above, according to the personal authentication device of this embodiment, by performing learning processing, inference processing for personal registration, and inference processing for personal authentication for each divided eyeball image generated according to the brightness information of the eyeball image, it is possible to suppress a decrease in authentication accuracy due to the way external light hits the eyeball or reflections.

[0080] [Other embodiments] In the above-described embodiment, an example was described in which the learning device 100 and the inference device 200 of the personal authentication device are configured separately, but they may also be configured as an integrated unit in which the imaging unit, memory unit, and image processing unit are used in common for the learning process and the inference process.

[0081] The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more of the functions.

[0082] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0083] 100... learning device, 101... imaging unit, 102... storage unit, 103... image processing unit, 104... learning unit, 200... inference device, 201... imaging unit, 202... storage unit, 203... image processing unit, 204... inference unit, 205... authentication unit, 206... notification unit

Claims

1. a first imaging means for capturing an eyeball image; a first image processing means for acquiring brightness information of the eyeball image captured by the first imaging means, and generating a divided eyeball image by dividing an area of ​​the eyeball image based on the brightness information of the eyeball image; a first storage means for storing brightness information of the divided eyeball images and position information of the divided eyeball images relative to the eyeball image in association with each other; a learning means for executing a different learning process for each of the divided eyeball images in accordance with brightness information and position information of the divided eyeball images.

2. a second imaging means for capturing an eyeball image; a second image processing means for acquiring brightness information of the eyeball image captured by the second imaging means, and generating a divided eyeball image by dividing an area of ​​the eyeball image based on the brightness information of the eyeball image; a second storage means for storing brightness information of the divided eyeball images and position information of the divided eyeball images relative to the eyeball image in association with each other; an inference means for performing an inference process for each of the segmented eyeball images of the person to be registered and the person to be authenticated, which are captured by the second imaging means, using weight information of a learning device obtained by the learning process for each of the segmented eyeball images by the learning means, and calculating feature amounts of the segmented eyeball images; an authentication means for performing an authentication process of comparing the feature amount for each of the segmented eyeball images of the person to be registered calculated by the inference means with the feature amount for each of the segmented eyeball images of the person to be authenticated; 2. The personal authentication device according to claim 1, further comprising a notification unit that notifies a result of the authentication process performed by the authentication unit.

3. 3. The personal authentication device according to claim 1, wherein the learning means performs the learning process on areas other than a predetermined area when the eyeball image has a predetermined area in which pixel values ​​exceed a predetermined value.

4. 3. The personal authentication device according to claim 2, wherein the first image processing means and the second image processing means set areas to divide the eyeball image based on a distribution of luminance values ​​detected from the eyeball image.

5. The brightness information includes a direction and intensity of external light incident on the eyeball, 3. The personal authentication device according to claim 2, wherein the learning means calculates weights to be used when the inference means executes machine learning for each of the segmented eyeball images.

6. The personal authentication device according to claim 2, wherein the authentication means performs the authentication process by determining whether there are a predetermined number or more of divided eyeball images whose degree of matching is a predetermined value or more, or whether the total area of ​​the divided eyeball images whose degree of matching is a predetermined value or more is a predetermined value or more.

7. capturing an eye image; acquiring brightness information of the captured eyeball image, and generating a divided eyeball image by dividing an area of ​​the eyeball image based on the brightness information of the eyeball image; storing brightness information of the divided eyeball images and position information of the divided eyeball images relative to the eyeball image in association with each other; and executing a different learning process for each of the divided eyeball images in accordance with brightness information and position information of the divided eyeball images.

8. A program for causing a computer to function as each of the means of the personal authentication device according to any one of claims 1 to 6.

Images