Information processing device, information processing method, and program

Abstract

The present invention extracts respective feature quantities of a plurality of regions cut out from a region of a target's eye included in an acquired image. A similarity weight for each region is identified, the similarity weight being calculated on the basis of the respective feature quantities of the plurality of regions and feature quantities which relate to the corresponding regions and are prestored for the target. The respective feature quantities of the plurality of regions and the weights identified corresponding to the feature quantities are used to calculate the similarity between the feature quantity of the target's eye included in the acquired image and the pre-stored feature quantity of the target's eye.

Description

Information processing device, information processing system, information processing method, and storage medium

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a storage medium.

[0002] There is an ensemble estimation method that generates multiple estimators and outputs a predetermined estimation result for an input using these multiple different estimators. In this ensemble estimation method, each of the multiple individual estimators performs estimation using an estimation model obtained by training using the same or different datasets. When calculating the estimation result, the estimation results of the individual estimators are integrated and used as the overall estimation result.

[0003] Related techniques are disclosed in Non-Patent Documents 1 to 4. Non-Patent Document 1 discloses a technique (bagging) in which multiple sub-datasets are created from a training dataset by sampling that allows overlap, and then separate weak learners are trained using these sub-datasets.

[0004] Non-Patent Document 2 discloses a technique (boosting) in which, when training a weak learner, the weight of the loss for the training data is determined from the output results of other learners, and the learner learns the data. In this method, for example, a new learner is trained so that it has high discrimination ability for input data on which other learners have produced incorrect estimation results.

[0005] Non-Patent Document 3 discloses a technique in which, when training a weak learner, learning is performed using a partial image obtained by randomly cutting out a part of the original image.

[0006] Non-Patent Document 4 discloses a technology in which there are two weak learners, one that receives an iris image as input and the other that receives an image of the area around the eye as input, and the results of these are integrated to output an estimation result.

[0007] Furthermore, Patent Document 1 discloses a related technique, which is a method for authenticating a subject using a plurality of biometric features, and which uses features of the iris pattern, iris color, and corneal surface.

[0008] Special table 2018-514046 publication

[0009] L. Breiman. Bagging predictors. Machine Learning, 24, 123-140, 1996. RE Schapire. A brief introduction to Boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, 1999. B. Cheng, W. Wu, D. Tao, S. Mei, T. Mao and J. Cheng, "Random Cropping Ensemble Neural Network for Image Classification in a Robotic Arm Grasping System," in IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 9, pp. 6795-6806, Sept. 2020, doi: 10.1109 / TIM.2020.2976420. Oishi, S., Ichino, M., Yoshiura, H.: Fusion of iris and periocular user authentication by AdaBoost for mobile devices.In: 2015 IEEE International Conference on Consumer Electronics (ICCE), 9-12 Jan. 2015, Piscataway, NJ, USA, pp. 428-429. (2015)

[0010] The present disclosure aims to provide an information processing device, an information processing system, an information processing method, and a storage medium that aim to improve upon the above-mentioned prior art documents.

[0011] According to a first aspect of the present disclosure, an information processing device includes: a feature extraction means for extracting features of each of a plurality of regions cut out from an acquired image including the eyes of a target; a weight determination means for determining a weight of similarity for each of the regions calculated based on the features of each of the plurality of regions and each feature related to the corresponding region pre-stored for the target; and a similarity calculation means for calculating a similarity between the features of the eyes of the target included in the acquired image and the features of the eyes of the target stored in advance, based on the features of each of the plurality of regions, each feature related to the corresponding region pre-stored for the target, and the weight.

[0012] According to a second aspect of the present disclosure, an information processing system includes a feature extraction means for extracting features of each of a plurality of regions cut out from an eye region of a subject included in an acquired image; a weight determination means for determining a weight of similarity for each of the regions calculated based on the features of each of the plurality of regions and each feature related to the corresponding region pre-stored for the subject; and a similarity calculation means for calculating a similarity between the features of the eye of the subject included in the acquired image and the pre-stored features of the subject's eye based on the features of each of the plurality of regions, each feature related to the corresponding region pre-stored for the subject, and the weight.

[0013] According to a third aspect of the present disclosure, an information processing method extracts features of each of a plurality of regions cut out from the eye region of a subject included in an acquired image, identifies a similarity weight for each of the regions calculated based on the features of each of the plurality of regions and each feature related to the corresponding region pre-stored for the subject, and calculates a similarity between the eye features of the subject included in the acquired image and the pre-stored eye features of the subject based on the features of each of the plurality of regions, each feature related to the corresponding region pre-stored for the subject, and the weight.

[0014] According to a fourth aspect of the present disclosure, a storage medium stores a program that causes a computer of an information processing device to function as a feature extraction means that extracts features of each of a plurality of regions cut out from the eye region of a subject included in an acquired image; a weight identification means that identifies a weight of the similarity of each of the regions, calculated based on the features of each of the plurality of regions and each feature related to the corresponding region pre-stored for the subject; and a similarity calculation means that calculates the similarity between the features of the eye of a subject included in the acquired image and the features of the eye of the subject pre-stored, based on the features of each of the plurality of regions, each feature related to the corresponding region pre-stored for the subject, and the weights.

[0015] FIG. 1 is a block diagram showing the configuration of the authentication device 1 in the first embodiment. FIG. 2 is a diagram showing an outline of landmark detection processing in the first embodiment. FIG. 3 is a first diagram showing an outline of normalization processing in the first embodiment. FIG. 4 is a second diagram showing an outline of normalization processing in the first embodiment. FIG. 5 is a third diagram showing an outline of normalization processing in the first embodiment. FIG. 6 is a diagram showing an outline of area selection processing in the first embodiment. FIG. 7 is a diagram showing a processing flow of feature amount recording processing performed by the authentication device 1 in the first embodiment. FIG. 8 is a diagram showing a processing flow of authentication processing performed by the authentication device 1 in the first embodiment. FIG. 9 is a first diagram showing an outline of weight specification processing in the first embodiment. FIG. 10 is a second diagram showing an outline of weight specification processing in the first embodiment. FIG. 11 is a block diagram of a function for generating a specified model of weights for authentication scores in the first embodiment. FIG. 12 is a diagram showing a processing flow of processing for generating a specified model of weights for authentication scores in the first embodiment. FIG. 13 is a block diagram showing the configuration of the authentication device 1 in the second embodiment. FIG. 14 is a diagram showing an outline of area selection processing in the second embodiment. FIG. 15 is a diagram showing a processing flow of feature amount recording processing performed by the authentication device 1 in the second embodiment. FIG. 16 is a diagram showing a processing flow of authentication processing performed by the authentication device 1 in the second embodiment.

[0016] An authentication device according to an embodiment of the present disclosure will now be described with reference to the drawings. The authentication device is one aspect of an information processing device.

[0017] 1 is a block diagram showing the configuration of an authentication device 1 according to Embodiment 1. As shown in Fig. 1, the authentication device 1 includes an image acquisition unit 10, a landmark detection unit 11, image region selection units 12.1 and 12.2, feature extraction units 13.1 and 13.2, a matching feature storage unit 14, score calculation units 15.1 and 15.2, a score integration unit 16, an authentication determination unit 17, and a weight specification unit 18.

[0018] The image acquisition unit 10 acquires an image including the iris of the eye and the area surrounding the eye of the person to be authenticated. The iris is the area of ​​the eye's muscle fiber pattern that surrounds the pupil in a circular shape. The iris muscle fiber pattern has characteristics unique to each individual and varies little. The authentication device 1 of this embodiment authenticates the person using iris pattern information. This is called iris authentication. For example, in iris authentication, the authentication device 1 identifies the iris area from an image including the eye and divides the iris area into multiple blocks. The authentication device 1 then extracts and digitizes the features of each block, and performs authentication by comparing them with pre-stored iris features. In this iris authentication process, the authentication device 1 may perform authentication by further comparing luminance change information, which encodes the luminance change between each block and adjacent blocks, with pre-stored luminance change information for the irises of multiple people.

[0019] The landmark detection unit 11 detects landmark information including landmark points and position information of important areas set so as to enable selection of a predetermined partial area related to the eye from the acquired image. In this disclosure, landmark information refers to points indicating the position information and areas of the pupil, iris, and eyelid, as well as shapes such as pupil circles and iris circles. The landmark information represents information including points and circles designed to enable extraction of areas such as the iris and the area around the eye from the eye image. The landmark information is not limited to points and circles, but may also be element information such as lines, ellipses, polygons, and Bezier curves. Furthermore, the landmark information may also be information on shapes created by combining these elements. The image region selection units 12.1 and 12.2 select a partial area including the iris area based on the landmark information detected by the landmark detection unit 11. More specifically, the image region selection unit 12.1 selects the entire circular area including the pupil area inside the outer circle c1 of the iris as the partial area a1. Alternatively, image region selection unit 12.1 may select a donut-shaped region surrounded by an outer circle c1 and an inner circle c2 of the iris as partial region a1. Image region selection unit 12.2 selects partial region a2 including the eyeball and the area around the eye (such as the eyelids). Image region selection units 12.1 and 12.2 are collectively referred to as image region selection unit 12.

[0020] The feature extraction unit 13.1 (13.2) extracts feature values ​​f1 (f2) from the partial area a1 (a2) selected by the image area selection unit 12.1 (12.2). If the partial areas a1 and a2 include a pupil area, the feature values ​​f1 and f2 corresponding to the partial areas a1 and a2 may be extracted by cutting out only the iris area excluding the pupil area. A feature value is a vector value representing the features of the eye, including the iris, which are necessary for iris authentication. The feature extraction units 13.1 and 13.2 are collectively referred to as feature extraction unit 13.

[0021] The matching feature storage unit 14 stores matching features indicating features of a target such as a person registered in advance. The matching feature is, for example, the Mth matching feature among a plurality of matching features of a person registered in advance before authentication, and is the feature extracted by the feature extraction units 13.1 and 13.2 in the pre-feature registration process and recorded in the matching feature storage unit 14.

[0022] The score calculation unit 15.1 (15.2) calculates the score SC1 (score SC2), which is the authentication score SC for each partial region, using the feature f1 (f2) extracted by the feature extraction unit 13.1 (13.2) and the matching feature f1 (f2) stored in the matching feature storage unit 14. The authentication score SC here is the degree of similarity between the matching features f1 and f2 and the corresponding features registered in advance, which is necessary for iris authentication. The score calculation units 15.1 and 15.2 are collectively referred to as score calculation unit 15.

[0023] The score integrating unit 16 calculates the authentication integrated score TSC using the scores SC1 and SC2 obtained from the score calculating units 15.1 and 15.2. When calculating the authentication integrated score TSC, the score integrating unit 16 calculates the authentication integrated score TSC using the weights of the authentication scores SC for each partial region calculated by the weight specifying unit 18.

[0024] The authentication assessment unit 17 makes an authentication assessment based on the integrated authentication score TSC obtained from the score integration unit 16. The weight specification unit 18 specifies weights for the feature amounts when calculating the similarity based on the feature amounts obtained from the features of each partial region and the feature amounts related to the corresponding regions stored in advance for the person to be authenticated.

[0025] The subject to be authenticated by the authentication device 1 of this embodiment may be a human being or an animal such as a dog or a snake.

[0026] FIG. 2 is a diagram illustrating an overview of the landmark detection process. The landmark detection unit 11 may detect the coordinates of each point p on the outline of the eyelid included in the acquired image, the center coordinate O1 of the pupil circle, the center coordinate O2 of the iris circle, the pupil radius r1, and the iris radius r2, and calculate a vector composed of these values ​​as landmark information. The coordinates of point p on the outline of the eyelid (upper eyelid, lower eyelid) included in the acquired image may be relative coordinates with a predetermined position on the eye as the origin. The predetermined position may be a point on the outer corner or the roof of the eye, or the midpoint of a line connecting the points on the outer corner or the roof of the eye.

[0027] 3 is a first diagram showing an overview of the normalization process. The image acquisition unit 10 identifies the outer corner point p1 and inner corner point p2 of the eye in the acquired image (G11), calculates the angle θ between a line L1 passing through these points and the horizontal direction L2 of the image, and generates an image (G12) by rotating the image using the angle θ so that the line L1 connecting the outer corner point and inner corner point of the eye coincides with the horizontal direction L2 of the image. The generation of this rotationally transformed image (G12) is one form of image normalization.

[0028] FIG. 4 is a second diagram illustrating an overview of the normalization process. The image acquisition unit 10 identifies the diameter of the pupil or the diameter of the iris in the eyeball captured in the acquired image (G21), and generates a reduced or enlarged image (G22) so that the diameters of the pupil and the iris are predetermined values. In this case, the image acquisition unit 10 may identify the number of pixels corresponding to the length of the pupil diameter and the number of pixels corresponding to the length of the iris diameter, based on the center coordinates of the pupil circle, and perform image processing such as geometric transformation so that the ratio between the number of pixels corresponding to the length of the iris diameter and the number of pixels corresponding to the length of the pupil diameter is constant, thereby generating the reduced or enlarged image. The generation of this reduced or enlarged image (G22) is one aspect of image normalization.

[0029] FIG. 5 is a third diagram illustrating an overview of the normalization process. The image acquisition unit 10 generates an image (G32) by shifting the position of the eye in the acquired image (G31) so that it is at the center of the image. At this time, the image acquisition unit 10 generates an image (G32) that has been transformed so that the center coordinates of the iris circle are located at predetermined positions within the image and the diameters of the pupil and iris are predetermined values. The generation of this transformed image (G32) is one aspect of image normalization. The image acquisition unit 10 may generate the transformed image (G32) by performing image processing such as geometric transformation so that the number of pixels corresponding to the length of the iris radius, based on the center coordinates of the iris circle, is constant. The generation of this transformed image (G32) is one aspect of image normalization.

[0030] FIG. 6 is a diagram illustrating an overview of the region selection process. The image region selection unit 12 sequentially performs one or more of the processes described above with reference to FIGS. 3, 4, and 5, and then cuts out an image of a predetermined partial region based on landmark information about the eye. As shown in FIG. 6, the image region selection unit 12.1 selects a rectangular partial region a1 including a circular region of the iris' outer circle c1 based on the center position of the iris detected by the landmark detection unit 11. Furthermore, the image region selection unit 12.2 selects a rectangular partial region a2 including the eyeball and the area surrounding the eye based on the center position of the iris detected by the landmark detection unit 11. The partial region a1 is a form of a region that includes at least the iris but does not include the area surrounding the eye (e.g., the eyelid, the outer corner of the eye, the inner corner of the eye, etc.). The partial region a2 is a form of a region that includes both the iris and the area surrounding the eye. The selected partial regions a1 and a2 may have shapes other than rectangles (e.g., circles or other shapes). The image region selection unit 12.1 generates an image a12 by polar coordinate development of the iris included in the partial region a1.

[0031] 7 is a diagram showing a processing flow of the feature amount recording process performed by the authentication device 1 in the first embodiment. Next, the feature amount recording process of the authentication device 1 in the first embodiment will be described with reference to FIG.

[0032] In the preliminary feature recording process, a person acquires a facial image including their eyes, or a partial facial image showing at least a part of their face including the eyes, in the authentication device 1. The authentication device 1 may photograph the person using a specified camera and acquire the image generated during the photographing. The image acquisition unit 10 acquires an image including the person's eyes (step S11). The image is assumed to include at least one or both of the person's eyes. The image is also assumed to include the pupils and irises of the eyes. The image acquisition unit 10 outputs the image to the landmark detection unit 11 and the image area selection units 12.1 and 12.2.

[0033] The landmark detection unit 11 detects landmark information based on the acquired image (step S12). The landmark detection unit 11 may calculate landmark information represented by a vector including the center coordinates and radius values ​​of an iris circle from the acquired image. As described with reference to FIG. 2 , the landmark detection unit 11 may generate eye landmark information represented by a vector using points on the eyelid contour, the center coordinates of the pupil circle, the center coordinates of the iris circle, the pupil radius, the iris radius, and the coordinates of the eyelid contours (upper eyelid and lower eyelid) included in the acquired image.

[0034] For example, the landmark detection unit 11 may output, as landmark information, the numerical values ​​of the center position of the iris circle and the radius of the iris circle, as well as the numerical values ​​of the center position of the pupil circle and the radius of the pupil and a vector representing the position coordinates of a point on the eyelid.The landmark detection unit 11 may calculate, as landmark information, a vector including the center coordinates of the outer circle c1 of the iris, the radius of the outer circle c1 of the iris, the coordinates of the outer corner of the eye, and the coordinates of the inner corner of the eye.

[0035] The landmark detection unit 11 may be configured, for example, as a recurrent neural network. The recurrent neural network may include multiple convolutional layers and multiple activation layers and extract landmark information from the acquired image. When the landmark detection unit 11 is configured as a neural network, any neural network structure may be used as long as the input-output relationship remains unchanged. For example, the neural network structure may be similar to that of VGG, ResNet, DenseNet, SETNet, MobileNet, EfficientNet, etc., but other structures may also be used. The landmark detection unit 11 may also be an image processing function other than a neural network. The landmark detection unit 11 may generate eye landmark information using an image after the conversion process (normalization) described with reference to Figures 3, 4, and 5. Note that the radius of the iris circle included in the landmark information may use information before normalization. The landmark detection unit 11 outputs the landmark information to the image region selection units 12.1 and 12.2.

[0036] Image region selection units 12.1 and 12.2 acquire the image input from image acquisition unit 10 and the landmark information input from landmark detection unit 11. Using the image and landmark information, image region selection units 12.1 and 12.2 each generate a normalized image as described in FIGS. 3, 4, and 5, and select different partial regions as shown in FIG. 6 (step S13). That is, image region selection unit 12.1 selects partial region a1 and outputs it to feature extraction unit 13.1. Image region selection unit 12.2 selects partial region a2 and outputs it to feature extraction unit 13.2.

[0037] Feature extraction units 13.1 and 13.2 extract features from the acquired partial region images after performing image preprocessing such as brightness histogram normalization, which converts the brightness of each pixel in the image so that the median or average of the brightness histogram of each pixel in the image matches a predetermined brightness, masking other than the iris circle, polar coordinate expansion with the center of the iris circle as the origin, and iris rubber sheet expansion using the pupil circle and iris circle (step S14). Feature extraction unit 13.1 receives the image of partial region a1 as input and extracts feature f1. Feature extraction unit 13.2 receives the image of partial region a2 as input and extracts feature f2. Feature extraction units 13.1 and 13.2 may be implemented, for example, as a convolutional neural network. To ensure that features can be appropriately extracted, feature extraction units 13.1 and 13.2 may train a model of the feature extractor in advance using images of partial regions and person labels selected by image region selection units 12.1 and 12.2. Feature extraction unit 13 may be an estimator using a model that can generate features with high accuracy, and may be another trained neural network. Furthermore, feature extraction units 13.1 and 13.2 may be an image processing function that extracts features that are not constructed using a neural network.

[0038] Feature extraction units 13.1 and 13.2 associate the extracted features f1 and f2 (matching features) with labels or the like of the person appearing in the image used in the feature recording process, and record them in matching feature storage unit 14 (step S15). As a result, the features f1 and f2 of the two partial regions with different eyes of the person appearing in the image used in the feature recording process are recorded in matching feature storage unit 14, respectively.

[0039] The authentication device 1 may perform the same process as above for both the left and right eyes shown in the image, and may further associate the feature f1 and feature f2 with the label of the left eye or the right eye, and record them in the matching feature storage unit 14. The authentication device 1 may also perform the same feature recording process using images of many people who are authenticated and provide predetermined services or processing functions, and similarly record the feature f1 and feature f2 in the matching feature storage unit 14. This completes the description of the preliminary feature recording process.

[0040] 8 is a diagram showing a processing flow of the authentication processing performed by the authentication device 1 in the first embodiment. Next, the authentication processing of the authentication device 1 in the first embodiment will be described with reference to FIG.

[0041] The authentication device 1 may photograph a person using a predetermined camera and acquire the image generated during the photographing. The image acquisition unit 10 acquires an image including the person's eyes (step S21). The image is assumed to include at least one or both of the person's eyes. The image acquisition unit 10 outputs the image to the landmark detection unit 11 and the image area selection units 12.1 and 12.2.

[0042] The landmark detection unit 11 detects landmark information of the eyes based on the acquired image (step S22). This process is similar to the process of step S12 described in the feature amount recording process above.

[0043] Image region selection units 12.1 and 12.2 receive an image from image acquisition unit 10 and landmark information from landmark detection unit 11. Similar to step S13 described in the feature recording process, image region selection units 12.1 and 12.2 each select a different partial region (step S23). That is, image region selection unit 12.1 selects partial region a1. Image region selection unit 12.1 selects partial region a2.

[0044] Feature extraction units 13.1 and 13.2 extract features from the images of the selected partial regions (step S24). This process is similar to the process of step S14 described in the feature recording process above. Feature extraction unit 13.1 outputs the extracted feature f1, and feature extraction unit 13.2 outputs the extracted feature f2, to the corresponding score calculation unit 15.

[0045] The score calculation unit 15.1 acquires the feature f1 extracted in the authentication process from the feature extraction unit 13.1. The score calculation unit 15.2 acquires the feature f2 extracted in the authentication process from the feature extraction unit 13.2. The score calculation unit 15.1 acquires the matching feature (feature f1) corresponding to one person extracted in the feature recording process and recorded in the matching feature storage unit 14. The score calculation unit 15.2 acquires the matching feature (feature f2) corresponding to one person extracted in the feature recording process and recorded in the matching feature storage unit 14. The score calculation units 15.1 and 15.2 each calculate an authentication score SC using the feature extracted in the authentication process and the feature extracted in the feature recording process (step S25). The authentication score SC calculated by the score calculation unit 15.1 is referred to as score SC1. The authentication score calculated by the score calculation unit 15.2 is referred to as score SC2.

[0046] The score calculation units 15.1 and 15.2 may calculate the scores SC1 and SC2 using, for example, the cosine similarity between the feature extracted in the authentication process and the feature extracted in the feature recording process. Alternatively, the score calculation units 15.1 and 15.2 may calculate the authentication score SC using an L2 distance function, an L1 distance function, or the like between the feature extracted in the authentication process and the feature extracted in the feature recording process. The score calculation units 15.1 and 15.2 may determine whether the features of data related to the same person are similar by utilizing the property that the distances between the features of the data related to the same person tend to be close, such as the cosine similarity, the L2 distance function, or the L1 distance function.

[0047] The score calculation units 15.1 and 15.2 may be constructed using, for example, a neural network. Furthermore, the score calculation units 15.1 and 15.2 may be a function for calculating the authentication score SC that is not constructed using a neural network, and may calculate the authentication score SC using, for example, the Hamming distance between the feature extracted in the authentication process and the feature extracted in the feature recording process. The score calculation units 15.1 and 15.2 output the calculated authentication score SC to the score integration unit 16.

[0048] In parallel with the above-described processing, the weight specifying unit 18 calculates a weight w for each authentication score SC calculated by the score calculation units 15.1 and 15.2. The weight for the score SC1 calculated by the score calculation unit 15.1 is set to w1, and the weight for the score SC2 calculated by the score calculation unit 15.2 is set to w2. The weight specifying unit 18 outputs the weights w1 and w2 to the score integrating unit 16. Details of the processing by the weight specifying unit 18 will be described later. The weights w1 and w2 will be collectively referred to as weight w.

[0049] The score integrating unit 16 calculates the integrated authentication score TSC using the scores SC1, SC2, weights w1, and w2 (step S26). The score integrating unit 16 calculates the integrated authentication score TSC by, for example, multiplying the scores SC1 and SC2 by their corresponding weights w1 and w2, and adding the resulting values ​​(TSC = SC1 * w1 + SC2 * w2). Note that in this formula, "*" indicates multiplication and "+" indicates addition. Alternatively, the score integrating unit 16 may calculate the integrated authentication score TSC using an estimation method such as a regression neural network or a support vector machine that inputs the scores SC1, SC2, and weights w1 and w2.

[0050] The score integrating unit 16 may calculate the integrated authentication score TSC by using an average of values ​​obtained by multiplying each authentication score SC by a corresponding weight w, or by using a weighted average. The score integrating unit 16 may calculate the integrated authentication score TSC by selecting the largest authentication score SC among the individual authentication scores SC of the person to be authenticated. The score integrating unit 16 may also be constructed using, for example, a neural network. The score integrating unit 16 may also be a processing function other than a neural network, and may use, for example, logistic regression or Ridge regression. The score integrating unit 16 outputs the integrated authentication score TSC to the authentication assessment unit 17.

[0051] The authentication assessment unit 17 acquires the integrated authentication score TSC. The authentication assessment unit 17 authenticates the person appearing in the image using the integrated authentication score TSC (step S27). For example, if the integrated authentication score TSC is equal to or greater than a threshold, the authentication assessment unit 17 determines that the person appearing in the image is a registered person and outputs information indicating successful authentication. If the integrated authentication score TSC is less than the threshold, the authentication assessment unit 17 determines that the person appearing in the image is an unregistered person and outputs information indicating unsuccessful authentication. The authentication assessment unit 17 may identify, in the matching feature storage unit 14, the matching feature used to calculate the highest integrated authentication score TSC among the integrated authentication scores TSC that are equal to or greater than the threshold, and identify the person appearing in the image based on the person's label associated with the matching feature. The authentication assessment unit 17 may determine that authentication has failed if the difference between the highest integrated authentication score TSC and the next highest integrated authentication score TSC among the integrated authentication scores TSC that are equal to or greater than the threshold is equal to or less than a predetermined threshold.

[0052] The authentication device 1 performs the above-mentioned processing for each of the left and right eyes of the subject appearing in the acquired image, and the authentication assessment unit 17 may determine that the subject appearing in the image has been successfully authenticated if the integrated authentication scores TSC for both eyes are both above a threshold value.

[0053] FIG. 9 is a first diagram illustrating an overview of the weight specification process. FIG. 10 is a second diagram illustrating an overview of the weight specification process. Next, the processing of the weight specification unit 18 will be described. The weight specification unit 18 calculates weights for the authentication scores SC calculated from the feature quantities of the partial regions a1 and a2 based on the landmark information detected by the landmark detection unit 11. Specifically, the weight specification unit 18 calculates the eye opening degree θ based on the distance h1 (the height from the lower eyelid to the upper eyelid) between a vertical line passing through the center O2 of the iris and each intersection p of the upper eyelid and the lower eyelid in the normalized images in FIGS. 3, 4, and 5. The distance h1 is one aspect of pixel information. The weight specification unit 18 may calculate the eye opening degree θ as the ratio of the distance h1 to the iris diameter. When the iris diameter (iris diameter) is adjusted by normalization so that it has approximately the same value D, the weight specification unit 18 may calculate the eye opening degree θ as the ratio of the distance h1 to the value D. The weight specifying unit 18 may calculate the eye opening / closing degree θ by other methods.

[0054] The weight specification unit 18 may acquire the eye opening degree θ and the iris diameter d from the calculation results of the landmark detection unit 11, and when the eye opening degree θ is greater than a predetermined threshold θ1, may assign a large weight to the authentication score SC1 for the normalized iris circular region (partial region a1) and a small weight to the authentication score SC2 for the region including the eye periphery (partial region a2) to calculate the integrated authentication score TSC ( FIG. 9 ). In this case, even if the eye opening degree θ is greater than the threshold θ1, the weight of the authentication score SC1 for the iris circular region may be calculated slightly larger than the weight for the region including the eye periphery, as shown in FIG. 9 ( FIG. 9 ). When the eye opening degree θ is smaller than the predetermined threshold θ1, the weight specification unit 18 may calculate the weight w2 for the partial region a2 to be larger than the weight w1 for the partial region a1, so that the integrated authentication score TSC is calculated by multiplying the authentication score SC2 for the region including the eye periphery (partial region a2) by a larger weight w2 ( FIG. 9 ). As a result, the larger the opening degree θ, the more clearly the iris is captured in the image, making it possible to calculate an integrated authentication score TSC that emphasizes iris features. Furthermore, the smaller the iris diameter d, the less the iris is captured in the image, making it possible to calculate an integrated authentication score TSC that emphasizes features around the eyes, such as the skin around the eyelids, wrinkles, and corners of the eyes. Note that this example shows how to determine which authentication score SC to weight based solely on a predetermined threshold θ1. However, multiple predetermined thresholds may be set, rather than just one, and the weight of each authentication score SC may be calculated based on the relationship between these multiple thresholds and the opening degree θ of the eyes. Furthermore, without using a threshold, the weight w of each partial region (iris and area around the eyes) may be calculated using a function of the opening degree θ of the eyes.

[0055] When the iris diameter d is greater than a predetermined threshold d1, the weight specification unit 18 may calculate the integrated authentication score TSC by weighting the authentication score SC1 for the normalized iris circle region (partial region a1) more heavily ( FIG. 10 ). When the iris diameter d is smaller than the predetermined threshold d1, the weight specification unit 18 may calculate the integrated authentication score TSC by weighting the authentication score SC2 for the region including the area around the eyes (partial region a2) more heavily ( FIG. 10 ). As a result, the larger the iris diameter d, the more clearly the iris is captured in the image, so that the integrated authentication score TSC can be calculated with emphasis on iris features. Furthermore, the smaller the iris diameter d, the less the iris is captured in the image, so that the integrated authentication score TSC can be calculated with emphasis on features around the eyes, such as the skin around the eyelids, wrinkles, and corners of the eyes. Note that this example shows how to determine which authentication score SC to weight based solely on the predetermined threshold d1. However, multiple thresholds may be set, rather than just one, and the weight of each authentication score SC may be calculated based on the relationship between the multiple thresholds and the iris diameter d. Alternatively, without using a threshold value, the weight w of each partial region (iris and surrounding area of ​​the eye) may be calculated using a function of the iris diameter d.

[0056] These weights w are determined by calculating the average value of the integrated authentication scores TSC calculated in advance using images and a score calculation model for various eye opening degrees θ and iris diameters d, and extracting the value of weight w that maximizes the authentication score SC of the target person's feature amount after calculating the integrated authentication score TSC, or the value of weight w that minimizes the authentication score SC of the target person's feature amount.The weight specifying unit 18 may then specify these previously extracted values ​​of weight w based on the opening degree θ and iris diameter d obtained from the image.

[0057] FIG. 11 is a block diagram of a function for generating a specific model of weights for authentication scores. The weight identification unit 18 performs functions such as a training data acquisition function 181, a normalization function 182, an estimation function 183, a loss function calculation function 184, a gradient calculation function 185, and a parameter update function 186. The weight identification unit 18 may learn a specific model for estimating the weight w using training data that includes a combination of landmark points set to enable selection of a predetermined partial area related to the eye, vectors representing the state of the eye image such as an iris circle or a pupil circle, a weight w, and a label for identifying an individual. The estimation function 183 of the weight identification unit 18 identifies the weight w using such a specific model. The weight identification unit 18 may previously determine the weight w for calculating the optimal integrated authentication score TSC using the training data and an existing specific model. For example, for an iris image having vectors related to a certain landmark point, iris circle, and pupil circle, the weight identification unit 18 calculates iris features and features around the eye. Next, a registered image of a predetermined corresponding person is identified based on the label, and two feature amounts (iris feature amount and eye area feature amount) are similarly extracted from the registered image. Using the feature amount extracted from the iris image having a certain landmark point, iris circle, and pupil circle vector, and the feature amount extracted from the registered image of the corresponding person identified based on the label, an authentication score SC is calculated by comparing the iris feature amount, and an authentication score SC is calculated by comparing the eye area feature amount. For each calculated authentication score SC, a weight w is estimated so that the authentication score SC is maximized when the person is authenticated based on the label, and so that the authentication score SC is minimized when the person is authenticated with a mismatched label.

[0058] The weight identification unit 18 may directly extract vectors (landmark information) representing the state of the eye image, such as landmark points, iris circle, and pupil circle, used as input to the neural network from the image using a trained landmark detection model. The vectors (landmark information) representing the landmark points, iris circle, and pupil circle acquired by the weight identification unit 18 may further include values ​​that may be related to the weights used to integrate the authentication scores, such as the size and position of occlusion areas due to reflections on the surfaces of the glasses and the iris, and the area of ​​the iris. The vectors (landmark information) representing eye features, such as landmark points, iris circle, and pupil circle acquired by the weight identification unit 18, may further have their values ​​normalized before input so that the values ​​across the entire dataset follow a Gaussian distribution with an average of 0 and a standard deviation of 1. The weight identification unit 18 may also normalize values ​​in the dimensional direction using the normalization function 182. The method of normalizing values ​​is not limited to a Gaussian distribution; values ​​may be normalized to a range of values ​​appropriate for input to a general neural network, such as [0, 1].

[0059] The weight specification unit 18 may calculate the weight w using information extracted from both a vector representing the state of the eye image, such as the landmark points, iris circle, and pupil circle extracted in the authentication process, and a vector representing the state of the eye image, such as the landmark points, iris circle, and pupil circle extracted in the feature recording process. For example, when calculating the weight w using the opening / closing degree θ, the opening / closing degree θ included in the vector extracted in the authentication process may be compared with the opening / closing degree θ included in the vector extracted in the feature recording process, and the smaller opening / closing degree θ may be used to calculate the weight w using the process described above with reference to FIG. 9. Alternatively, when calculating the weight w using the iris diameter d, the weight w may be calculated using the average value of the iris diameter d included in the vector extracted in the authentication process and the iris diameter d included in the vector extracted in the feature recording process, using the process described above with reference to FIG. 10. Note that the vector value used to calculate the weight w is not limited to an average value or a small value. Any calculation or function may be used as long as the vector extracted in the authentication process and the two vectors extracted in the feature recording process are used. When a neural network is used to calculate the weight w, the network may be trained to calculate the weight w by inputting both a vector representing the state of the eye image extracted in the authentication process and a vector representing the state of the eye image extracted in the feature recording process.

[0060] The vector (landmark information) representing the state of the eye image may include the iris center coordinates, iris radius, iris diameter, pupil center coordinates, pupil radius, pupil diameter, position of the outer corner of the eye, position of the forehead, and eyelid openness / closeness before normalization, the iris center coordinates, iris radius, iris diameter, pupil center coordinates, pupil radius, pupil diameter, position of the outer corner of the eye, position of the forehead, eyelid openness / closeness, position and area of ​​occlusions in the image such as lighting reflection, as well as information such as the presence or absence of glasses, the presence or absence of contact lenses, information on the transparency or non-transparency of the contact lenses, information on the transparency of the contact lenses, information on the presence or absence of makeup, information on the intensity of makeup, the presence or absence of false eyelashes, and the presence or absence of mascara. The weight specifying unit 18 calculates the weights w of the iris similarity authentication score SC and the authentication score SC of the image including the periphery of the eye, which are used to calculate the integrated authentication score TSC using the detection results of these features. As a method for calculating the weight of the authentication score SC, the weight specification unit 18 may use a value determined empirically by a person, such as changing the weight of the authentication score SC depending on the size of the iris radius. Alternatively, the weight specification unit 18 may determine the weight w of the authentication score SC using a regression model obtained by learning. In this case, the regression model may be learned, for example, by optimizing a neural network using learning data containing information such as iris features, eye peripheral features, detection results, and labels. In this case, a regression model is learned that extracts the iris detection position as input and the weight w of the authentication score SC as output. The calculated weights w may be normalized so that their sum equals 1.

[0061] The weight w of each authentication score SC described above may be calculated in advance by a person and recorded in a storage unit or the like or set in a setting file, and the weight specifying unit 18 may acquire the recorded or set value of the weight w. The weight specifying unit 18 may also modify or update the weight described above using the parameter update function 186. For example, the weight specifying unit 18 may modify or update the value of the weight w when the diameter of the iris photographed increases or decreases depending on the installation location of the camera of the authentication device 1.

[0062] 12 is a diagram showing a process flow for generating a weight specification model for authentication scores. The weight specification unit 18 acquires the training data described above for learning the weight specification model (step S31). The weight specification unit 18 randomly extracts a predetermined number of pairs of vectors representing the state of the eye image, such as landmark points, iris circle, and pupil circle, and correct weight information from the training data, and inputs these pairs to the neural network (step S32). The number of pairs is not particularly limited.

[0063] At this stage, the input eye features, such as landmark points, iris circle, and pupil circle, are normalized by the normalization function 182 in the same manner as in the processes shown in FIGS. 3, 4, and 5. Then, after the input normalization process (FIGS. 3, 4, and 5), the weight specification unit 18 uses the estimation function 183 to estimate weights of the authentication score SC for each partial region for calculating the integrated authentication score TSC based on vectors representing the state of the eye image, such as landmark points, iris circle, and pupil circle (step S33). Note that if the image acquisition unit 10 has previously normalized the image using the processes shown in FIGS. 3, 4, and 5, the normalization process in generating the weight specification model is unnecessary. Note that the radius of the iris circle in the image may be determined using information before normalization. The architecture of the weight specification model for calculating the integrated authentication score is not particularly limited. For example, a multi-layer perceptron (MLP) with multiple layers may be used. The number of layers, the number of channels, and the type of layers are not particularly limited.

[0064] The weight identification unit 18 uses the loss function calculation function 184 to calculate a loss from the output of the neural network (step S34). For example, the L2 distance between the estimated result and the correct answer may be used as the loss. The distance is not limited to the L2 distance and may be any distance, such as the L1 distance or cosine similarity. The weight identification unit 18 uses the gradient calculation function 185 to determine the gradient of each parameter of the neural network, for example, by backpropagation (step S35).

[0065] The weight specification unit 18 uses the parameter update function 186 to optimize the neural network parameters using the gradient of each parameter (step S36). In the parameter update, the weight specification unit 18 may use, for example, stochastic gradient descent. In the parameter update procedure, the weight specification unit 18 may use other methods for optimizing parameters, such as Adam, without being limited to stochastic gradient descent. In this process, hyperparameters such as the learning rate, weight decay, and momentum are not particularly limited. In this learning, the weight specification model is optimized for a predetermined number of repetitions (iterations). Hyperparameters such as the learning rate may be changed during the optimization process to facilitate convergence of the learning to a better optimal value. Furthermore, the learning may be stopped midway when the loss has dropped to a certain extent. The weight specification unit 18 records the optimized parameters (step S37).

[0066] The weight specifying unit 18 calculates a weight for each authentication score SC using the specified model of the weight w calculated in this way. That is, the weight specifying unit 18 estimates the weights w1 and w2. The weight specifying unit 18 outputs the weights w1 and w2 to the score integrating unit 16.

[0067] The authentication device 1 extracts features from each of multiple regions extracted from the target's eye region in an acquired image, and determines a weight for each authentication score when calculating an authentication score based on the features and features related to corresponding regions pre-stored for the target. The authentication device 1 then calculates an integrated authentication score TSC for the target's features included in the acquired image and the pre-stored features using the features extracted from the features of each of the multiple regions and the weights determined for the features. The authentication device 1 then calculates an integrated authentication score TSC using the authentication scores SC weighted according to partial region a1 and partial region a2, and performs authentication based on the integrated authentication score TSC. The integrated authentication score TSC is weighted higher for partial region a1, which has a larger iris area, the more iris information there is. As a result, when authenticating using an image in which the eyes are widely opened or the iris diameter is wide, authentication is performed with emphasis on the iris features. When authenticating using an image in which the eyes are relatively closed or the iris diameter is relatively narrow, authentication is performed with emphasis on the features around the eyes. Therefore, when the amount of iris information is large, authentication can be performed with emphasis on the amount of iris information, and when the amount of iris information is small, authentication can be performed with emphasis on the amount of information around the eye, so authentication is possible whether the amount of iris information is large or small, and a more accurate integrated authentication score TSC (similarity) can be calculated. This makes it possible to improve the authentication accuracy of targets in authentication techniques using ensemble estimation.

[0068] <Second embodiment> Fig. 13 is a block diagram showing the configuration of an authentication device 1 in the second embodiment. As shown in Fig. 13, the authentication device 1 includes an image acquisition unit 10, a landmark detection unit 11, image region selection units 12.1 to 12.N, feature extraction units 13.1 to 13.N, a matching feature storage unit 14, score calculation units 15.1 to 15.N, a score integration unit 16, an authentication determination unit 17, and a weight specification unit 18.

[0069] The image acquisition unit 10, landmark detection unit 11, matching feature storage unit 14, and authentication determination unit 17 are the same as those in the first embodiment. Image region selection units 12.1, ..., 12.N select multiple different partial regions including at least a portion of the iris region based on landmark information detected by landmark detection unit 11. Image region selection units 12.1, ..., 12.N operate in parallel and select different image regions in the acquired image. Image region selection units 12.1, ..., 12.N may select partial regions including the iris region. One or more of image region selection units 12.1, ..., 12.N may select different partial regions of the eye including the entire iris region. Image region selection units 12.1, ..., 12.N are collectively referred to as image region selection unit 12.

[0070] Feature extraction units 13.1, ..., 13.N extract feature f for the partial regions selected by image region selection unit 12. That is, feature extraction unit 13.1 extracts feature f1 for partial region a1 selected by image region selection unit 12.1, feature extraction unit 13.2 extracts feature f2 for partial region a2 selected by image region selection unit 12.2, and feature extraction unit 13.N extracts feature fn for partial region an selected by image region selection unit 12.N. Feature f is a value representing the features of the eye, including the iris, which are necessary for iris authentication. Feature extraction units 13.1, ..., 13.N are collectively referred to as feature extraction unit 13.

[0071] Score calculation units 15.1, ..., 15.N calculate an authentication score SC for each partial region using the feature f extracted by feature extraction unit 13 and the matching feature f stored in matching feature storage unit 14. That is, score calculation unit 15.1 calculates an authentication score SC for partial region a1 using the feature f1 extracted by feature extraction unit 13.1 and the matching feature f1 stored in matching feature storage unit 14. Score calculation unit 15.2 calculates an authentication score SC for partial region a2 using the feature f2 extracted by feature extraction unit 13.2 and the matching feature f2 stored in matching feature storage unit 14. Score calculation unit 15.N calculates an authentication score SC for partial region an using the feature fn extracted by feature extraction unit 13.N and the matching feature fn stored in matching feature storage unit 14. The authentication score SC here is the degree of similarity with a corresponding feature amount registered in advance, which is necessary for iris authentication. The score calculation units 15.1, ..., 15.N are collectively referred to as score calculation unit 15.

[0072] The score integration unit 16 calculates an integrated authentication score TSC using the scores SC1, ..., SCn obtained from the score calculation units 15.1, ..., 15.N. The weight specification unit 18 calculates weights w for the authentication scores SC1, ..., SCn.

[0073] The processing of the weight specification unit 18 generates a weight specification model in the same manner as in the first embodiment, using training data of pairs of vectors indicating features and correct weights for each partial region selected by the image region selection unit 12. The weight specification unit 18 uses this weight specification model to calculate weights for the scores SC1, ..., SCn in the same manner as in the first embodiment.

[0074] FIG. 14 is a diagram illustrating an overview of the region selection process according to the second embodiment. After sequentially performing one or more normalization processes described above with reference to FIGS. 3, 4, and 5, the image region selection unit 12 cuts out an image of a predetermined partial region based on the eye feature information. As shown in FIG. 14, the image region selection units 12.1, ..., 12.N may cut out images of partial regions at different positions based on the eye feature information. The partial regions selected by each image region selection unit 12 may be multiple partial regions with different center positions. The partial regions selected by each image region selection unit 12 may be multiple partial regions with different selected area sizes. Each image region selection unit 12 may select multiple different partial regions, including a partial region that includes the area within the eyeball and a partial region that includes the skin surrounding the eyeball. The image region selection unit 12 may select multiple different regions that include landmark points that are set to enable selection of predetermined partial regions related to the eye. The authentication device 1 according to this embodiment may use the image features of different partial regions to perform learning to generate estimation models, and then perform ensemble estimation using the image features of the different partial regions and each estimation model, thereby improving the accuracy of authentication.

[0075] 15 is a diagram showing a processing flow of the feature amount recording process performed by the authentication device 1 in the second embodiment. Next, the feature amount recording process of the authentication device 1 in the second embodiment will be described with reference to FIG.

[0076] In the preliminary feature recording process, the authentication device 1 inputs a facial image of a person or a partial image of the area around the eyes. The authentication device 1 may photograph the person using a predetermined camera and acquire the image generated during the photographing. The image acquisition unit 10 acquires an image including the person's eyes (step S41). The image is assumed to include at least one or both of the person's eyes. The image acquisition unit 10 outputs the image to the landmark detection unit 11 and the image area selection units 12.1, ..., 12.N.

[0077] The landmark detection unit 11 detects landmark information including landmark points of the eyes, etc., based on the acquired image (step S42). The processing by the landmark detection unit 11 is the same as in the first embodiment.

[0078] Image region selection units 12.1, ..., 12.N input an image from image acquisition unit 10 and landmark information including landmark points and the like from landmark detection unit 11. Each of image region selection units 12.1, ..., 12.N uses the image and landmark information including landmark points and the like to select different partial regions using the method described with reference to FIG. 14 (step S43). Image region selection units 12.1, ..., 12.N generate images of the selected partial regions. The images of the partial regions selected by image region selection units 12.1, ..., 12.N are referred to as images of partial region a1, ..., partial region an, respectively. Image region selection unit 12.1 outputs partial region a1 to feature extraction unit 13.1. Image region selection unit 12.2 outputs partial region a2 to feature extraction unit 13.2. Similarly, image region selection units 12.3, ..., 12.N output the images of the generated partial regions to the corresponding feature extraction units 13.

[0079] The feature extraction units 13.1, ..., 13.N extract features from the partial region images input from the image region selection unit 12 after performing image preprocessing such as brightness histogram normalization, masking of areas other than the iris circle, polar coordinate expansion with the center of the iris circle as the origin, and iris rubber sheet expansion using the pupil circle and iris circle (step S44). The feature extraction units 13.1, ..., 13.N receive the images of partial regions a1, ..., partial region an from the image region selection unit 12 and extract feature quantities f1, ..., feature quantities fn. The feature extraction units 13.1, ..., 13.N may each extract feature quantities using a different method. The feature extraction units 13.1, ..., 13.N may be constructed, for example, with a convolutional neural network. The image region selection units 12.1, ..., 12.N may be trained in advance using images of partial regions selected by the image region selection units 12.1, ..., 12.N so that features can be appropriately extracted. The feature extraction unit 13 may be an estimator using an estimation model that can generate features with high accuracy, or may be another trained neural network. Furthermore, the feature extraction units 13.1, ..., 13.N may be image processing functions that extract features that are not constructed using a neural network.

[0080] Feature extraction units 13.1, ..., 13.N associate the extracted features f1, ..., feature fn (matching features) with labels of people appearing in the images used in the feature recording process and labels of feature extraction unit 13 that extracted the features, and record the linked features in matching feature storage unit 14 (step S45). As a result, the eye features of people appearing in the images used in the feature recording process, which are features of different partial regions of the eyes, are recorded in matching feature storage unit 14.

[0081] The authentication device 1 may perform the same process as above for both the left and right eyes shown in the image, and may further associate the features f1, ..., fn with the label of the left eye or the right eye, and record the features f1, ..., fn in the matching feature storage unit 14. The authentication device 1 may also perform the same feature recording process using images of many people who are authenticated and provide predetermined services or processing functions, and similarly record the features f1, ..., fn in the matching feature storage unit 14. This completes the description of the preliminary feature recording process.

[0082] 16 is a diagram showing the processing flow of the authentication processing performed by the authentication device 1 in the second embodiment. Next, the authentication processing of the authentication device 1 in the second embodiment will be described with reference to FIG.

[0083] In the authentication process, the authentication device 1 inputs a facial image or a partial image of the eyes of a person. The authentication device 1 may photograph the person using a predetermined camera and acquire the image generated during the photographing. The image acquisition unit 10 acquires an image including the person's eyes (step S51). The image is assumed to include at least one or both of the person's eyes. The image acquisition unit 10 outputs the image to the landmark detection unit 11 and the image region selection units 12.1, ..., 12.N.

[0084] The landmark detection unit 11 detects landmark information including landmark points of the eyes based on the acquired image (step S52). This process is similar to the process of step S42 described in the feature amount recording process above.

[0085] Image region selection units 12.1, ..., 12.N input the image from image acquisition unit 10 and the landmark information from landmark detection unit 11. Using the image and landmark information, image region selection units 12.1, ..., 12.N each select a different partial region using the method described in Fig. 14 (step S53). This process is similar to the process in step S43 described in the feature amount recording process above.

[0086] The feature extraction units 13.1, ..., 13.N extract features from the partial region images input from the image region selection unit 12 (step S54). This process is the same as the process in step S44 described in the feature recording process above. The feature extraction units 13.1, ..., 13.N output the extracted features f1, ..., feature fn to the corresponding score calculation units 15.

[0087] The score calculation units 15.1, ..., 15.N acquire features f1, ..., fn extracted in the authentication process from the corresponding feature extraction units 13. The score calculation units 15.1, ..., 15.N also acquire features (features f1, ..., fn) corresponding to one person extracted in the feature recording process and recorded in the matching feature storage unit 14. The score calculation units 15.1, ..., 15.N calculate an authentication score SC using the features extracted in the authentication process and the features extracted in the feature recording process (step S55). The authentication scores SC calculated by the score calculation units 15.1, ..., 15.N are referred to as scores SC1, ..., SCn, respectively.

[0088] The score calculation units 15.1, ..., 15.N may calculate the scores SC1, ..., SCn using, for example, the cosine similarity between the feature extracted in the authentication process and the feature extracted in the feature recording process. Alternatively, the score calculation units 15.1, ..., 15.N may calculate the authentication scores using an L2 distance function, an L1 distance function, or the like between the feature extracted in the authentication process and the feature extracted in the feature recording process. The score calculation units 15.1, ..., 15.N may determine whether each feature is similar by utilizing the property that feature values ​​of data related to the same person, such as cosine similarity, L2 distance function, or L1 distance function, tend to be close in distance.

[0089] The score calculation units 15.1, ..., 15.N may be constructed, for example, by a neural network. Furthermore, the score calculation units 15.1, ..., 15.N may be a score calculation processing function that is not constructed by a neural network, and may calculate an authentication score based on the Hamming distance between the feature extracted in the authentication processing and the feature extracted in the feature recording processing, for example. The score calculation units 15.1, ..., 15.N output the calculated authentication scores to the score integration unit 16.

[0090] The score integrating unit 16 acquires weights w1, ..., wn for each of the scores SC1, ..., SCn from the weight specifying unit 18. The score integrating unit 16 calculates the integrated authentication score TSC using the scores SC1, ..., SCn and the weights w1, ..., wn (step S56). Specifically, the score integrating unit 16 calculates the integrated authentication score TSC by "TSC = SC1 * w1 + ... + SCn * wn." Note that in this formula, "*" indicates multiplication and "+" indicates addition. Alternatively, the score integrating unit 16 may calculate the integrated authentication score TSC using an estimation method such as a regression neural network or a support vector machine that inputs the scores SC1, SC2, and the weights w1 and w2. The processing of the authentication determining unit 17 is the same as that in the first embodiment.

[0091] The authentication device 1 according to the second embodiment also extracts features from each of multiple regions cut out from an acquired image containing the target's eyes, and determines a weight for each authentication score SC when calculating the authentication score SC based on the features of each of the multiple regions and the features of corresponding regions pre-stored for the target. The authentication device 1 then calculates an integrated authentication score TSC for the features of the target included in the acquired image and the pre-stored features of the target using the features of each of the multiple regions and the weights determined for those features. Through this process, the authentication device 1 assigns weights according to the partial regions to the authentication scores corresponding to partial regions a1, a2, ..., and an, and performs authentication based on the integrated authentication score TSC. The integrated authentication score TSC is weighted higher for partial regions with larger iris areas as the amount of iris information increases. As a result, when authentication is performed using an image in which the eyes are widely opened or the iris diameter is wide, emphasis is placed on the iris features. When authentication is performed using an image in which the eyes are relatively closed or the iris diameter is relatively narrow, emphasis is placed on the features around the eyes. Therefore, even when the amount of iris information is low, authentication can be performed by prioritizing the amount of information around the eye, and on the other hand, when the amount of iris information is high, authentication can be performed by prioritizing the amount of iris information, so authentication is possible whether the amount of iris information is high or low, and a more accurate integrated authentication score TSC (similarity) can be calculated. This makes it possible to improve the authentication accuracy of targets in authentication techniques using ensemble estimation.

[0092] In the authentication device 1, the image region selection unit 12 selects multiple different partial regions including at least a portion of the iris based on eye features included in the acquired image, and the feature extraction unit 13 calculates the feature values ​​for each of the different partial regions. The score calculation unit 15 calculates the similarity between each of the different partial regions based on the relationship between the feature values ​​for each of the different partial regions and the feature values ​​for each of the different partial regions of a person stored in advance, and the authentication assessment unit 17 authenticates the person whose eyes are included in the acquired image based on the similarity between each of the different partial regions. This process allows for authentication using ensemble estimation using different estimators corresponding to the different partial regions including the iris, thereby easily improving the authentication accuracy of the target.

[0093] Iris recognition technology requires high authentication accuracy for practical use. One method for improving authentication accuracy is to use higher-resolution, in-focus images, but capturing images with a high number of pixels requires more expensive cameras and places stricter restrictions on the imaging environment. Therefore, a method for improving accuracy by improving information processing functions is needed. Ensemble estimation is one way to improve estimation accuracy. Ensemble estimation is a method that combines the results of multiple estimators to achieve higher accuracy than the results of individual estimators. For effective ensemble estimation, each estimator must be able to make accurate estimates and the correlation between their results must be low. Common ensemble estimation methods use random numbers to divide and generate training data for generating estimation models, or link estimators together to perform estimation. However, these methods require trial and error to improve performance and have the disadvantage of high training costs for the estimation model.

[0094] When an image including eyes is input, the authentication device 1 according to this embodiment extracts landmark information including landmark points set so as to enable selection of a predetermined partial area related to the eyes, and selects a predetermined partial area using the obtained landmark information, thereby obtaining multiple partial areas each with different characteristics regardless of the iris position or rotation state in the eye image. Because these partial area images contain iris information but include different areas, feature amounts with low correlation with each other can be reliably extracted. This allows the authentication device 1 according to this embodiment to perform effective ensemble estimation without the need for trial and error using random numbers as in general ensemble estimation methods.

[0095] 17 is a hardware configuration diagram of an authentication device. As shown in this diagram, the authentication device 1 may be a computer equipped with various hardware components such as a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a database 104, and a communication module 105. The functions of the authentication device 1 according to each of the above-described embodiments may be realized by an information processing system in which a plurality of information processing devices are equipped with one or more of the above-described functions and are configured to work together to perform overall processing.

[0096] FIG. 18 shows the minimum configuration of an authentication device. FIG. 19 shows the processing flow of an authentication device with a minimum configuration. As shown in this figure, the authentication device 1 performs at least the functions of a feature extraction unit 81, a weight determination unit 82, and a similarity calculation unit 83. The feature extraction unit 81 extracts features from each of multiple regions cut out from an acquired image containing the target's eyes (step S91). The weight determination unit 82 determines a weight of similarity for each region, calculated based on the features of each of the multiple regions and the features of corresponding regions pre-stored for the target (step S92). Note that the weights calculated for each region may be normalized so that the sum of the weights for all regions is 1. The similarity calculation unit 83 calculates the similarity between the features of the target's eyes included in the acquired image and the pre-stored features of the target's eyes, based on the features of each region, the features of corresponding regions pre-stored for the target, and the weights (step S93).

[0097] The program may be a program for realizing some of the functions described above, or may be a so-called differential file (differential program) that can realize the functions described above in combination with a program already recorded in the computer system.

[0098] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes: Furthermore, the configurations of the above-described embodiments can be freely combined and modified.

[0099] (Supplementary Note 1) An information processing device comprising: a feature extraction means for extracting feature amounts for each of a plurality of regions cut out from an acquired image including the eyes of a target; a weight specification means for specifying a weight of similarity for each of the plurality of regions, calculated based on the feature amounts for each of the plurality of regions and each feature amount for the corresponding region pre-stored for the target; and a similarity calculation means for calculating a similarity between the feature amounts of the eyes of the target included in the acquired image and the pre-stored feature amounts of the eyes of the target, based on the feature amounts for each of the plurality of regions, each feature amount for the corresponding region pre-stored for the target, and the weights.

[0100] (Supplementary Note 2) The information processing device according to claim 1, further comprising: a detection means for detecting landmark information indicating the position of the subject's eyes included in the acquired image; and an image region selection means for cutting out each of the plurality of regions based on the landmark information, wherein the feature extraction means extracts features of each of the plurality of regions cut out by the image region selection means.

[0101] (Supplementary Note 3) The information processing device according to claim 2, wherein the detecting means detects the landmark information included in the acquired image, and the weight specifying means calculates the weight of the similarity based on the landmark information.

[0102] (Supplementary Note 4) The information processing device according to claim 2 or 3, wherein the weight specifying means calculates the weight of the similarity based on a parameter calculated using the landmark information.

[0103] (Supplementary Note 5) The information processing device according to any one of claims 2 to 4, wherein the weight specifying means calculates a weight of the similarity for each of the regions based on the degree of eye opening or closing calculated based on the landmark information.

[0104] (Supplementary Note 6) The information processing device according to any one of claims 2 to 4, wherein the weight specifying means calculates a weight for the similarity for each of the regions based on pixel information of the iris of the eye calculated based on the landmark information.

[0105] (Supplementary Note 7) The information processing device according to any one of claims 1 to 6, wherein the feature extraction means extracts features of a first region that includes at least the iris region of the eye but does not include the region surrounding the eye, and features of a second region that includes both the iris region and the region surrounding the eye, and the similarity calculation means calculates the similarity between the features of the object included in the acquired image and the features of the object stored in advance, using the weight specified for the similarity between the features obtained from the features of the first region and the second region and the features stored in advance for those regions.

[0106] (Supplementary Note 8) The information processing device according to claim 2 or 3, wherein the weight specification means calculates the weight based on the landmark information and a model obtained by machine learning.

[0107] 1... authentication device (information processing device, information processing system), 10... image acquisition unit, 11... landmark detection unit (detection means), 12 (12.1, 12.2, ... 12.N)... image region selection unit (region selection means), 13 (13.1, 13.2, ... 13.N)... feature extraction unit (feature extraction means), 14... matching feature storage unit, 15 (15.1, 15.2, ... 15.N)... score calculation unit (similarity calculation means), 16... score integration unit (similarity calculation means), 17... authentication judgment unit (authentication means), 18... weight identification unit (weight identification means)

Claims

1. A feature extraction means for extracting feature amounts of each of a plurality of regions cut out from an acquired image including the eye of a subject; a weight specifying means for specifying a weight of the similarity of each of the plurality of regions, the weight being calculated based on a feature amount of each of the plurality of regions and each feature amount of the corresponding region that is stored in advance for the object; a similarity calculation means for calculating a similarity between an eye feature of the object included in the acquired image and a pre-stored eye feature of the object based on the feature of each of the plurality of regions, each feature of the object related to the corresponding region pre-stored, and the weight; An information processing device comprising:

2. a detection means for detecting landmark information indicating a position related to the eye of the subject included in the acquired image; an image region selection means for cutting out each of the plurality of regions based on the landmark information; The feature extraction means extracts a feature of each of the plurality of regions cut out by the image region selection means. The information processing device according to claim 1 .

3. The detection means detects the landmark information included in the acquired image, The weight specification means calculates the weight of the similarity based on the landmark information. The information processing device according to claim 2 .

4. The weight specification means calculates the weight of the similarity based on a parameter calculated using the landmark information. The information processing device according to claim 3 .

5. The weight specifying means calculates a weight of the similarity for each of the regions based on the eye opening / closing degree calculated based on the landmark information. The information processing device according to claim 3 .

6. The weight specifying means calculates a weight for the similarity for each of the regions based on pixel information of the iris of the eye calculated based on the landmark information. The information processing device according to claim 3 .

7. the feature extraction means extracts a feature of a first region including at least an iris region of the eye and not including a region around the eye, and a feature of a second region including both the iris region and a region around the eye; The similarity calculation means calculates a similarity between a feature amount of an object included in the acquired image and a feature amount of the object stored in advance, using the weighting specified for the similarity between a feature amount obtained from the features of the first region and the second region and a feature amount pre-stored for these regions. The information processing device according to any one of claims 1 to 6.

8. The weight specification means calculates the weight based on the landmark information and a model obtained by machine learning. The information processing device according to claim 3 .

9. Extracting feature amounts for each of a plurality of regions cut out from the eye region of the subject included in the acquired image; specifying a weight of the similarity of each of the plurality of regions calculated based on the feature amount of each of the plurality of regions and the feature amount of the corresponding region pre-stored for the object; Based on the feature amounts of each of the plurality of regions, the feature amounts of the corresponding regions of the object stored in advance, and the weights, a similarity between the feature amounts of the eye of the object included in the acquired image and the feature amounts of the eye of the object stored in advance is calculated. Information processing methods.

10. The computer of the information processing device, A feature extraction means for extracting feature amounts of each of a plurality of regions cut out from an eye region of a subject included in an acquired image; a weight specifying means for specifying a weight of the similarity of each of the plurality of regions, the weight being calculated based on the feature amount of each of the plurality of regions and the feature amount of the corresponding region pre-stored for the object; a similarity calculation means for calculating a similarity between an eye feature of an object included in the acquired image and a pre-stored eye feature of the object based on the feature of each of the plurality of areas, each feature of a corresponding area of ​​the object stored in advance, and the weight; A program that functions as a