Information processing device, information processing system, authentication method, and program

Abstract

The present invention selects a plurality of different partial regions, regarding partial regions that at least include some iris regions, on the basis of the features of the eyes of a subject included in an acquired image. The present invention calculates the feature quantity of each of the different partial regions. The present invention calculates the degree of similarity between each of the different partial regions and preliminarily stored feature quantities on the basis of the feature quantity of each of the different partial regions and the preliminarily stored feature quantities. The present invention authenticates the subject on the basis of the degree of similarity of each of the different partial regions.

Description

Information processing device, information processing system, authentication method, and storage medium

[0001] The present disclosure relates to an information processing device, an information processing system, an authentication method, and a storage medium.

[0002] There is an ensemble learning method that generates multiple learning devices and uses these multiple different learning devices to output a predetermined estimation result for an input. In this ensemble learning method, multiple individual learning devices train using the same or different data sets to generate a model. These learning devices are called weak learners. When calculating an estimation result, the estimation results of the individual weak learners are integrated to obtain the overall estimation result. Such ensemble learning may also be used for authentication.

[0003] Related techniques are disclosed in Non-Patent Documents 1 to 4. Patent Document 1 discloses a technique (bagging) in which multiple sub-datasets are created from a training dataset by sampling that allows overlap, and then separate weak learners are trained using these sub-datasets.

[0004] Patent Literature 2 discloses a technique (boosting) in which, when training a weak learner, the weight of the loss for the training data is determined from the output results of other learners, and the learner learns the training data. In this method, for example, a new learner is trained so that it has high discrimination ability for input data for which other learners have produced incorrect estimation results.

[0005] Patent Document 3 discloses a technique in which, when training a weak learner, a partial image obtained by randomly cutting out a part of an original image is used for learning.

[0006] Patent Document 4 discloses a technology in which there are two weak learners, one that receives an iris image as input and the other that receives an image of the area around the eye as input, and the results of these are integrated to output an estimation result.

[0007] L. Breiman. Bagging predictors. Machine Learning, 24, 123-140, 1996. RE Schapire. A brief introduction to Boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, 1999. B. Cheng, W. Wu, D. Tao, S. Mei, T. Mao and J. Cheng, "Random Cropping Ensemble Neural Network for Image Classification in a Robotic Arm Grasping System," in IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 9, pp. 6795-6806, Sept. 2020, doi: 10.1109 / TIM.2020.2976420. Oishi, S., Ichino, M., Yoshiura, H.: Fusion of iris and periocular user authentication by AdaBoost for mobile devices. In: 2015 IEEE International Conference on Consumer Electronics (ICCE), 9-12 Jan. 2015, Piscataway, NJ, USA, pp. 428-429. (2015)

[0008] The present disclosure aims to provide an information processing device, an information processing system, an authentication method, and a storage medium that improve upon the above-mentioned documents.

[0009] According to a first aspect of the present disclosure, an information processing device includes an area selection means for selecting a plurality of different partial areas including at least a portion of an iris area based on eye features included in an acquired image, a feature calculation means for calculating feature amounts of each of the different partial areas, a similarity calculation means for calculating a similarity between each of the different partial areas based on a relationship between the feature amounts of each of the different partial areas and the feature amounts of each of the different partial areas of a person stored in advance, and an authentication means for authenticating a person whose eyes are included in the acquired image based on the similarity between each of the different partial areas.

[0010] According to a second aspect of the present disclosure, an information processing device includes an area selection means for selecting one area including at least a portion of an iris area based on eye features included in an acquired image, a size conversion means for converting the one area into different partial areas with different numbers of pixels, a feature calculation means for calculating features of each of the different partial areas, a similarity calculation means for calculating a similarity between each of the different partial areas based on the relationship between the feature of each of the different partial areas and the feature of each of the different partial areas of a person stored in advance, and an authentication means for authenticating a person whose eyes are included in the acquired image based on the similarity between each of the different partial areas.

[0011] According to a third aspect of the present disclosure, an information processing system includes an area selection means for selecting a plurality of different partial areas including at least a portion of an iris area based on eye features included in an acquired image; a feature calculation means for calculating feature values ​​for each of the different partial areas; a similarity calculation means for calculating a similarity between each of the different partial areas based on a relationship between the feature values ​​for each of the different partial areas and the feature values ​​for each of the different partial areas of a person stored in advance; and an authentication means for authenticating a person having eyes included in the acquired image based on the similarity between each of the different partial areas.

[0012] According to a fourth aspect of the present disclosure, an information processing system includes an area selection means for selecting one area including at least a portion of an iris area based on eye features included in an acquired image; a size conversion means for converting the one area into different partial areas with different numbers of pixels; a feature calculation means for calculating features of each of the different partial areas; a similarity calculation means for calculating a similarity between each of the different partial areas based on a relationship between the feature of each of the different partial areas and the feature of each of the different partial areas of a person stored in advance; and an authentication means for authenticating a person whose eyes are included in the acquired image based on the similarity between each of the different partial areas.

[0013] According to a fifth aspect of the present disclosure, an authentication method selects a plurality of different partial regions including at least a portion of an iris region based on eye features included in an acquired image, calculates feature values ​​for each of the different partial regions, calculates a similarity between each of the different partial regions based on a relationship between the feature values ​​for each of the different partial regions and the feature values ​​for each of the different partial regions of a person stored in advance, and authenticates the person whose eyes are included in the acquired image based on the similarity between each of the different partial regions.

[0014] According to a sixth aspect of the present disclosure, an authentication method selects one region including at least a portion of an iris region based on eye features included in an acquired image, converts the one region into different partial regions with different numbers of pixels, calculates features of each of the different partial regions, calculates a similarity between each of the different partial regions based on a relationship between the features of each of the different partial regions and the features of each of the different partial regions of a person stored in advance, and authenticates the person whose eyes are included in the acquired image based on the similarity between each of the different partial regions.

[0015] According to a seventh aspect of the present disclosure, a storage medium stores a program that causes a computer of an information processing device to function as an area selection means that selects a plurality of different partial areas including at least a portion of an iris area based on eye features included in an acquired image, a feature calculation means that calculates feature amounts of each of the different partial areas, a similarity calculation means that calculates a similarity between each of the different partial areas based on the relationship between the feature amounts of each of the different partial areas and the feature amounts of each of the different partial areas of a person stored in advance, and an authentication means that authenticates a person with eyes included in the acquired image based on the similarity between each of the different partial areas.

[0016] According to an eighth aspect of the present disclosure, a storage medium stores a program that causes a computer of an information processing device to function as an area selection means that selects one area including at least a portion of an iris area based on eye features included in an acquired image, a size conversion means that converts the one area into different partial areas with different numbers of pixels, a feature calculation means that calculates feature amounts of each of the different partial areas, a similarity calculation means that calculates a similarity between each of the different partial areas based on the relationship between the feature amounts of each of the different partial areas and the feature amounts of each of the different partial areas of a person that is stored in advance, and an authentication means that authenticates a person with eyes included in the acquired image based on the similarity between each of the different partial areas.

[0017] 1 is a block diagram showing the configuration of an information processing device 1 in a first embodiment. FIG. 2 is a diagram showing an outline of feature point detection processing in the first embodiment. FIG. 3 is a first diagram showing an outline of area selection processing in the first embodiment. FIG. 4 is a first diagram showing an outline of area selection processing in the first embodiment. FIG. 5 is a second diagram showing an outline of area selection processing in the first embodiment. FIG. 6 is a third diagram showing an outline of area selection processing in the first embodiment. FIG. 7 is a fourth diagram showing an outline of area selection processing in the first embodiment. FIG. 8 is a diagram showing a processing flow of feature amount recording processing performed by the information processing device 1 in the first embodiment. FIG. 9 is a block diagram showing the configuration of an information processing device 1 in a second embodiment. FIG. 10 is a block diagram showing the configuration of an information processing device 1 in a third embodiment. FIG. 11 is a diagram showing an outline of area selection and size conversion processing in the third embodiment. FIG. 12 is a diagram showing a processing flow of feature amount recording processing performed by the information processing device 1 in the third embodiment. FIG. 13 is a diagram showing a processing flow of authentication processing performed by the information processing device 1 in the third embodiment. FIG. 14 is a block diagram showing the configuration of an information processing device 1 in a fourth embodiment. FIG. 15 is a diagram showing an outline of area selection processing in the fourth embodiment. FIG. 16 is a diagram showing a hardware configuration of an information processing device 1. FIG. 17 is a diagram showing the minimum configuration of an information processing device 1.

[0018] An information processing device 1 according to an embodiment of the present disclosure will be described below with reference to the drawings. The information processing device 1 according to this embodiment improves the authentication accuracy of a target in an authentication technique using ensemble learning.

[0019] <First embodiment> Fig. 1 is a block diagram showing the configuration of an information processing device 1 according to the first embodiment. As shown in Fig. 1, the information processing device 1 includes an image acquisition unit 10, a feature point detection unit 11, image region selection units 12.1 to 12.N, feature extraction units 13.1 to 13.N, a matching feature storage unit 14, score calculation units 15.1 to 15.N, a score integration unit 16, and an authentication determination unit 17.

[0020] The image acquisition unit 10 acquires an image of a living body to be authenticated, including at least the iris of the eye. The image may include not only the iris but also the sclera and surrounding area of ​​the eye. The iris indicates the pattern of muscle fibers in the eye that circle the pupil. The muscle fiber pattern of the iris has characteristics unique to each individual. The information processing device 1 of this embodiment authenticates the subject using at least iris pattern information. This is called iris authentication.

[0021] The feature point detection unit 11 detects feature points, which are characteristic information of the eyes, from the acquired image. The image region selection units 12.1, ..., 12.N select multiple different partial regions, including at least a portion of the iris region, from the image based on feature information, such as the eye feature points, detected by the feature point detection unit 11. The image region selection units 12.1, ..., 12.N operate in parallel, and each selects a different partial region from the acquired image. The image region selection units 12.1, ..., 12.N may select a partial region that includes the iris region. One or more of the image region selection units 12.1, ..., 12.N may select different partial regions of the eye that include the entire iris region. The image region selection units 12.1, ..., 12.N are collectively referred to as image region selection unit 12.

[0022] Feature extraction units 13.1, ..., 13.N extract feature quantities f1, ..., fn for partial regions a1, ..., partial regions an selected by image region selection units 12.1, ..., 12.N. Feature quantities are values ​​that represent iris features. Feature extraction units 13.1, ..., 13.N are collectively referred to as feature extraction unit 13.

[0023] The matching feature storage unit 14 stores a matching feature indicating a feature of a previously registered target. The matching feature is, for example, the Mth feature among a plurality of matching features of a person previously registered before authentication, and is the feature extracted by the feature extraction unit 13.M and recorded in the matching feature storage unit 14 in the previous feature registration process.

[0024] The score calculation units 15.1, ..., 15.N calculate scores SC1, ..., SCn for each partial region using the features f1, ..., feature fn extracted by the feature extraction units 13.1, ..., 13.N and the matching features f1, ..., matching features fn stored in the matching feature storage unit 14. The score here refers to the degree of similarity with the corresponding feature registered in advance. The score calculation units 15.1, ..., 15.N are collectively referred to as score calculation unit 15.

[0025] The score integrating unit 16 calculates an integrated score using the scores SC1, ..., SCn obtained from the score calculation units 15.1, ..., 15.N. The integrated score is a statistical value of the scores calculated by the score calculation units 15.1, ..., 15.N.

[0026] The authentication determination unit 17 determines whether to authenticate based on the integrated score obtained from the score integration unit 16 .

[0027] The subject to be authenticated by the information processing device 1 of this embodiment may be a human being or an animal such as a dog or a snake.

[0028] FIG. 2 is a diagram illustrating an outline of the feature point detection process. The feature point detection unit 11 may detect features such as an arbitrary coordinate p on the outline of the eyelid included in the acquired image, the center coordinate O1 of the pupil circle, the center coordinate O2 of the iris circle, the pupil radius r1, and the iris radius r2, and calculate the vector of these values ​​as feature point information. The positions of the center coordinate O1 of the pupil circle and the center coordinate O2 of the iris circle may be offset. The arbitrary coordinate p on the outline of the eyelid (upper eyelid, lower eyelid) included in the acquired image may be a value calculated using a predetermined position of the eye as the center of the image, for example. The predetermined position may be a point on the outer corner or the roof of the eye, or the midpoint of a line connecting the points on the outer corner and the roof of the eye.

[0029] FIG. 3 is a first diagram illustrating an overview of the region selection process. Image region selection units 12.1, ..., 12.N are referred to as image region selection unit 12. Image region selection unit 12 identifies the outer corner point p1 and inner corner point p2 of the eye in the acquired image (G11), calculates the angle θ between a line L1 passing through these points and the horizontal direction L2 of the image, and generates an image (G12) by rotating the image using the angle θ so that the line L1 connecting the outer corner point and inner corner point of the eye coincides with the horizontal direction L2 of the image. Generating this rotationally transformed image (G12) is one form of image normalization. Image region selection unit 12 identifies a predetermined partial region (G13) including the iris region in image (G12) and cuts out an image (G14) of that partial region. Image region selection units 12.1, ..., 12.N are pre-configured to cut out images of partial regions at different positions based on eye feature information.

[0030] FIG. 4 is a second diagram illustrating an overview of the region selection process. The image region selection unit 12 identifies the diameter of the pupil or the diameter of the iris in the eyeball of the acquired image (G21) and generates a reduced or enlarged image (G22) so that the pupil or iris diameters are predetermined values. In this case, the image region selection unit 12 may identify the number of pixels corresponding to the length of the pupil diameter and the number of pixels corresponding to the length of the iris diameter, based on the center coordinates of the pupil circle, and perform image processing such as affine transformation to maintain a constant ratio between the number of pixels corresponding to the iris diameter and the number of pixels corresponding to the pupil diameter, thereby generating the reduced or enlarged image. The generation of this reduced or enlarged image (G22) is one form of image normalization. The image region selection unit 12 may also identify the radius of the pupil or the radius of the iris in the eyeball of the acquired image (G21) and generate a reduced or enlarged image (G22) so that the radius of the pupil or the iris is predetermined values. Image region selection unit 12 identifies a predetermined partial region including the iris region in image (G22) (G23) and cuts out an image of that partial region (G24). Image region selection units 12.1, ..., 12.N cut out images of partial regions at different positions based on the eye feature information.

[0031] FIG. 5 is a third diagram illustrating an overview of the region selection process. The image region selection unit 12 generates an image (G32) by transforming the acquired image (G31) so that the position of the eye in the image is at the center of the image. The image region selection unit 12 then generates the transformed image (G32) so that the center coordinate of the iris circle is located at a predetermined position within the image and the diameter or radius of the pupil and iris are predetermined values. Generating this transformed image (G32) is one form of image normalization. The image region selection unit 12 may generate the transformed image (G32) by performing image processing such as affine transformation so that the number of pixels corresponding to the length of the iris radius based on the center coordinate of the iris circle is constant. The image region selection unit 12 identifies a predetermined partial region (G33) including the iris region in the image (G32) and cuts out an image (G34) of that partial region. The image region selection units 12.1, ..., 12.N each cut out an image of a partial region at a different position based on eye feature information.

[0032] The processing shown in FIGS. 3, 4, and 5 is one aspect of processing in which the region of a specific part of the eye in the acquired image is normalized to a specified orientation or a specified size.

[0033] FIG. 6 is a fourth diagram illustrating an overview of the region selection process. The image region selection unit 12 may sequentially perform one or more of the processes described above with reference to FIGS. 3, 4, and 5, and then cut out an image of a predetermined partial region based on the eye feature information. As shown in FIG. 6, the image region selection units 12.1, ..., 12.N cut out images of partial regions at different positions based on the eye feature information. The partial regions selected by each image region selection unit 12 may be multiple partial regions with different center positions. The partial regions selected by each image region selection unit 12 may be multiple partial regions with different sizes. Each image region selection unit 12 may select multiple partial regions including a partial region that includes the area within the eyeball and a partial region that includes the skin, etc., surrounding the eyeball. The image region selection unit 12 may select multiple different regions that include feature points around the eyeball. The information processing device 1 according to this embodiment performs ensemble learning using the feature amounts of images in different partial regions in this way, and improves the accuracy of authentication by using the feature amounts of images in the different partial regions in authentication processing.

[0034] 7 is a diagram showing a processing flow of the feature amount recording process performed by the information processing device 1 in the first embodiment. Next, the feature amount recording process of the information processing device 1 in the first embodiment will be described with reference to FIG.

[0035] In the preliminary feature recording process, a person inputs an image of their face to the information processing device 1. The information processing device 1 may use a predetermined camera to capture an image of the area including the person's eyes and acquire the image generated during the capture. The image acquisition unit 10 acquires an image including the person's eyes (step S11). The image acquisition unit 10 may acquire an image of the area including the person's eyes captured by a predetermined camera, or may acquire an image including the person's face and cut out an image of a predetermined area including the eyes from that image. The image is assumed to include at least one eye of the subject. The image is also assumed to capture the pupil and iris of the eye. The image acquisition unit 10 outputs the image to the feature point detection unit 11 and the image area selection units 12.1, ..., 12.N.

[0036] The feature point detection unit 11 detects eye feature points based on the acquired image (step S12). The feature point detection unit 11 may calculate a vector including the center coordinate and radius of an iris circle from the acquired image as information indicating the feature points. This vector is information representing the eye feature points. The feature point detection unit 11 may detect other features in the iris region as eye feature points. As described with reference to FIG. 2 , the feature point detection unit 11 may generate information representing the eye feature points using the center coordinates of the pupil circle, the center coordinates of the iris circle, the pupil radius, the iris radius, any coordinates on the outline of the eyelids (upper eyelid, lower eyelid), or the like included in the acquired image. For example, the feature point detection unit 11 may output, as information indicating the feature points, vectors representing the center position of the pupil circle, the pupil radius, and the position coordinates of feature points on the eyelids, in addition to the center position of the iris circle and the radius of the iris circle. The feature point detection unit 11 may calculate information indicating feature points, including the coordinates of the center of the outer circle of the iris, the radius of the outer circle of the iris, the coordinates of the outer corner of the eye, and the coordinates of the inner corner of the eye, as a vector. Alternatively, the feature point detection unit 11 may segment the circular regions of the outer and inner circles of the iris, the circular region of the pupil, and the eyelid region (skin) around the eye, perform circle detection on the resulting two-dimensional map, and calculate information indicating feature points, including the coordinates of the center of the outer circle of the iris, the radius of the outer circle of the iris, the coordinates of the outer corner of the eye, and output the information as a vector. If the feature point detection unit 11 is unable to detect the iris circle, it may calculate information indicating other feature points, including the coordinates of the outer corner of the eye, the coordinates of the inner corner of the eye, and output the information as a vector, excluding information about the iris region, such as the coordinates of the center of the outer circle of the iris and the radius of the outer circle of the iris. The feature point detection unit 11 may be configured, for example, as a recurrent neural network (RNN). The RNN may include multiple convolutional layers and multiple activation layers, extract feature points from an input image, convert the extracted feature points into vectors representing corresponding regions using a linear layer, and output the vectors as information representing the feature points. When the feature point detection unit 11 is constructed as a neural network, any neural network structure may be used as long as it meets the requirements.For example, the neural network structure may be similar to that of a VGG (Visual Geometry Group), a Residual Network (ResNet), or a DenseNet, but other structures may also be used. The feature point detection unit 11 may be an image processing mechanism other than a neural network. The feature point detection unit 11 may generate information representing eye feature points using an image after the conversion process (normalization) described with reference to FIGS. 3, 4, and 5 has been performed. The feature point detection unit 11 outputs information representing the feature points to image region selection units 12.1, ..., 12.N.

[0037] Image region selection units 12.1, ..., 12.N receive the image input from image acquisition unit 10 and information indicating feature points input from feature point detection unit 11. Each of image region selection units 12.1, ..., 12.N uses the image and information indicating feature points to select a different partial region using the method described with reference to Figures 3, 4, and 5 (step S13). Images of the partial regions selected by image region selection units 12.1, ..., 12.N are referred to as images of partial region a1, ..., partial region an, respectively. Image region selection unit 12.1 outputs partial region a1 to feature amount extraction unit 13.1. Image region selection unit 12.2 outputs partial region a2 to feature amount extraction unit 13.2. Similarly, image region selection units 12.3, ..., 12.N generate images of the partial regions selected by image region selection units 12.1, ..., 12.N. N outputs the generated image of the partial region to the corresponding feature extraction unit 13.

[0038] The feature extraction units 13.1, ..., 13.N extract features from the input partial region images after performing image preprocessing such as brightness histogram normalization, masking of areas other than the iris circle, and polar coordinate expansion with the center of the iris circle as the origin (step S14). The feature extraction units 13.1, ..., 13.N receive images of partial regions a1, ..., and an and extract features f1, ..., and fn. The feature extraction units 13.1, ..., 13.N may each extract features using a different method. The feature extraction units 13.1, ..., 13.N may be constructed, for example, with a convolutional neural network. The feature extraction units 13.1, ..., 13.N may be trained in advance using images of the partial regions selected by the image region selection units 12.1, ..., 12.N to appropriately extract features. The feature extraction unit 13 may be a weak learner using a model that can generate features with high accuracy, or may be another trained neural network. The feature extraction units 13.1, ..., 13.N may also be image processing mechanisms that extract features that are not constructed using neural networks.

[0039] Feature extraction units 13.1, ..., 13.N associate the extracted features f1, ..., feature fn (matching features) with identifiers of people appearing in the images used in the feature recording process and identifiers of feature extraction units 13 that extracted the features, and record the results in matching feature storage unit 14 (step S15). As a result, the eye features of the subjects appearing in the images used in the feature recording process, that is, the feature amounts of different partial regions of the eyes, are recorded in matching feature storage unit 14.

[0040] The information processing device 1 may perform the same process as above for each of the left and right eyes shown in the image, and may further link the feature amounts f1, ..., fn to the identifier of the left or right eye, and record the feature amounts f1, ..., fn in the matching feature storage unit 14. The information processing device 1 may also perform the same feature recording process using images of many targets for which authentication is performed and predetermined services or processing functions are provided, and similarly record the feature amounts f1, ..., fn in the matching feature storage unit 14. This completes the description of the preliminary feature recording process.

[0041] 8 is a diagram showing a processing flow of authentication processing performed by the information processing device 1 in the first embodiment. Next, the authentication processing of the information processing device 1 in the first embodiment will be described with reference to FIG.

[0042] In the authentication process, a person inputs an image of their face into the information processing device 1. The information processing device 1 may photograph the person using a predetermined camera and acquire the image generated during the photographing. The image acquisition unit 10 acquires an image including the person's eyes (step S21). The image is assumed to include at least one eye of the subject. The image is also assumed to show the iris of the eye. The image acquisition unit 10 outputs the image to the feature point detection unit 11 and the image area selection units 12.1, ..., 12.N.

[0043] The feature point detection unit 11 detects eye feature points based on the acquired image (step S22). This process is similar to the process in step S12 described above in the feature amount recording process.

[0044] Image region selection units 12.1, ..., 12.N receive the image input from image acquisition unit 10 and information indicating feature points input from feature point detection unit 11. Using the image and the information indicating feature points, image region selection units 12.1, ..., 12.N each select a different partial region using the method described with reference to Figures 3, 4, and 5 (step S23). This process is similar to the process in step S13 described in the feature amount recording process above.

[0045] The feature extraction units 13.1, ..., 13.N extract features from the input partial region images (step S24). This process is the same as the process in step S14 described in the feature recording process above. The feature extraction units 13.1, ..., 13.N output the extracted features f1, ..., feature fn to the corresponding score calculation units 15.

[0046] The score calculation units 15.1, ..., 15.N acquire features f1, ..., fn extracted in the authentication process from the corresponding feature extraction units 13. The score calculation units 15.1, ..., 15.N also acquire corresponding features (features f1, ..., fn) of one person extracted in the feature recording process and recorded in the matching feature storage unit 14. The score calculation units 15.1, ..., 15.N calculate scores using the features extracted in the authentication process and the features extracted in the feature recording process (step S25). The scores calculated by the score calculation units 15.1, ..., 15.N are referred to as scores SC1, ..., SCn, respectively.

[0047] The score calculation units 15.1, ..., 15.N may calculate the scores SC1, ..., SCn using, for example, the cosine similarity between the feature extracted in the authentication process and the feature extracted in the feature recording process. Alternatively, the score calculation units 15.1, ..., 15.N may calculate the scores using an L2 distance (Euclidean distance) function or an L1 distance (Manhattan distance) function between the feature extracted in the authentication process and the feature extracted in the feature recording process. The score calculation units 15.1, ..., 15.N may determine whether the features of data related to the same person are similar by utilizing the property that cosine similarity, L2 distance function, L1 distance function, or the like tends to result in close distances.

[0048] The score calculation units 15.1, ..., 15.N may be constructed, for example, by a neural network. Furthermore, the score calculation units 15.1, ..., 15.N may be a score calculation processing mechanism that is not constructed by a neural network, and may calculate scores based on the Hamming distance between the feature extracted in the authentication processing and the feature extracted in the feature recording processing, for example. The score calculation units 15.1, ..., 15.N output the calculated scores to the score integration unit 16. The scores calculated by the score calculation units 15.1, ..., 15.N are referred to as scores SC1, ..., SCn, respectively.

[0049] The score integrating unit 16 calculates an integrated score using the scores SC1, ..., SCn (step S26). The score integrating unit 16 may calculate the integrated score by, for example, selecting the maximum value from the scores SC1, ..., SCn as the integrated score. Alternatively, the score integrating unit 16 may calculate the integrated score using a representative value such as the average or mode of the scores SC1, ..., SCn. Alternatively, the score integrating unit 16 may calculate the integrated score using an estimation method such as a regression neural network (RNN) or a support vector machine that inputs the scores SC1, ..., SCn.

[0050] The score integrating unit 16 may use an average of the scores or a weighted average as a means for calculating the integrated score. Alternatively, the score integrating unit 16 may select the largest score and calculate it as the integrated score. Alternatively, the score integrating unit 16 may be constructed using, for example, a neural network. Alternatively, the score integrating unit 16 may be a processing mechanism other than a neural network, such as logistic regression or Ridge regression. The score integrating unit 16 outputs the integrated score to the authentication assessment unit 17.

[0051] The authentication assessment unit 17 acquires the integrated score. The authentication assessment unit 17 authenticates the person appearing in the image using the integrated score (step S27). For example, if the integrated score is equal to or greater than a threshold, the authentication assessment unit 17 determines that the person appearing in the image is a registered person and outputs information indicating successful authentication. If the integrated score is less than the threshold, the authentication assessment unit 17 determines that the person appearing in the image is an unregistered person and outputs information indicating unsuccessful authentication. The authentication assessment unit 17 may identify, in the matching feature storage unit 14, the matching feature used to calculate the highest integrated score among the integrated scores equal to or greater than the threshold, and identify the person appearing in the image based on the person's identifier associated with the matching feature. The authentication assessment unit 17 may determine that authentication is unsuccessful if the difference between the highest integrated score and the next highest integrated score among the integrated scores equal to or greater than the threshold is equal to or less than a predetermined threshold.

[0052] In the information processing device 1 described above, the image region selection unit 12 selects a plurality of different partial regions, each including at least a portion of the iris region, based on the features of the target's eye included in the acquired image, and the feature extraction unit 13 calculates the feature values ​​for each of the different partial regions. Furthermore, the score calculation unit 15 calculates the similarity between each of the different partial regions and the pre-stored feature values ​​based on the feature values ​​for each of the different partial regions, and the authentication assessment unit 17 authenticates the target based on the similarity values ​​for each of the different partial regions. This process enables authentication to be performed using ensemble learning using different weak learners corresponding to the different partial regions including the iris, thereby easily improving the authentication accuracy of the target.

[0053] Iris recognition technology requires high accuracy for practical use. While using higher-resolution, more focused images is one way to improve accuracy, capturing images with a higher pixel count requires more expensive cameras and stricter imaging environment constraints. Therefore, a method is needed to improve authentication accuracy by modifying the iris recognition process without satisfying these constraints. Ensemble learning is a method for improving the accuracy of estimating whether a target is pre-stored for authentication. Ensemble learning is a method that combines the estimation results of multiple learning modules to achieve higher accuracy than the estimation results of individual learning modules. Effective ensemble learning requires each learning module to be able to make accurate estimations and to minimize the correlation between their estimation results. Common ensemble learning methods use random numbers to divide and generate training data or connect weak learning modules to improve the effectiveness of the ensemble. However, these methods require trial and error to improve performance, resulting in high training costs.

[0054] When an image including eyes is input, the information processing device 1 according to this embodiment detects feature points and selects predetermined partial regions using the detected feature points, thereby obtaining multiple partial regions each with different characteristics regardless of the iris position or rotation state in the eye image. Because these partial region images contain iris information but include different regions, feature amounts with low correlation with each other can be reliably extracted. This allows the information processing device 1 according to this embodiment to perform effective ensemble learning without the need for trial and error using random numbers as in general ensemble learning methods.

[0055] Second Embodiment Fig. 9 is a block diagram showing the configuration of an information processing device 1 according to a second embodiment. As shown in Fig. 9, the information processing device 1 includes an image acquisition unit 10, a feature point detection unit 11, image region selection units 22.1, ..., 22.N, feature extraction units 13.1, ..., 13.N, a matching feature storage unit 14, score calculation units 15.1, ..., 15.N, a score integration unit 16, and an authentication determination unit 17. In the information processing device 1 according to the second embodiment, the processing of the image region selection units 22.1, ..., 22.N differs from the processing of the image region selection units 12.1, ..., 12.N of the information processing device 1 according to the first embodiment. The processing of the other processing units is the same as that described in the first embodiment. The image region selection units 22.1, ..., 22.N are collectively referred to as the image region selection unit 22.

[0056] FIG. 10 is a diagram illustrating an overview of the region selection process in the second embodiment. Based on the center position of the iris detected by the feature point detection unit 11, the image region selection units 22.1, ..., 22.N select partial regions of different areas that have the same center as the iris center, as shown in FIG. 10 . In the example of FIG. 10 , the image region selection units 22.1, ..., 22.N select rectangles (quadrilaterals) that have the same center as the iris center and vertices that are different distances from the center as partial regions. The image region selection units 22.1, ..., 22.N may each select triangular or polygonal partial regions that have the same center as the iris center and vertices that are different distances from the center as well as polygonal partial regions with pentagons or more. The image region selection units 22.1, ..., 22.N may each select circular partial regions that have the same center as the iris center and have different radii that are different distances from the center as well. Each partial region may include the entire iris. Alternatively, one or more of the partial regions having a small area may include a part of the iris. The example in Fig. 10 shows one mode of processing in which the image region selection unit 22 selects multiple partial regions having the same center position but different sizes.

[0057] The preliminary feature recording process (steps S11 to S15) and authentication process (steps S21 to S27) performed by the information processing device 1 in the second embodiment are similar to those in the first embodiment, except for the method of selecting partial regions in the image region selection units 22.1, ..., 22.N, as described above. Each of the image region selection units 22.1, ..., 22.N selects a rectangular, polygonal, or circular partial region of a predetermined size with the center position of the iris based on a predetermined program. As a result, each of the image region selection units 22.1, ..., 22.N selects partial regions of different sizes with the center position of the iris center. The image region selection units 22.1, ..., 22.N according to the second embodiment may select partial regions using an image that has undergone the conversion process (normalization) described with reference to FIGS. 3, 4, and 5 in the first embodiment.

[0058] According to the processing of the information processing device 1 of the second embodiment described above, each of the image region selection units 22.1, ..., 22.N selects a partial region of a different size whose center is the center of the iris, and extracts different features from each partial region, and performs authentication using ensemble learning for each feature. This allows the information processing device 1 to perform iris authentication with higher accuracy than when authentication is performed using only features from an image containing a single iris.

[0059] <Third Embodiment> Fig. 11 is a block diagram showing the configuration of an information processing device 1 according to a third embodiment. As shown in Fig. 11, the information processing device 1 includes an image acquisition unit 10, a feature point detection unit 11, an image region selection unit 32, size conversion units 33.1 to 33.N, feature extraction units 13.1 to 13.N, a matching feature storage unit 14, score calculation units 15.1 to 15.N, a score integration unit 16, and an authentication determination unit 17. The information processing device 1 according to the third embodiment differs from the first and second embodiments in that it includes one image region selection unit 32 and size conversion units 33.1 to 33.N. The processing of the processing units other than the image region selection unit 32 and the size conversion units 33.1 to 33.N is the same as that described in the first embodiment.

[0060] FIG. 12 is a diagram illustrating an overview of the region selection and size conversion process. The image region selection unit 32 selects a partial region (iris region) containing the iris in the acquired image using feature point information, including the center position of the iris and the radius of the iris, acquired from the feature point detection unit 11. This partial region may be selected so as to include the entire iris region. The selected partial region may not include part of the iris. In other words, the partial region may be at least a part of the iris. In the following description, it is assumed that the image region selection unit 32 selects a circular region surrounded by the outer circle of the iris. The image region selection unit 32 outputs the partial regions representing the selected circular regions to the size conversion units 33.1, ..., 33.N. Note that the image processed by the image region selection unit 32 may be an image after the conversion process (normalization) described with reference to FIGS. 3, 4, and 5 in the first embodiment. 11 shows only one image region selection unit 32, multiple image region selection units 32 may similarly select the same partial region and output the image information to size conversion units 33.1, ..., 33.N.

[0061] The size conversion units 33.1, ..., 33.N acquire image information for the same partial region from the image region selection unit 32. The size conversion units 33.1, ..., 33.N each convert the acquired image information for the same partial region into different sizes. The number of pixels in the image of the partial region after each size conversion by the size conversion units 33.1, ..., 33.N may be different. The size conversion units 33.1, ..., 33.N may perform size conversion using, for example, a nearest neighbor interpolation method, a bilinear interpolation method, a bicubic interpolation method, or the like, as an image interpolation method that preserves the individuality of the iris. The size conversion units 33.1, ..., 33.N may also perform size conversion using other interpolation methods. Furthermore, the size conversion units 33.1, ..., 33.N may be configured using a neural network.

[0062] 13 is a diagram showing a processing flow of the feature recording process performed by the information processing device 1 in the third embodiment. After the information processing device 1 performs steps S11 and S12 in the feature recording process of the third embodiment, the image region selection unit 32 selects a partial region (iris region) including the iris in the acquired image by the process described with reference to FIG. 12 (step S33). The size conversion units 33.1, ..., 33.N each convert the size of the acquired image information of the same partial region (iris region) into different sizes (step S34).

[0063] Then, similar to the process of step S14 in the first embodiment, the feature extraction units 13.1, ..., 13.N extract features after performing image preprocessing on the input partial region image, such as normalizing the brightness histogram, masking areas other than the iris circle, and polar coordinate expansion with the center of the iris circle as the origin (step S14). Also, similar to the process of step S15 in the first embodiment, the feature extraction units 13.1, ..., 13.N associate the extracted features f1, ..., fn (matching features) with identifiers of people appearing in the images used in the feature recording process and identifiers of the feature extraction units 13 that extracted the features, and record the results in the matching feature storage unit 14 (step S15).

[0064] 14 is a diagram showing the processing flow of authentication processing performed by information processing device 1 in the third embodiment. In the authentication processing of the third embodiment, information processing device 1 performs steps S21 and S22 in the same way as in the first embodiment. After that, image region selection unit 32 selects a partial region (iris region) including an iris in the acquired image by the process described with reference to FIG. 12 (step S43). Size conversion units 33.1, ..., 33.N each convert the size of the acquired image information of the same partial region (iris region) into different sizes (step S44).

[0065] The subsequent processing is the same as in the first embodiment. That is, the feature extraction units 13.1, ..., 13.N extract features from the input partial region (iris region) image (step S24). The score calculation units 15.1, ..., 15.N calculate scores using the features extracted in the authentication process and the features extracted in the feature recording process (step S25). The scores calculated by the score calculation units 15.1, ..., 15.N are respectively referred to as scores SC1, ..., SCn. The score integration unit 16 calculates an integrated score using scores SC1, ..., SCn (step S26). The authentication assessment unit 17 uses the integrated score to authenticate the target person appearing in the image (step S27).

[0066] According to the processing of the information processing device 1 of the third embodiment described above, feature extraction units 13.1, ..., 13.N can extract multiple individual features, each of which is different, based on the size-converted image of a partial region including an iris output by size conversion units 33.1, ..., 33.N. The information processing device 1 performs authentication using ensemble learning with each of these features. This allows the information processing device 1 to perform iris authentication with higher accuracy than when authentication is performed using only features from an image including a single iris.

[0067] <Fourth Embodiment> Fig. 15 is a block diagram showing the configuration of an information processing device 1 according to the fourth embodiment. As shown in Fig. 15, the information processing device 1 includes an image acquisition unit 10, a feature point detection unit 11, image region selection units 42.1 and 42.2, feature extraction units 13.1 and 13.2, a matching feature storage unit 14, score calculation units 15.1 and 15.2, a score integration unit 16, and an authentication determination unit 17. In the information processing device 1 according to the fourth embodiment, the processing of image region selection units 42.1 and 42.2 differs from the processing of image region selection units 12.1, ..., 12.N of the information processing device 1 according to the first embodiment. The processing of the other processing units is the same as that described in the first embodiment.

[0068] FIG. 16 is a diagram illustrating an outline of the region selection process in the fourth embodiment. Using feature point information including the center position and radius of the iris acquired from the feature point detection unit 11, the image region selection units 42.1 and 42.2 select a partial region (iris region) of the same size as the outer circle of the iris in the acquired image, and a partial region (eye peripheral region) including the outer circle of the iris and other eye peripheral features (see FIG. 16 ). The processing in the processing units other than the image region selection units 42.1 and 42.2 is the same as in the first embodiment. Note that the images processed by the image region selection units 42.1 and 42.2 may be images that have undergone the conversion process (normalization) described with reference to FIGS. 3 , 4 , and 5 in the first embodiment. The processing in the fourth embodiment is one aspect of the process in which the image region selection unit 42 selects multiple different partial regions, including partial regions including the area within the eyeball and partial regions including the periphery of the eyeball.

[0069] The iris region can distinguish multiple people if a high-resolution image can be acquired, and changes over time are extremely small. However, it has the problem of being an area where image degradation due to low resolution, focus loss, camera shake, etc. is easily noticeable. On the other hand, an image of the area around the eyes contains information about the eyelids and cannot distinguish multiple people as well as the iris region. However, even at relatively low resolution, the image retains individual characteristics and has characteristics different from the iris. By extracting features from the iris image and the image of the partial area including the area around the eyes, it is possible to obtain features that complement each other while retaining individuality. The information processing device 1 performs authentication using ensemble learning with each of these features. This allows the information processing device 1 to perform iris authentication with higher accuracy and robust authentication processing compared to authentication using only features from an image containing a single iris.

[0070] In the above embodiments, the information processing device 1 that performs authentication using an iris image has been described as an example, but this ensemble learning technology can also be applied to other image processing fields, such as detection of fraudulent authentication (presentation attack) using counterfeits that simulate biometric information (PAD) and facial recognition.

[0071] According to the processing of the information processing device 1 in each of the above-described embodiments, by using feature information such as eye positions to perform ensemble learning using images showing different partial regions including at least the iris, estimation processing including authentication processing can be performed with low learning cost and high accuracy. The reason for the high accuracy of estimation processing is that, unlike methods for creating training datasets using random numbers, etc., weak learners are trained using images of partial regions selected using detected eye position information. By using features unique to individuals or other subjects while training each weak learner using images of different partial regions, output correlation can be reliably reduced. The reason for the low learning cost is that the learning of the weak learners in the above-described embodiments can be performed in parallel, eliminating the need for repeated learning.

[0072] 17 is a hardware configuration diagram of the information processing device 1. As shown in this diagram, the information processing device 1 may be a computer including various pieces of hardware such as a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a database 104, and a communication module 105. The functions of the information processing device 1 according to each of the above-described embodiments may be realized by an information processing system in which a plurality of information processing devices are provided with one or more of the above-described functions and are configured to work together to perform overall processing.

[0073] FIG. 18 is a diagram showing the minimum configuration of the information processing device 1. As shown in this diagram, the information processing device 1 performs at least the functions of an area selection means 81, a feature calculation means 82, a similarity calculation means 83, and an authentication means 84. The area selection means 81 selects a plurality of different partial areas, including at least a part of the iris area, based on the features of the target's eyes included in the acquired image. The feature calculation means 82 calculates the feature amounts of each of the different partial areas. The similarity calculation means 83 calculates the similarity between each of the different partial areas and the pre-stored feature amounts based on the feature amounts of each of the different partial areas and the pre-stored feature amounts. The authentication means 84 authenticates the target based on the similarity between each of the different partial areas.

[0074] The program may be a program for realizing some of the functions described above, or may be a so-called differential file (differential program) that can realize the functions described above in combination with a program already recorded in the computer system.

[0075] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes.

[0076] (Supplementary Note 1) An information processing device comprising: an area selection means for selecting a plurality of different partial areas, each of which includes at least a part of an iris area, based on eye features of a subject included in an acquired image; a feature calculation means for calculating a feature value for each of the different partial areas; a similarity calculation means for calculating a similarity between a pre-stored feature value and each of the different partial areas, based on the feature value of each of the different partial areas and a pre-stored feature value; and an authentication means for authenticating the subject based on the similarity between each of the different partial areas.

[0077] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the region selection means selects the plurality of different partial regions having different center positions.

[0078] (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the region selection means selects the plurality of different partial regions having different sizes.

[0079] (Supplementary Note 4) The information processing device according to Supplementary Note 1, wherein the region selection means selects the plurality of different partial regions having the same center position and different sizes.

[0080] (Supplementary Note 5) The information processing device according to Supplementary Note 1, wherein the region selection means selects a plurality of different partial regions consisting of a partial region that includes an area within the eyeball and a partial region that includes a periphery of the eyeball.

[0081] (Supplementary Note 6) The information processing device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the region selection means selects the plurality of different regions including feature points around an eyeball of the eye.

[0082] (Supplementary Note 7) The information processing device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the acquired image is normalized based on at least one of a specified orientation and a specified size of the region of the specific part of the eye.

[0083] (Supplementary Note 8) An information processing device comprising: an area selection means for selecting one area including at least a part of an iris area based on features of the eye of a subject included in an acquired image; a size conversion means for converting the one area into different partial areas having different numbers of pixels; a feature calculation means for calculating features of each of the different partial areas; a similarity calculation means for calculating similarities between pre-stored features and each of the different partial areas based on the features of each of the different partial areas and pre-stored features; and an authentication means for authenticating the subject based on the similarities of each of the different partial areas.

[0084] (Supplementary Note 9) An information processing system comprising: an area selection means for selecting a plurality of different partial areas, each of which includes at least a part of an iris area, based on eye features of a subject included in an acquired image; a feature calculation means for calculating a feature value for each of the different partial areas; a similarity calculation means for calculating a similarity between a pre-stored feature value and each of the different partial areas, based on the feature value of each of the different partial areas and a pre-stored feature value; and an authentication means for authenticating the subject based on the similarity between each of the different partial areas.

[0085] (Supplementary Note 10) An information processing system comprising: an area selection means for selecting one area including at least a part of an iris area based on features of the eye of a subject included in an acquired image; a size conversion means for converting the one area into different partial areas having different numbers of pixels; a feature calculation means for calculating features of each of the different partial areas; a similarity calculation means for calculating similarities between pre-stored features and each of the different partial areas based on the features of each of the different partial areas and pre-stored features; and an authentication means for authenticating the subject based on the similarities of each of the different partial areas.

[0086] (Supplementary Note 11) An authentication method comprising: selecting a plurality of different partial regions, each of which includes at least a part of an iris region, based on eye features of a subject included in an acquired image; calculating a feature value for each of the different partial regions; calculating a similarity between the pre-stored feature value and each of the different partial regions based on the feature value for each of the different partial regions and a pre-stored feature value; and authenticating the subject based on the similarity between each of the different partial regions.

[0087] (Supplementary Note 12) An authentication method comprising: selecting one region including at least a part of an iris region based on eye features of a subject included in an acquired image; converting the one region into different partial regions with different numbers of pixels; calculating feature values ​​for each of the different partial regions; calculating a similarity between the pre-stored feature values ​​and each of the different partial regions based on the feature values ​​for each of the different partial regions and pre-stored feature values; and authenticating the subject based on the similarity between each of the different partial regions.

[0088] (Supplementary Note 13) A storage medium storing a program that causes a computer of an information processing device to function as: an area selection means that selects a plurality of different partial areas from partial areas including at least a part of an iris area based on features of the eyes of a subject included in an acquired image; a feature calculation means that calculates feature values ​​for each of the different partial areas; a similarity calculation means that calculates similarities between pre-stored feature values ​​and each of the different partial areas based on the feature values ​​of each of the different partial areas and pre-stored feature values; and an authentication means that authenticates the subject based on the similarities between each of the different partial areas.

[0089] (Supplementary Note 14) A storage medium storing a program that causes a computer of an information processing device to function as: an area selection means that selects one area including at least a part of an iris area based on features of the eye of a subject included in an acquired image; a size conversion means that converts the one area into different partial areas with different numbers of pixels; a feature calculation means that calculates features of each of the different partial areas; a similarity calculation means that calculates similarities between pre-stored features and each of the different partial areas based on the features of each of the different partial areas and pre-stored features; and an authentication means that authenticates the subject based on the similarities of each of the different partial areas.

[0090] 1... Information processing device 1 (information processing device, information processing system), 10... Image acquisition unit, 11... Feature point detection unit, 12 (12.1, 12.2, ... 12.N), 22 (22.1, 22.2, ... 22.N), 32, 42 (42.1, 42.4)... Image region selection unit (region selection means), 13 (13.1, 13.2, ... 13.N)... Feature extraction unit (feature calculation means), 14... Matching feature storage unit, 15 (15.1, 15.2, ... 15.N)... Score calculation unit (similarity calculation means), 16... Score integration unit (similarity calculation means), 17... Authentication judgment unit (authentication means)

Claims

1. a region selection means for selecting a plurality of different partial regions including at least a part of an iris region based on characteristics of an eye of a subject included in an acquired image; A feature amount calculation means for calculating a feature amount of each of the different partial regions; a similarity calculation means for calculating a similarity between the pre-stored feature amount and each of the different partial regions based on the feature amount of each of the different partial regions and a pre-stored feature amount; an authentication means for authenticating the object based on the similarity of each of the different partial regions; An information processing device comprising:

2. The region selection means selects the plurality of different partial regions having different center positions. The information processing device according to claim 1 .

3. The information processing apparatus according to claim 2 , wherein the region selection means selects the plurality of different partial regions having different sizes.

4. The area selection means selects the plurality of different partial areas having the same center position and different sizes. The information processing device according to claim 1 .

5. The region selection means selects a plurality of different partial regions consisting of a partial region including an area within the eyeball and a partial region including a periphery of the eyeball. The information processing device according to claim 1 .

6. The information processing apparatus according to claim 1 , wherein the region selection means selects the plurality of different regions including feature points around an eyeball of the eye.

7. The acquired image is normalized based on at least one of a specified orientation and a specified size of the region of the specific part of the eye. The information processing device according to any one of claims 1 to 4.

8. a region selection means for selecting a plurality of different partial regions including at least a part of an iris region based on characteristics of an eye of a subject included in an acquired image; A feature amount calculation means for calculating a feature amount of each of the different partial regions; a similarity calculation means for calculating a similarity between the pre-stored feature amount and each of the different partial regions based on the feature amount of each of the different partial regions and a pre-stored feature amount; an authentication means for authenticating the object based on the similarity of each of the different partial regions; An information processing system comprising:

9. selecting a plurality of different sub-regions, the sub-regions including at least a portion of the iris region, based on eye features of the subject included in the acquired image; Calculating feature amounts for each of the different partial regions; calculating a similarity between the pre-stored feature amount and each of the different partial regions based on the feature amount of each of the different partial regions and the pre-stored feature amount; authenticating the object based on the similarity of each of the different sub-regions; Authentication method.

10. The computer of the information processing device, a region selection means for selecting a plurality of different partial regions including at least a part of an iris region based on eye features of a subject included in an acquired image; A feature amount calculation means for calculating a feature amount of each of the different partial regions; a similarity calculation means for calculating a similarity between the pre-stored feature amount and each of the different partial regions based on the feature amount of each of the different partial regions and a pre-stored feature amount; an authentication means for authenticating the object based on the similarity of each of the different partial regions; A program that functions as a