Information processing system, information processing device, information processing method, and recording medium

Abstract

An information processing system according to the present invention comprises a registration storage unit, an imaging unit, a calculation unit, and an iris registration unit. The registration storage unit stores registered face information pertaining to a face and personal identification information in association with each other. The imaging unit images a subject and acquires a subject face image and a subject iris image, which include the face and the iris of the subject, respectively. The calculation unit calculates, on the basis of iris extraction information which has been extracted from the subject iris image using a first extraction engine, a first affinity score which indicates the affinity between the subject iris image and the first extraction engine. The iris registration unit further associates, with the registered face information and the personal identification information, iris information which pertains to a subject iris image selected on the basis of the first affinity score, and stores the iris information in association with the registered face information and the personal identification information in the registration storage unit.

Description

Information processing system, information processing device, information processing method, and recording medium

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a recording medium.

[0002] Iris images are commonly used for identity authentication. In this type of authentication, a match is generally performed using feature information (e.g., feature vectors) extracted from a previously registered iris image and an iris image for authentication. Depending on the match result, authentication is performed to determine, for example, whether the person shown in the iris image for authentication is the same as the person shown in the previously registered iris image.

[0003] For example, Patent Document 1 describes a method for determining whether a captured image can be used for authentication, regardless of whether authentication has succeeded or failed. According to the description in Patent Document 1, if the captured image has low resolution, is blurred, is overexposed, the subject's eyes are closed, or the like, it is determined that the captured image cannot be used for authentication.

[0004] International Publication No. 2021 / 199188

[0005] The present disclosure aims to improve upon the techniques described in the prior art documents mentioned above.

[0006] The information processing system of the present disclosure comprises a registration storage means for storing registered face information relating to a face in association with personal identification information; a photographing means for photographing a target to obtain a target face image and a target iris image including the face and iris of the target, respectively; a calculation means for calculating a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine; and an iris registration means for storing iris information relating to the target iris image selected based on the first affinity score in the registration storage means in further association with the registered face information and the personal identification information.

[0007] The information processing device of the present disclosure comprises an image acquisition means for acquiring a target face image and a target iris image obtained by photographing a target, a calculation means for calculating a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine, and an iris registration means for storing iris information related to the target iris image selected based on the first affinity score in a registration storage means.

[0008] The information processing method of the present disclosure includes one or more computers acquiring a target face image and a target iris image by photographing a target, calculating a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine, and storing iris information related to the target iris image selected based on the first affinity score in a registration memory unit.

[0009] The recording medium in the present disclosure is a recording medium having recorded thereon a program for causing one or more computers to acquire a target face image and a target iris image obtained by photographing a target, calculate a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine, and store iris information related to the target iris image selected based on the first affinity score in a registration memory unit.

[0010] 1 is a block diagram showing an example configuration of a first information processing system according to the present disclosure. FIG. 2 is a block diagram showing an example configuration of a first information processing device according to the present disclosure. FIG. 3 is a flowchart showing an example processing operation of the first information processing device according to the present disclosure. FIG. 4 is a diagram showing an example device configuration of the first information processing system according to the present disclosure. FIG. 5 is a block diagram showing a detailed example configuration of the first information processing device according to the present disclosure. FIG. 6 is a flowchart showing an example processing operation of the first information processing system according to the present disclosure. FIG. 7 is a block diagram showing an example configuration of a first imaging unit according to the present disclosure. FIG. 8 is a flowchart showing an example processing operation of the first imaging unit according to the present disclosure. FIG. 9 is a diagram showing an example position of an image, etc. in a feature space according to the present disclosure. FIG. 10 is a block diagram showing an example configuration of a first iris registration unit according to the present disclosure. FIG. 11 is a flowchart showing an example processing operation of the first iris registration unit according to the present disclosure. FIG. 12 is a diagram showing an example physical configuration of a first information processing device according to the present disclosure. FIG. 13 is a block diagram showing an example configuration of a second information processing system according to the present disclosure. FIG. 14 is a block diagram showing an example configuration of a second information processing device according to the present disclosure. FIG. 15 is a flowchart showing an example processing operation of the second information processing system according to the present disclosure. FIG. 16 is a block diagram showing an example configuration of a third information processing system according to the present disclosure. FIG. 17 is a block diagram showing an example configuration of a third information processing device according to the present disclosure. FIG. 18 is a flowchart showing an example processing operation of the third information processing system according to the present disclosure. FIG. 1 is a flowchart showing an example of a processing operation of a third information processing system according to the present disclosure. FIG. 2 is a block diagram showing an example of a configuration of a second imaging unit according to the present disclosure. FIG. 3 is a diagram showing an example of a device configuration of a fourth information processing system according to the present disclosure. FIG. 4 is a block diagram showing an example of a configuration of a fourth information processing device according to the present disclosure. FIG. 5 is a flowchart showing an example of a processing operation of the fourth information processing system according to the present disclosure. FIG. 6 is a block diagram showing an example of a configuration of a second iris registration unit according to the present disclosure. FIG. 7 is a flowchart showing an example of a processing operation of the second iris registration unit according to the present disclosure. FIG. 8 is a diagram showing an example of a device configuration of a fifth information processing system according to the present disclosure. FIG. 9 is a block diagram showing an example of a configuration of a fifth information processing device according to the present disclosure. FIG. 10 is a flowchart showing an example of a processing operation of the fifth information processing system according to the present disclosure.FIG. 1 is a block diagram showing a configuration example of a third iris registration unit according to the present disclosure. FIG. 2 is a flowchart showing a processing operation example of the third iris registration unit according to the present disclosure. FIG. 3 is a diagram showing a device configuration example of a sixth information processing system according to the present disclosure. FIG. 4 is a block diagram showing a configuration example of a sixth information processing device according to the present disclosure. FIG. 5 is a flowchart showing a processing operation example of the sixth information processing system according to the present disclosure. FIG. 6 is a block diagram showing a configuration example of a third imaging unit according to the present disclosure. FIG. 7 is a flowchart showing a processing operation example of the third imaging unit according to the present disclosure. FIG. 8 is a block diagram showing a configuration example of a third iris registration unit according to the present disclosure. FIG. 9 is a flowchart showing a processing operation example of the third iris registration unit according to the present disclosure.

[0011] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In addition, in all drawings, similar components are given similar reference numerals and descriptions thereof will be omitted as appropriate.

[0012] [First Embodiment] (Summary) In Patent Document 1, it is determined that a captured image cannot be used for authentication if the image has low resolution, is blurred, is overexposed, the eyes are closed, etc. Generally, low resolution, blurred, is overexposed, the eyes are closed, etc. are factors that cause image degradation, which may result in a decrease in authentication accuracy.

[0013] However, in general authentication using an iris image, authentication is often performed using an iris feature vector, which is a feature vector extracted from the iris image using a first extraction engine. The degree to which each of the above-mentioned image degradation factors affects the accuracy of authentication often depends on the characteristics of the specific first extraction engine used for authentication.

[0014] Therefore, even if the above-described degradation factors are used to determine whether or not a captured image can be used for authentication, it may be difficult to perform authentication with high accuracy.

[0015] For example, Patent Document 2 (International Publication No. 2021 / 177214) states that if there is master face information whose score for authentication face information is equal to or greater than a standard value, the settlement device will execute settlement processing using payment information corresponding to the master face information.

[0016] Furthermore, for example, Patent Document 3 (JP 2007-159610 A) describes a registration process in which the eye image evaluated as the best among a plurality of eye images is stored in a storage unit as registered authentication information. It also describes the use of eye openness and image quality for this evaluation. The eye openness described in Patent Document 3 indicates whether the eyes of the person to be authenticated are wide open. The image quality described in Patent Document 3 uses a method in which an image is frequency-analyzed, the amount of information of predetermined high-frequency components is integrated, and the larger the integrated value, the better the image quality is determined.

[0017] As described above, neither of the documents mentions that the degree to which each of the image degradation factors affects the accuracy of authentication depends on the characteristics of the first extraction engine used for authentication. Therefore, even if the technologies of Patent Documents 2 and 3 are used, it may be difficult to perform authentication with high accuracy.

[0018] One of the objectives of the present disclosure is to improve the accuracy of authentication using iris information.

[0019] As shown in FIG. 1, the information processing system S1 includes a registration storage unit 110, an imaging unit 50, a calculation unit 130, and an iris registration unit 140.

[0020] The registration storage unit 110 stores registered face information relating to faces and personal identification information in association with each other.

[0021] The photographing unit 50 photographs a target to obtain a target face image and a target iris image, each including the face and iris of the target.

[0022] The calculation unit 130 calculates a first affinity score indicating the affinity between the target iris image and the first extraction engine based on iris extraction information extracted from the target iris image using the first extraction engine.

[0023] The iris registration unit 140 stores iris information related to the target iris image selected based on the first affinity score in the registration storage unit 110. The iris registration unit 140 further associates the iris information with registered face information and personal identification information and stores the iris information in the registration storage unit 110.

[0024] According to this information processing system S1, the first affinity score indicates the affinity between the target iris image and the first extraction engine. Therefore, it is possible to select a target iris image that can be accurately authenticated using the first extraction engine, and perform authentication using iris information related to the target iris image. Therefore, it is possible to improve the accuracy of authentication using iris information.

[0025] As shown in FIG. 2, the information processing device 100 includes an image acquisition unit 120, a calculation unit 130, and an iris registration unit 140.

[0026] The image acquisition unit 120 acquires a target face image and a target iris image.

[0027] The calculation unit 130 calculates a first affinity score indicating the affinity between the target iris image and the first extraction engine based on iris extraction information extracted from the target iris image using the first extraction engine.

[0028] The iris registration unit 140 stores, in the registration storage unit 110, iris information relating to the target iris image selected based on the first affinity score.

[0029] According to this information processing device 100, the first affinity score is a score indicating the affinity between a target iris image and the first extraction engine. Therefore, a target iris image that can be accurately authenticated using the first extraction engine can be selected, and authentication can be performed using iris information related to the target iris image. Therefore, it is possible to improve the accuracy of authentication using iris information.

[0030] The information processing device 100 executes information processing as shown in FIG.

[0031] The image acquisition unit 120 acquires a target face image and a target iris image (step S120).

[0032] The calculation unit 130 calculates a first affinity score indicating the affinity between the target iris image and the first extraction engine based on the iris extraction information extracted from the target iris image using the first extraction engine (step S130).

[0033] The iris registration unit 140 stores the iris information relating to the target iris image selected based on the first affinity score in the registration storage unit 110 (step S140).

[0034] According to this information processing, the first affinity score indicates the affinity between the target iris image and the first extraction engine. Therefore, a target iris image that can be accurately authenticated using the first extraction engine can be selected, and authentication can be performed using iris information related to the target iris image. Therefore, it is possible to improve the accuracy of authentication using iris information.

[0035] A detailed example of the information processing system S1 will be described below. Note that the information processing system may be composed of one or more devices, and the configuration of the information processing system described below is an example. In other words, the number of devices constituting the information processing system, the functions provided by each device, the processes executed, etc. are not limited to the example described below and may be changed as appropriate.

[0036] 4, the information processing system S1 includes the above-described image capturing unit 50 and information processing device 100. The image capturing device 40 and the information processing device 100 may be connected to each other via a communication network NT configured by wired or wireless means or a combination thereof so as to be able to transmit and receive information to and from each other.

[0037] The photographing device 40 includes a photographing unit 50. The information processing device 100 includes a registration storage unit 110, an image acquisition unit 120, a calculation unit 130, and an iris registration unit 140, as shown in FIG.

[0038] The information processing system S1 executes information processing such as that shown in FIG.

[0039] The photographing unit 50 photographs a target to obtain a target face image and a target iris image, which respectively include the face and iris of the target (step S50).

[0040] The above-described steps S120, S130 and S140 are executed.

[0041] (Regarding the Imaging Unit 50) The imaging unit 50 includes, for example, a face camera 51, an iris camera 52, and a control unit 53, as shown in FIG.

[0042] The face camera 51 photographs a target and acquires a target face image including the face of the target.

[0043] The iris camera 52 photographs a target and acquires a target iris image including the iris of the target.

[0044] The control unit 53 transmits the target face image and target iris image captured by the face camera 51 and the iris camera 52, respectively.

[0045] The photographing unit 50 executes a photographing process (step S50) as shown in FIG. 8, for example.

[0046] The face camera 51 photographs a target and acquires a target face image including the face of the target (step S51).

[0047] The iris camera 52 photographs the target and acquires a target iris image including the iris of the target (step S52).

[0048] The control unit 53 transmits the target face image and the target iris image acquired in steps S51 and S52, respectively (step S53).

[0049] (Regarding the face camera 51 and the iris camera 52) The face camera 51 and the iris camera 52 are cameras that capture an image of a target for authentication. The target is a target for authentication, and is typically a person. The target may also be an animal such as a dog or a snake.

[0050] The face camera 51 and the iris camera 52 have different imaging areas, for example.

[0051] The photographing area of ​​the face camera 51 is, for example, the face of the target and its surroundings. That is, the face camera 51 photographs the target and acquires a target face image. The target face image is a face image. The face image is an image that includes the target's face.

[0052] The imaging area of ​​the iris camera 52 is, for example, the eye of the subject and its surroundings. That is, the iris camera 52 captures an image of the subject and acquires an iris image of the subject. The iris image of the subject is an iris image.

[0053] The iris image is an image including an iris. For example, the iris image may be an iris region image showing only the iris region, a monocular image including one predetermined eye of the left or right eye, or a binocular image including both eyes.

[0054] The photographing unit 50 only needs to be able to capture a target face image including the face of the target and a target iris image including the iris of the target by photographing, and it is sufficient to be equipped with at least one camera (for example, a face camera 51) for this purpose.

[0055] (Regarding the control unit 53) The control unit 53 transmits, for example, the target face image and the target iris image acquired by the face camera 51 and the iris camera 52, respectively, simultaneously or sequentially to the information processing device 100. When the target face image and the target iris image are transmitted sequentially, the control unit 53 may transmit either the target face image or the target iris image first.

[0056] The control unit 53 may control the timing at which the face camera 51 and the iris camera 52 take pictures.

[0057] For example, the control unit 53 may cause the facial camera 51 to take a photograph when a predetermined start condition is met, such as receiving a signal from an entry detection sensor (not shown) that detects that the subject has entered a specified shooting area, or a start instruction based on user input, etc.

[0058] Furthermore, for example, the control unit 53 may cause the face camera 51 to constantly capture images and may perform face detection processing to detect a face from the captured image. The control unit 53 may cause the iris camera to capture an image when a face is detected, when the face enters a predetermined area in the target face image, when the interocular distance is equal to or greater than a predetermined value, etc. The interocular distance is the distance between the right eye and the left eye in the target face image.

[0059] Then, the control unit 53 may identify an area including the iris of the target (for example, an iris area, a single eye area, or both eye areas) from the target face image acquired by the face camera 51, and cause the iris camera 52 to capture an image of the identified area. At this time, the control unit 53 may control the orientation of the iris camera 52 so as to capture an image of the identified area (i.e., the area including the iris of the target), or may control the exposure of the iris camera 52. In order to control the orientation of the iris camera 52, the image capturing unit 50 may include, for example, a mechanism (not shown) configured with a motor or the like for changing the orientation in which the iris camera 52 captures images.

[0060] A common technique may be applied to the method of identifying a region including the iris of the target from the target face image. For example, the control unit 53 may input the target face image into a region identification model to identify a region including the iris of the target. The region identification model may be, for example, a machine learning model that has been trained to identify a region including the iris of the target from the target face image. The region identification model may output, for example, information for identifying a region including the iris of the target in the input target face image (iris region identification information). The iris region identification information may be, for example, information indicating the position of the iris in the target face image. In more detail, for example, the iris region identification information may be information including the position of at least one circle, including a circle representing the inner boundary or edge of the iris and a circle representing the outer boundary or edge of the iris, and may further include the size of the at least one circle.

[0061] The control unit 53 may, for example, receive a signal from an exit detection sensor (not shown) that detects that the subject has exited a predetermined photographing area, an end instruction based on a user's input, or the like. In this case, the photographing unit 50 may repeatedly execute the photographing process (step S50) until a predetermined photographing end condition, such as receiving a signal from the exit detection sensor or an end instruction, is met. By repeatedly executing the photographing process (step S50), multiple sets of target face images and target iris images may be acquired for the subject. Examples using multiple sets of target face images and target iris images will be described in other embodiments.

[0062] The control method and content of the control unit 53 are not limited to those exemplified here.

[0063] (Regarding the Registration Storage Unit 110) The registration storage unit 110 may store in advance registered face information relating to faces and personal identification information in association with each other.

[0064] (Image Acquisition Unit 120) The image acquisition unit 120 acquires a target face image and a target iris image acquired by, for example, photographing a target with the photographing unit 50. The image acquisition unit 120 may acquire the target face image and target iris image from the photographing unit 50 or via another device. The image acquisition unit 120 may acquire these images from a storage device in which the target face image and target iris image are stored in advance.

[0065] (Calculation Unit 130) The calculation unit 130 calculates a first affinity score based on iris extraction information extracted from a target iris image using the first extraction engine.

[0066] (First Extraction Engine) The first extraction engine is an engine for extracting iris extraction information from a target iris image. When a target iris image is input, the first extraction engine outputs the iris extraction information.

[0067] The iris extraction information may include, for example, at least one of an iris feature vector and iris keypoints. The iris extraction information does not necessarily include the target iris image itself used to extract the iris feature vector and iris keypoints. In this respect, the iris extraction information differs from iris information.

[0068] The iris feature vector is a feature vector extracted from a target iris image, for example, a vector indicating the features of the iris pattern.

[0069] Here, a vector may be composed of one or more values, and the same applies hereinafter. Each value constituting a vector is typically a numerical value. When a vector is composed of one value, the vector represents a scalar quantity. Furthermore, although the value is typically a numerical value, it may be, for example, one or more combinations of numerical values, letters, symbols, etc., as long as they are defined so that their magnitudes or highs and lows can be compared according to a predetermined standard, and the same applies hereinafter.

[0070] The iris keypoints are information about characteristic points (predetermined locations) of the iris included in the iris image, for example, vectors that indicate the geometric characteristics of the points (predetermined locations) of the iris.

[0071] In detail, for example, the iris keypoint includes parameters indicating the shape, position, size, etc. of a predetermined location included in the iris. The predetermined location is, for example, at least one of the iris circle, pupil circle, eyelid, etc. The iris keypoint includes at least one parameter indicating, for example, the shape of a circle, arc, curve, etc. representing the iris circle, pupil circle, eyelid, etc., a point at a predetermined representative position such as the center, size, etc.

[0072] The first extraction engine may be, for example, a machine learning model configured using a neural network. The first extraction engine may be, for example, a machine learning model trained using training data, or may be a part of the machine learning model that extracts iris extraction information. The training data may include, for example, target iris images for training and ground truth data that includes the correct class to which the target iris images belong.

[0073] (First Affinity Score) The first affinity score is the affinity score between the target iris image and the first extraction engine, in other words, the affinity score related to the target iris image.

[0074] (Affinity Score) The affinity score is a score indicating the affinity between an image and an engine for extracting extracted information from the image. The score may be a vector consisting of one or more values.

[0075] Each value constituting the affinity score may be, for example, a score corresponding to the accuracy of the authentication estimated when authentication is performed using the target image. In detail, for example, the larger each value constituting the affinity score, the higher the accuracy of the authentication (i.e., the higher the affinity) is estimated to be. Note that the values ​​constituting the affinity score are not limited to this, and for example, the smaller the value, the higher the accuracy of the authentication is estimated to be.

[0076] The index representing the accuracy of authentication may be, for example, at least one index related to the error rate in authentication. Examples of the index related to the error rate include FAR (False Acceptance Rate), FRR (False Reject Rate), ERR (Equal Error Rate), AUC (Area Under the Curve), etc. Note that the index related to the error rate is not limited to these.

[0077] Generally, the error rate index values ​​are obtained by verifying the authentication results using the image. The lower the error rate indicated by these index values, the higher the accuracy of the authentication using the image.

[0078] In this way, the affinity score may be a score corresponding to the accuracy of authentication (i.e., for example, the error rate) estimated when authentication is performed using the target image. Alternatively, the affinity score may be a score correlated with the accuracy of authentication. The higher this correlation, the more desirable.

[0079] Below, examples 1 to 4 of affinity scores are described. Any one of the examples described below may be used for the affinity score, or an appropriate combination of multiple examples described below may be used. The affinity scores are not limited to examples 1 to 4. In other words, the affinity score of an image may include at least one of (example 1) to (example 4), etc.

[0080] (Example 1) An index value that indicates the accuracy of authentication. (Example 2) A value based on the degree of deviation between the position of the image in the feature space that represents the feature vector and a predetermined high-accuracy position. (Example 3) A value based on the norm of the feature vector. (Example 4) A value based on the degree of deviation between the position of the image in the feature space that represents the feature vector and a predetermined low-accuracy position.

[0081] (Example 1 of Affinity Score) The affinity score of an image may be an index value (i.e., for example, an error rate) that represents the accuracy of authentication when authentication is performed using the image and a reference image. The reference image is an image that is predetermined as a standard.

[0082] (Example 2 of Affinity Score) The affinity score of an image may be a value based on the degree of deviation between the position of the image in the feature space and a predetermined high-accuracy position.

[0083] The feature space is a space representing a feature vector, and the same applies hereinafter. The position of an image is the position in the feature space of a feature vector extracted from the image, and the same applies hereinafter.

[0084] The high-precision position is the position in the feature space of a high-precision image that is predetermined as an image that can be authenticated with high accuracy, as shown in Fig. 9. In other words, the high-precision position is the position in the feature space of a feature vector extracted from the high-precision image.

[0085] The degree of deviation is a value indicating the degree of separation in the feature space, and the same applies hereinafter. The degree of deviation may be, for example, at least one of Euclidean distance, Mahalanobis distance, cosine similarity, etc. in the feature space.

[0086] In this case, the affinity score of the image may indicate, for example, that the smaller the deviation between the position of the image in the feature space and a predetermined high-precision position, the higher the accuracy of the authentication (i.e., the higher the affinity). In detail, for example, the affinity score in this case is, for example, the inverse of this deviation, but is not limited to this.

[0087] The predetermined high-precision position may be one or more. In the case where there are multiple predetermined high-precision positions, the affinity score of the image may be a value based on the degree of deviation between the position of the image in the feature space and at least one of the multiple high-precision positions.

[0088] (Example 3 of Affinity Score) The affinity score of an image may be a value based on the norm of the feature vector (for example, L1 norm, L2 norm).

[0089] Generally, for images that can be authenticated with high authentication accuracy (i.e., with a low error rate, for example), the norm of the feature vector extracted from the image is often large. Therefore, a value indicating that the larger the norm of the feature vector, the higher the authentication accuracy (i.e., the higher the affinity) may be used as an affinity score according to the authentication accuracy (i.e., with a low error rate, for example). The value used as such an affinity score (i.e., a value based on the norm) may be the norm itself, or may be a value obtained by performing an appropriate calculation on the norm.

[0090] (Example 4 of Affinity Score) The affinity score of an image may be a value based on the degree of deviation between the position of the image in the feature space and a predetermined low-precision position.

[0091] The low-accuracy position is the position in the feature space of a low-accuracy image that is predetermined as an image that will be authenticated with low accuracy, as shown in Fig. 9. In other words, the low-accuracy position is the position in the feature space of a feature vector extracted from the low-accuracy image.

[0092] In this case, the affinity score of the image may indicate, for example, that the greater the deviation between the position of the image in the feature space and a predetermined low-precision position, the higher the accuracy of the authentication (i.e., the higher the affinity). In detail, for example, the affinity score in this case is, for example, this deviation, but is not limited to this.

[0093] The predetermined low-precision location may be one or more. In the case where there are multiple predetermined low-precision locations, the affinity score of the image may be a value based on the degree of deviation between the location of the image in the feature space and at least one of the multiple low-precision locations.

[0094] When these affinity score examples 1 to 4 are applied to the first affinity score (ie, the affinity score for the target iris image), the result is as follows:

[0095] (Example 1 of first affinity score) An index value that represents the accuracy of authentication using a target iris image. (Example 2 of first affinity score) A value based on the degree of deviation between the position of the target iris image in the feature space and a predetermined high-quality position. (Example 3 of first affinity score) A value based on the norm of the iris feature vector. (Example 4 of first affinity score) A value based on the degree of deviation between the position of the target iris image in the feature space and a predetermined low-quality position.

[0096] The second affinity score may include at least one of these examples 1-4, etc.

[0097] (Regarding the iris registration unit 140) As described above, the iris registration unit 140 stores iris information related to the target iris image selected based on the first affinity score in the registration storage unit 110. The iris registration unit 140 may further associate the iris information with registered face information and personal identification information and store the iris information in the registration storage unit 110. When storing the iris information in the registration storage unit 110, the iris registration unit 140 may further associate the iris information related to the target iris image with registered face information and personal identification information of the target corresponding to the target iris image and store the iris information in the registration storage unit 110.

[0098] For example, as shown in FIG. 10, the iris registration unit 140 includes a selection unit 141, a specification unit 142, and a registration unit 143.

[0099] The selection unit 141 selects the target iris image based on the first affinity score.

[0100] The identification unit 142 identifies at least one of the registered face information and the personal identification information of the target from the registered face information and the personal identification information stored in association with each other in the registration storage unit 110 .

[0101] The registration unit 143 stores the iris information relating to the target iris image that satisfies the first condition in the registration storage unit 110 in further association with the registered face information and personal identification information of the target.

[0102] The iris registration unit 140 executes a registration process (step S140) as shown in Fig. 11. The registration process (step S140) is a process for storing iris information in the registration storage unit 110.

[0103] The selection unit 141 selects target iris images based on the first affinity scores (step S141).

[0104] The identifying unit 142 identifies at least one of the registered face information and the personal identification information of the target from the registered face information and the personal identification information stored in association with each other in the registration storage unit 110 (step S142).

[0105] The registration unit 143 stores the iris information relating to the target iris image that satisfies the first condition in the registration storage unit 110 in association with the registered face information and personal identification information of the target (step S143).

[0106] (Regarding the Selector 141) The selector 141 selects target iris images that satisfy a predetermined first condition regarding the first affinity score, for example.

[0107] For example, the selection unit 141 determines whether the first affinity score calculated for the target iris image satisfies a first condition, and then selects the target iris image for which the first affinity score that satisfies the first condition has been calculated as the target iris image that satisfies the first condition.

[0108] (Regarding the First Condition) The first condition is a predetermined condition regarding the first affinity score. The first condition may be defined using, for example, a threshold value regarding the first affinity score.

[0109] The first affinity score is configured to have a value that increases as the accuracy of authentication increases (i.e., the affinity increases).

[0110] In this case, the first condition may be, for example, that each value constituting the first affinity score is equal to or greater than a threshold value.

[0111] In detail, for example, when the first affinity score is composed of multiple values, the first condition may be, for example, that a predetermined number or more of the multiple values ​​constituting the first affinity score are equal to or greater than a threshold. The predetermined number may be set as appropriate, for example, between 1 and the number of values ​​constituting the first affinity score. Furthermore, for example, the first condition may be that a value obtained by statistically processing the multiple values ​​constituting the first affinity score is equal to or greater than a threshold. The value obtained by statistically processing the multiple values ​​is, for example, the average, maximum, median, etc. of the multiple values. Furthermore, the first condition may be that a predetermined number or more of the first affinity score, or a value obtained by statistically processing the multiple values ​​constituting the first affinity score, is equal to or greater than a threshold, and the first affinity score is the maximum.

[0112] For example, when the target iris image has one first affinity score, the first affinity score being the largest may mean that the first affinity score is the largest. Also, when the target iris image has multiple first affinity scores, the first affinity score being the largest may mean that the value obtained by statistically processing the multiple values ​​that make up the first affinity score is the largest.

[0113] For example, the first condition may be defined without using a threshold, and in particular, the first condition may be that the first affinity score is the maximum.

[0114] The first condition is not limited to the example given here, and may be, for example, a threshold value.

[0115] (Regarding the Identification Unit 142) The identification unit 142 identifies at least one of the registered face information and the personal identification information of the target from the registered face information and the personal identification information stored in association with each other in the registration storage unit 110.

[0116] There are various methods for identifying the registered face information and / or personal identification information of the subject, examples of which are described below.

[0117] (Method of Identifying Personal Identification Information of a Target) The identification unit 142 may acquire personal identification information based on, for example, an input by the target. Then, the identification unit 142 may identify the personal identification information of the target from the personal identification information stored in the registration storage unit 110 based on the acquired personal identification information. In detail, for example, the identification unit 142 may identify the personal identification information of the target from the personal identification information stored in the registration storage unit 110, by selecting personal identification information that matches the acquired personal identification information.

[0118] (Method for Identifying Registered Face Information of a Target) The identification unit 142 may, for example, use the target face image acquired by the image acquisition unit 120 to identify the registered face information of the target from the registered face information stored in the registration storage unit 110. A machine learning model may be used to identify the registered face information of the target using the target face image.

[0119] In detail, for example, the identification unit 142 compares the face extraction information extracted from the target face image using the second extraction engine with the registered face information, and identifies the target registered face information based on the comparison result.

[0120] The second extraction engine is an engine for extracting face extraction information from a target face image, and when the target face image is input, the second extraction engine outputs the face extraction information.

[0121] The face extraction information includes, for example, at least one of a facial feature vector and a facial keypoint, but does not necessarily include the target face image used to extract the facial feature vector and the facial keypoint.

[0122] The facial feature vector is a feature vector extracted from a target face image. For example, the facial feature vector is a vector indicating the features of a predetermined part included in a face. The predetermined part may be, for example, a predetermined part such as the eyes, nose, or mouth. The features of the predetermined part include, for example, the position, shape, etc. of the predetermined part, but are not limited to these.

[0123] The face keypoints are information about characteristic points (predetermined locations) of a face included in a target face image. For example, the face keypoints are vectors that indicate the geometric characteristics of the points (predetermined locations) of the face.

[0124] In detail, for example, face key points include parameters indicating the shape, position, size, etc. of predetermined parts included in the face. The predetermined parts are, for example, at least one of the facial contour, eyebrows, eyes, nose, mouth, etc. Iris key points include at least one parameter indicating, for example, the shape of an arc, curve, etc. representing the facial contour, eyebrows, eyes, nose, mouth, etc., a point at a predetermined representative position such as the center, at least one edge above, below, left, or right, the size between points, the size of the shape, etc.

[0125] The result of comparing the face extraction information extracted from the target face image with the registered face information is, for example, a result of comparing the cosine similarity with a predetermined threshold. For example, the identification unit 142 identifies the registered face information whose cosine similarity is equal to or greater than the threshold as the target registered face information.

[0126] The results of the comparison are not limited to those exemplified here. The identification unit 142 may extract face extraction information using the second extraction engine, compare the face extraction information extracted from the target face image with registered face information, or obtain the results of such processing by another functional unit. General image processing techniques other than machine learning models may also be applied to identify the target registered face image using the target face image.

[0127] (Method for identifying personal identification information and registered face information of a target) For example, a combination of the above-described methods for identifying the personal identification information and registered face information of a target may be applied to the identification unit 142. In this way, the identification unit 142 may identify the registered face information and personal identification information of a target from the registered face information and personal identification information stored in association with each other in the registration storage unit 110.

[0128] (Regarding the registration unit 143) As described above, the registration unit 143 further associates the iris information related to the target iris image that satisfies the first condition with the registered face information and personal identification information of the target, and stores the associated information in the registration storage unit 110. Here, the registered face information and personal identification information of the target may include at least one of the registered face information and personal identification information identified by the identification unit 142.

[0129] (Regarding Iris Information) The iris information may include at least one of a target iris image, an iris feature vector, and iris key points. The iris information may include at least one of a target iris image and an iris feature vector. The iris information may also include an iris feature amount indicating iris features together with or instead of the iris feature vector. The iris feature amount may be, for example, a feature map indicating iris features in a two-dimensional matrix, but is not limited to this.

[0130] (Example of physical configuration of information processing device 100) The information processing device 100 physically includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070, as shown in FIG.

[0131] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0132] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0133] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0134] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the information processing device 100. The processor 1020 reads each of these program modules into the memory 1030 and executes them to realize the function corresponding to the program module.

[0135] The network interface 1050 is an interface for connecting the information processing device 100 to a network. The network is a communication network for transmitting and receiving information to and from other devices (not shown), and may be wired, wireless, or a combination of these.

[0136] The input interface 1060 is an interface for the user to input information, and is composed of, for example, a touch panel, a keyboard, a mouse, and the like.

[0137] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0138] The photographing device 40 may have a physical configuration similar to that of the information processing device 100, and may further include a camera.

[0139] The number of devices constituting the information processing system is not limited to two, and may be one or more. When the information processing system is composed of multiple devices, the multiple devices may be connected to each other so as to be able to send and receive information, for example, via a communication network NT. Furthermore, some or all of the functions of the information processing device 100 may be provided in the image capturing device 40 instead of the information processing device 100. The image capturing device 40 may execute information processing executed by the information processing device 100 instead of the information processing device 100. In detail, for example, the first extraction engine or calculation unit 130 that extracts an iris feature vector from a target iris image may be provided in the information processing device 100 or the image capturing device 40.

[0140] (Operations and Effects) As described above, according to this embodiment, the information processing system S1 includes the registration storage unit 110 , the image capturing unit 50 , the calculation unit 130 , and the iris registration unit 140 .

[0141] The registration storage unit 110 stores registered face information relating to faces and personal identification information in association with each other. The photographing unit 50 photographs a target and acquires a target face image and a target iris image, each including the face and iris of the target, in association with each other.

[0142] The calculation unit 130 calculates a first affinity score indicating the affinity between the target iris image and the first extraction engine based on iris extraction information extracted from the target iris image using the first extraction engine.

[0143] The iris registration unit 140 stores iris information related to a target iris image corresponding to a first affinity score that satisfies a predetermined first condition regarding the first affinity score in the registration storage unit 110. The iris registration unit 140 further associates the iris information with registered face information corresponding to the target face image and personal identification information associated with the registered face information, and stores the iris information in the registration storage unit 110.

[0144] According to this, the first affinity score is a score indicating the affinity between the target iris image and the first extraction engine. Therefore, it is possible to select a target iris image that can be authenticated with high accuracy using the first extraction engine, and register iris information related to the target iris image for authentication. Therefore, it is possible to improve the accuracy of authentication using iris information.

[0145] [Embodiment 2] In this embodiment, an example will be described in which an information processing system further includes a function of inputting a target iris image into a first extraction engine to extract iris extraction information. Also, an example will be described in which the information processing system further includes a function of authenticating a target using the iris extraction information. Note that, in this embodiment, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0146] As shown in FIG. 13, the information processing system S2 includes a registration storage unit 110, an imaging unit 50, a calculation unit 130, an iris registration unit 140, an extraction unit 260, and an authentication unit 270, similar to those in the first embodiment.

[0147] The extraction unit 260 inputs the target iris image to the first extraction engine to extract iris extraction information.

[0148] The authentication unit 270 uses the extracted iris information to authenticate the subject.

[0149] The information processing system S2 is configured, for example, with an image capturing device 40 similar to that of the first embodiment, and an information processing device 200 replacing the information processing device 100 (see FIG. 4). As shown in FIG. 14, for example, the information processing device 200 includes the registration storage unit 110, image acquisition unit 120, calculation unit 130, and iris registration unit 140 similar to those of the first embodiment, as well as the extraction unit 260 and authentication unit 270 described above.

[0150] The information processing system S2 executes information processing such as that shown in FIG.

[0151] Steps S50 and S120 are executed in the same manner as in the first embodiment.

[0152] The extraction unit 260 inputs the target iris image acquired in S120 into the first extraction engine to extract iris extraction information (step S260).

[0153] Steps S130 and S140 are executed in the same manner as in the first embodiment.

[0154] Furthermore, the information processing system S2 executes information processing, for example, as shown in Fig. 16. This information processing may be executed when authentication is performed in response to an instruction from a user (e.g., a target). The information processing shown in Fig. 16 includes an authentication processing (step S270) instead of a registration processing (step S140). Except for this point, the information processing shown in Fig. 16 is the same as the information processing shown in Fig. 15.

[0155] That is, steps S50 and S120 similar to those in the first embodiment, step S260 described above, and step S130 similar to those in the first embodiment are executed. The authentication unit 270 performs authentication of the target using the extracted iris information (step S270).

[0156] (Extraction Unit 260) The extraction unit 260 may include the first extraction engine described in embodiment 1. For example, the extraction unit 260 inputs a target iris image acquired by the image acquisition unit 120 to the first extraction engine. The first extraction engine outputs iris extraction information according to the input. As a result, the extraction unit 260 extracts iris extraction information from the target iris image using the first extraction engine.

[0157] (Regarding the authentication unit 270) The authentication unit 270 performs authentication of the target using, for example, registered iris information and the iris extraction information extracted in step S260. The registered iris information is iris information pre-stored in the registration storage unit 110. The registered iris information may be, for example, the iris information stored in the registration storage unit 110 in step S140.

[0158] In detail, for example, the authentication unit 270 acquires registered iris information from the registration storage unit 110. The authentication unit 270 performs authentication of the target using the acquired registered iris information and the iris extraction information extracted in step S260.

[0159] For example, the authentication unit 270 may perform authentication based on the result of comparing the similarity (for example, cosine similarity) of the iris feature vectors included in the registered iris information and the extracted iris information with a predetermined threshold value.

[0160] The first affinity score may be expressed as a value corresponding to the accuracy of target authentication using iris extraction information, as described in this embodiment, for example. That is, the first affinity score may be a value corresponding to the accuracy of authentication estimated in authentication using iris extraction information extracted by the first extraction engine.

[0161] In addition, in the authentication, it may be possible to use a general technique to confirm that the objects indicated by the registered iris information and the extracted iris information are the same, and the method is not limited to the example given here.

[0162] (Actions and Effects) As described above, according to this embodiment, the information processing system S2 further includes the extraction unit 260 that inputs a target iris image to the first extraction engine and extracts iris extraction information, and the authentication unit 270 that authenticates the target using the iris extraction information. The first affinity score is expressed as a value corresponding to the accuracy of target authentication using the iris extraction information, for example, as described in this embodiment.

[0163] According to this, the affinity (first affinity score) between the target iris image and the first extraction engine is expressed using a value corresponding to the accuracy of target authentication using iris extraction information. Therefore, a target iris image that can be accurately authenticated using the first extraction engine can be selected, and the iris information related to the target iris image can be used to register and authenticate the target. Therefore, it is possible to improve the accuracy of authentication using iris information.

[0164] [Embodiment 3] The calculation unit 130 may further calculate a second affinity score, which is an affinity score related to the target face image. In addition, the information processing system may further include a configuration for extracting face extraction information, which is extraction information for calculating the second affinity score, from the target face image.

[0165] The process for using the second affinity score varies.

[0166] For example, the second affinity score may be used for authentication.

[0167] Furthermore, for example, the second affinity score may be used to improve the shooting environment when the imaging unit 50 captures an image of the target's iris. This is effective, for example, when, after the imaging unit 50 captures an image of the target's face as described above, imaging is performed to obtain a target iris image to be associated with the target face image acquired by this imaging. This is because, by using the second affinity score based on the target face image captured of the target's face, it is possible to change the shooting environment as necessary to capture the target's iris.

[0168] In the third embodiment, an example will be described in which the information processing system S3 extracts face extraction information from a target face image and calculates a second affinity score. In addition, in the third embodiment, an example will be described in which the second affinity score is used for both authentication and improving the photographing environment for photographing the target's iris. Note that in this embodiment, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0169] (Regarding the information processing system S3) As shown in FIG. 17 , the information processing system S3 includes a registration storage unit 110 and an iris registration unit 140 similar to those in the first embodiment, an imaging unit 350, an extraction unit 360, a calculation unit 330, an authentication unit 370, and an imaging environment change unit 380.

[0170] The photographing section 350 photographs a target to obtain a target face image including the face of the target, and then photographs the iris of the target to obtain a target iris image.

[0171] The extraction unit 360 includes a first extraction unit 360a and a second extraction unit 360b.

[0172] The first extraction unit 360a inputs a target iris image to a first extraction engine to extract iris extraction information, and the second extraction unit 360b inputs a target face image to a second extraction engine to extract face extraction information.

[0173] The calculation unit 330 includes a first calculation unit 330a and a second calculation unit 330b.

[0174] The first calculation unit 330a calculates a first affinity score. The second calculation unit 330b calculates a second affinity score indicating the affinity between the target face image and the second extraction engine based on the face extraction information.

[0175] The authentication unit 370 authenticates the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score.

[0176] The imaging environment change unit 380 performs processing to change the imaging environment for photographing the target. The imaging environment change unit 380 generates a change instruction for changing the imaging environment for photographing the target's iris based on the second affinity score.

[0177] The information processing system S3 may include, for example, an imaging device 340 instead of the imaging device 40 and an information processing device 300 instead of the information processing device 100 (see FIG. 4). Furthermore, the imaging device 340 may include, for example, the above-described imaging unit 350 instead of the imaging unit 50.

[0178] (Regarding the information processing device 300) As shown in FIG. 18 , the information processing device 300 includes a registration storage unit 110 and an iris registration unit 140 similar to those in the first embodiment, an image acquisition unit 320, and the above-described extraction unit 360, calculation unit 330, authentication unit 370, and shooting environment change unit 380.

[0179] The image acquisition unit 320 acquires a target face image acquired by photographing a target with the photographing unit 350, and then acquires a target iris image acquired by photographing the target with the photographing unit 350.

[0180] (Example of Operation of Information Processing System S3) The information processing system S3 executes information processing as shown in FIG. 19, for example.

[0181] The photographing unit 350 photographs and acquires a target face image including the face of the target (step S350b).

[0182] The image acquisition unit 320 acquires the target face image acquired in step S350b (step S320b).

[0183] The second extraction unit 360b inputs the target face image acquired in step S320b into a second extraction engine to extract face extraction information (step S360b).

[0184] The second calculation unit 330b calculates a second affinity score indicating the affinity between the target face image and the second extraction engine based on the face extraction information extracted in step S360b (step S330b).

[0185] The imaging environment change unit 380 generates a change instruction for changing the imaging environment for photographing the iris of the target based on the second affinity score calculated in step S330b (step S380).

[0186] The image capturing unit 350 acquires the change instruction generated in step S380 (step S350b).

[0187] The photographing unit 350 changes the photographing environment for photographing the iris of the subject based on the change instruction acquired in step S350b (step S350c).

[0188] The photographing unit 350 photographs the target and acquires a target iris image including the iris of the target (step S350d).

[0189] The image acquisition unit 320 acquires the target iris image acquired in step S350d (step S320a).

[0190] The first extraction unit 360a inputs the target iris image acquired in step S320a into the first extraction engine to extract iris extraction information (step S360a).

[0191] The first calculation unit 330a calculates a first affinity score indicating the affinity between the target iris image and the first engine based on the iris extraction information extracted in step S360a (step S330a).

[0192] Step S140 is executed in the same manner as in the first embodiment.

[0193] The information processing system S3 also executes information processing such as that shown in Fig. 20. This information processing may be executed when authentication is performed in response to a user instruction, etc. The information processing shown in Fig. 20 is similar to the information processing shown in Fig. 19, except that it includes authentication processing (step S370) instead of registration processing (step S140).

[0194] That is, steps S350a, S320b, S360b, S330b, S380, S350b to S350d, S320a, S360a, and S330a similar to those described above are executed.

[0195] The authentication unit 370 authenticates the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score (step S370).

[0196] (Regarding the Image Capturing Unit 350) As shown in FIG. 21, the image capturing unit 350 includes a face camera 51 and an iris camera 52 similar to those in the first embodiment, and a control unit 353.

[0197] In this case, for example, the face camera 51 may execute step S350a. For example, the control unit 353 may execute steps S350b and S350c. For example, the iris camera 52 may execute step S350d.

[0198] (Regarding the control unit 353) The control unit 353 may, for example, transmit a target face image acquired by the face camera 51. For example, after transmitting a target face image of the target, the control unit 353 may change the imaging environment for capturing an image of the target's iris based on a change instruction. The control unit 353 may, for example, transmit a target iris image acquired by the iris camera 52.

[0199] In detail, for example, when the above-described start conditions are satisfied, the control unit 353 causes the face camera 51 to photograph the target. The control unit 353 transmits the target face image thus acquired by the face camera 51 to, for example, the information processing device 100.

[0200] Thereafter, the control unit 353 controls various devices (not shown) based on the change instruction, for example, to change the imaging environment for capturing an image of the iris of the subject. Subsequently, the control unit 353 causes the iris camera 52 to capture an image of the subject.

[0201] The change instruction is an instruction regarding a change in the imaging environment in which the iris of the subject is imaged. For example, the change instruction may include an instruction as to whether or not to change the imaging environment. Furthermore, for example, when the imaging environment is changed, the change instruction may include the details of the change in the imaging environment.

[0202] The imaging environment is, for example, the brightness for capturing an image of the iris of the subject, the position of the subject, etc. In other words, the change instruction when changing the imaging environment may include, as the change content of the imaging environment, how to change the brightness for capturing an image of the iris of the subject, the position of the subject, the direction of the subject's face, etc.

[0203] The brightness may be changed, for example, by controlling a photography light (not shown) attached to the iris camera 52. For example, when photography is performed indoors, the brightness may be changed by controlling the room lighting provided in the room or a light blocking device (curtains, blinds, etc.) attached to a window. Control of lighting such as photography lighting and room lighting may include, for example, at least one of switching the lighting on and off, changing the brightness of the lighting, changing the number of lights turned on, changing the direction in which the lighting illuminates the subject, etc. Note that the method of changing the brightness is not limited to the example given here.

[0204] Changes to the target, such as the target's position or the direction of the target's face, may be made, for example, by having the image capture device 340 further include a display and a speaker (not shown) and having the control unit 353 control these to notify the target of a message. The message may be, for example, a message that guides the target to a desirable image capture environment. In detail, for example, the message may be a message that guides the target to a desirable position or a message that guides the target's face to a desirable direction.

[0205] The control unit 353 may perform control to change one or more of the photographic lighting, the room lighting, the shading device, and the position of the target based on the change instruction.

[0206] For example, even if the face in the target face image is too bright (so-called face blown out), there are various reasons for this, such as when the background other than the face is dark, when the subject's clothing is dark, or when the subject's position or facial orientation is undesirable. By combining the above-described multiple controls based on the change instruction, the control unit 353 can change the shooting environment in detail, for example, by moving the subject to a preferable position and increasing the brightness of the room lighting and shooting lighting. This allows the iris camera 52 to take a picture in a good shooting environment, improving the likelihood of capturing a good target iris image.

[0207] Note that the photographing environment and the method for changing the photographing environment are not limited to those described here. Furthermore, similar to the control unit 53 in the first embodiment, the control unit 353 may specify an area including the iris of the target from the target face image acquired by the face camera 51, and cause the iris camera 52 to photograph the specified area.

[0208] The extraction unit 360 includes the first extraction unit 360 a and the second extraction unit 360 b described above. As a result, the extraction unit 360 inputs a target iris image to the first extraction engine to extract iris extraction information, and inputs a target face image to the second extraction engine to extract face extraction information.

[0209] The first extraction unit 360a may be configured similarly to the extraction unit 260 of the second embodiment.

[0210] The second extraction unit 360b may include a second extraction engine. For example, the second extraction unit 360b inputs the target face image acquired by the image acquisition unit 320 to the second extraction engine. The second extraction engine outputs face extraction information according to the input. As a result, the second extraction unit 360b extracts face extraction information from the target face image using the second extraction engine.

[0211] As described above, the second extraction engine is an engine for extracting face extraction information from a target face image. When the target face image is input, the second extraction engine outputs face extraction information. As described above, the face extraction information includes, for example, at least one of a face feature vector and face key points, but does not include the target face image itself used to extract them.

[0212] The second extraction engine may be, for example, a machine learning model configured using a neural network. The second extraction engine may be, for example, a machine learning model trained using training data, or may be a part of the machine learning model that extracts face extraction information.

[0213] (Regarding the calculation unit 330) The calculation unit 330 includes the first calculation unit 330a and the second calculation unit 330b. As a result, the calculation unit 330 calculates a first affinity score based on the iris extraction information and calculates a second affinity score based on the face extraction information. The second affinity score indicates the affinity between the target face image and the second extraction engine.

[0214] The first calculation unit 330a may be configured similarly to the calculation unit 130 of the first embodiment.

[0215] The second calculation unit 330b calculates a second affinity score based on face extraction information extracted from the target face image using the second extraction engine.

[0216] (Regarding the second affinity score) The second affinity score is the affinity score between the target face image and the second extraction engine, in other words, the affinity score related to the target face image. The affinity score is as described above and may include at least one of (Example 1) to (Example 4), etc.

[0217] When the above-mentioned affinity score examples 1 to 4 are applied to the second affinity score (i.e., the affinity score for the target face image), the result is as follows.

[0218] (Example 1 of second affinity score) An index value that indicates the accuracy of authentication using a target face image. (Example 2 of second affinity score) A value based on the degree of deviation between the position of the target face image in the feature space and a predetermined high-quality position. (Example 3 of second affinity score) A value based on the norm of a face feature vector. (Example 4 of second affinity score) A value based on the degree of deviation between the position of the target face image in the feature space and a predetermined low-quality position.

[0219] The second affinity score may include at least one of these examples 1-4, etc.

[0220] (Regarding Authentication Unit 370) The authentication unit 370 authenticates a target using, for example, a target iris image selected based on the first affinity score and a target face image selected based on the second affinity score.

[0221] For example, the authentication unit 370 may select the target iris image based on the first affinity score. This selection may be performed using the first condition as described above. Alternatively, for example, the authentication unit 370 may select the target face image based on the second affinity score. This selection may be performed using a condition in which the first affinity score in the first condition as described above is changed to the second affinity score.

[0222] The authentication unit 370 may then perform authentication of the target using, for example, the iris extraction information and face extraction information extracted from the target iris image and target face image that satisfy the respective conditions. Here, for example, the iris extraction information and face extraction information extracted by the first extraction unit 360 a and the second extraction unit 360 b, respectively, may be used.

[0223] The authentication method performed by the authentication unit 370 using such iris extraction information and face extraction information (e.g., iris feature vector and face feature vector) may use a general technique of so-called multimodal authentication, which uses information extracted from multiple pieces of biometric information (e.g., feature vectors).

[0224] (Regarding the imaging environment changing unit 380) As described above, the imaging environment changing unit 380 performs processing for changing the imaging environment for photographing the target, such as generating a change instruction, based on the second affinity score. As described above, the change instruction is an instruction regarding a change of the imaging environment for photographing the iris of the target.

[0225] For example, the imaging environment change unit 380 may generate a change instruction depending on whether the second affinity score satisfies a predetermined second condition. Then, the imaging environment change unit 380 may transmit the generated change instruction to the imaging device 340. As a result, the imaging unit 350 may acquire the change instruction.

[0226] In detail, for example, if the second affinity score satisfies the second condition, the imaging environment change unit 380 may generate a change instruction indicating not to change the imaging environment. If the second affinity score does not satisfy the second condition, the imaging environment change unit 380 may generate a change instruction indicating to change the imaging environment. In this case, the change instruction may include the details of the changes to the imaging environment. The details of the changes may be, for example, information indicating how to change the brightness for photographing the target's iris, the position of the target, the orientation of the target's face, etc., as described above.

[0227] Note that the imaging environment change unit 380 may not generate a change instruction if the second affinity score does not satisfy the second condition. In this case, for example, if a predetermined waiting time has elapsed since the target face image was transmitted without receiving a change instruction, the control unit 353 may cause the iris camera 52 to capture an image of the target without changing the imaging environment.

[0228] (Regarding the Second Condition) The second condition is a predetermined condition regarding the second affinity score. The second condition may be defined using, for example, a threshold value regarding the second affinity score.

[0229] The second affinity score is configured to have a value that increases as the accuracy of authentication increases (i.e., the affinity increases).

[0230] In this case, the second condition is, for example, that each value constituting the second affinity score is equal to or greater than a threshold value. The threshold value included in the second condition may be different from or the same as the threshold value included in the first condition.

[0231] In this case, when the second affinity score is composed of multiple values, the second condition may be, for example, that all or a predetermined number of the multiple values ​​constituting the second affinity score are equal to or greater than a threshold. Furthermore, for example, the second condition may be that a value obtained by statistically processing the multiple values ​​constituting the second affinity score is equal to or greater than a threshold. The value obtained by statistically processing the multiple values ​​is, for example, the average, maximum, median, etc. of the multiple values.

[0232] The second condition is not limited to the example given here, and may be, for example, a threshold value.

[0233] (Method for Generating Changes to the Shooting Environment) The shooting environment change section 380 may generate changes to the shooting environment using, for example, at least one of the target face image and face extraction information.

[0234] The imaging environment change unit 380 may analyze the target face image using, for example, general image processing. Alternatively, for example, the imaging environment change unit 380 may analyze at least one of the target face image and the face extraction information using an estimation model, which is a machine learning model using a neural network or the like. The analysis results may be, for example, a deterioration factor of the target face image, or changes to the imaging environment according to the deterioration factor.

[0235] The degradation factors include focus blur, motion blur, eyeglass reflection, noise level, resolution, contrast, size of the face area included in the target face image, occlusion, target facial expression, etc. Occlusion may be caused by, for example, a mask or sunglasses.

[0236] If the analysis result indicates a deterioration factor, the imaging environment change unit 380 may store in advance change content data indicating a pattern of changes to the imaging environment according to a combination of deterioration factors, for example. Then, the imaging environment change unit 380 may generate changes to the imaging environment based on the analysis result and the change content data.

[0237] The method for generating changes to the shooting environment is not limited to the example described here.

[0238] (Operations and Effects) As described above, according to this embodiment, the information processing system S3 further includes the image capturing environment change unit 380 for changing the image capturing environment for capturing an image of a target.

[0239] The extraction unit 360 includes a first extraction unit 360 a and a second extraction unit 360 b. The first extraction unit 360 a inputs a target iris image to a first extraction engine to extract iris extraction information. The second extraction unit 360 b inputs a target face image to a second extraction engine to extract face extraction information.

[0240] The calculation unit 330 includes a first calculation unit 330 a and a second calculation unit 330 b. The first calculation unit 330 a calculates a first affinity score based on the iris extraction information. The second calculation unit 330 b calculates a second affinity score indicating the affinity between the target face image and the second extraction engine based on the face extraction information.

[0241] The photographing unit 350 photographs the target to obtain a target face image including the face of the target, and then photographs the iris of the target to obtain a target iris image. The photographing environment changing unit 380 generates a change instruction for changing the photographing environment for photographing the iris of the target, based on the second affinity score.

[0242] According to this, the photographing environment for photographing the target's iris is changed based on the second affinity score, which increases the possibility of capturing an iris image with minimal degradation, thereby improving the accuracy of authentication using iris information.

[0243] According to this embodiment, the authentication unit 370 authenticates the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score.

[0244] This allows the first extraction engine and the second extraction engine to select target iris images and iris-face images that can be authenticated with high accuracy, respectively, and then perform authentication using the extracted iris information and extracted face information. Therefore, it is possible to improve the accuracy of multimodal authentication using iris information and face information.

[0245] [Fourth Embodiment] In the first embodiment, an example was given in which the identification unit 142 identifies the target personal identification information based on a user input. In this embodiment, an example will be described in which this input is made to a terminal device that is a device other than the information processing device 100. Note that in this embodiment, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0246] The information processing system S4 includes, for example, an image capturing device 40 similar to that of the first embodiment, an information processing device 400, and a communication terminal 60, as shown in FIG.

[0247] The communication terminal 60 includes a communication unit 70 that transmits personal identification information.

[0248] It should be noted that the information processing system S4 only needs to include the communication unit 70, and the device including the communication unit 70 is not limited to the communication terminal 60. For example, the image capturing device 40 may further include the communication unit 70.

[0249] The information processing device 400 includes, for example, a registration storage unit 110, an image acquisition unit 120, a calculation unit 130, and an iris registration unit 440, which are similar to those in the first embodiment, as shown in FIG.

[0250] The iris registration unit 440 further associates the iris information regarding the selected target iris image with the registered face information and personal identification information of the target identified using the personal identification information transmitted by the communication unit 70 and stores it in the registration memory unit 110.

[0251] The information processing system S4 executes information processing such as that shown in FIG.

[0252] Steps S50, S120, and S130 are executed in the same manner as in the first embodiment.

[0253] The communication unit 70 transmits the personal identification information (step S70).

[0254] The iris registration unit 440 further associates the iris information regarding the selected target iris image with the registered face information and personal identification information of the target identified using the personal identification information transmitted in step S70 and stores it in the registration memory unit 110 (step S440).

[0255] (Regarding the communication terminal 60 and the communication unit 70) The communication terminal 60 is, for example, a mobile terminal (smartphone, tablet terminal, etc.) of a user (e.g., a target), but is not limited to this. The communication terminal 60 may be connected to at least the information processing device 400 via a communication network NT so as to be able to transmit and receive information to and from each other.

[0256] The communication unit 70 receives, for example, input of personal identification information by a user (for example, a subject), and transmits the input personal identification information to the information processing device 400 .

[0257] The communication unit 70 may store personal identification information in advance and transmit the stored personal identification information to the information processing device 400 in response to an instruction from a user (for example, a subject).

[0258] (Regarding the Iris Registration Unit 440) The iris registration unit 440 includes a selection unit 141 and a registration unit 143 similar to those in the first embodiment, and an identification unit 442, as shown in FIG. 25, for example.

[0259] The identifying unit 442 uses the personal identification information transmitted by the communication unit 70 to identify the target personal identification information from among the registered face information and personal identification information stored in association with each other in the registration storage unit 110 .

[0260] The iris registration unit 440 executes a registration process (step S440) as shown in Fig. 26. The registration process (step S440) is a process for storing iris information in the registration storage unit 110.

[0261] Step S141 is executed in the same manner as in the first embodiment.

[0262] The identification unit 442 uses the personal identification information transmitted in step S70 to identify the target personal identification information from the registered face information and personal identification information stored in association with each other in the registration storage unit 110.

[0263] (Regarding the Identification Unit 442) The identification unit 442, for example, acquires personal identification information transmitted by the communication unit 450. The identification unit 442 may, for example, identify target personal identification information from the personal identification information stored in the registration storage unit 110 based on the acquired personal identification information. The target personal identification information may, for example, be personal identification information that matches the acquired personal identification information. This allows the registration unit 143 to further associate the iris information related to the selected target iris image (for example, satisfying the first condition) with the personal identification information identified by the identification unit 142 and store it in the registration storage unit 110.

[0264] (Actions and Effects) As described above, according to this embodiment, the information processing system S4 further includes a communication unit 70 that transmits personal identification information. The iris registration unit 440 includes a selection unit 141, an identification unit 442, and a registration unit 143. The selection unit 141 selects a target iris image based on the first affinity score. The identification unit 442 identifies the target personal identification information based on the transmitted personal identification information. The registration unit 143 further associates the selected iris information with the identified personal identification information and stores it in the registration storage unit 110.

[0265] This allows the first extraction engine to select a target iris image that can be authenticated with high accuracy, and then register and authenticate the target iris image using the iris information related to the selected target iris image, thereby improving the accuracy of authentication using iris information.

[0266] [Embodiment 5] In the first embodiment, an example was given in which the identification unit 142 identifies the target personal identification information based on user input. In this embodiment, an example is described in which a first terminal device, which is a device other than the information processing device 100, stores personal identification information in advance without using user input. In addition, in this embodiment, an example is described in which a second terminal device reads a display image including personal identification information from the first terminal device and transmits the read personal identification information. Note that in this embodiment, for the sake of brevity, explanations that overlap with other embodiments will be omitted as appropriate.

[0267] As shown in FIG. 27, the information processing system S5 includes the same image capturing device 40 as in the first embodiment, an information processing device 500, a first communication terminal 80, and a second communication terminal 90.

[0268] The first communication terminal 80 includes a holding unit 81 and a display unit 82 .

[0269] The storage unit 81 stores personal identification information in advance.

[0270] The display unit 82 displays a display image including the retained personal identification information.

[0271] The second communication terminal 90 includes a reading unit 91 and a communication unit 92 .

[0272] The reading unit 91 reads the personal identification information from the displayed image.

[0273] The communication unit 92 transmits the read personal identification information.

[0274] It should be noted that the information processing system S5 may include the holding unit 81, the display unit 82, the reading unit 91, and the communication unit, and the device including these is not limited to the communication terminals 80 and 90. For example, the photographing device 40 may further include the reading unit 91 and the communication unit 92. Also, for example, the control unit 53 may transmit a target iris image generated within a predetermined time period when the reading unit 91 reads the personal identification information.

[0275] As shown in FIG. 28, the information processing device 500 includes a registration storage unit 110, an image acquisition unit 120, and a calculation unit 130 similar to those in the first embodiment, as well as an iris registration unit 540.

[0276] The iris registration unit 540 further associates the iris information regarding the selected target iris image with the registered face information and personal identification information of the target identified using the personal identification information transmitted by the communication unit 92 and stores it in the registration memory unit 110.

[0277] The information processing system S5 executes information processing such as that shown in FIG.

[0278] Steps S50, S120, and S130 are executed in the same manner as in the first embodiment.

[0279] The display unit 82 displays a display image including the retained personal identification information (step S82).

[0280] The reading unit 91 reads the personal identification information from the displayed image (step S91).

[0281] The communication unit 92 transmits the personal identification information (step S92).

[0282] The iris registration unit 540 further associates the iris information regarding the selected target iris image with the registered face information and personal identification information of the target identified using the personal identification information transmitted in step S92 and stores it in the registration memory unit 110 (step S540).

[0283] (Regarding the first communication terminal 80) The first communication terminal 80 is, for example, a mobile terminal (smartphone, tablet terminal, etc.) of a user (e.g., a target), but is not limited to this. Note that the first communication terminal 80 may be connected to the information processing device 500, etc. via the communication network NT so that they can send and receive information to and from each other, but they do not have to be connected.

[0284] (Regarding the storage unit 81) As described above, the storage unit 81 stores personal identification information in advance. The storage unit 81 may store in advance personal identification information for the same subject that is the same as the personal identification information stored in the registration storage unit 110. In detail, for example, when registered face information and personal identification information are stored in the registration storage unit 110, the storage unit 81 may acquire and store the personal identification information.

[0285] The method by which the storage unit 81 stores personal identification information in advance is not limited to the example described here.

[0286] (Regarding the display unit 82) The display unit 82 displays a display image including the personal identification information stored in the storage unit 81. For example, the display unit 82 acquires the personal identification information from the storage unit 81 based on a user instruction or the like, and displays a display image including the acquired personal identification information.

[0287] The display image is an image showing a one-dimensional code, a two-dimensional code, or the like, but is not limited to these examples.

[0288] (Regarding the second communication terminal 90) The second communication terminal 90 is, for example, a device disposed close to the image capturing device 40 or a device incorporated into the image capturing device 40, but is not limited to this. Note that the second communication terminal 90 may be connected to at least the information processing device 500 via the communication network NT so as to be able to transmit and receive information to and from each other.

[0289] (Regarding the reading unit 91) The reading unit 91 reads personal identification information from the display image displayed on the display unit 82. For example, the reading unit 91 may be configured with a scanner or the like, and when a user (e.g., a subject) carrying the first communication terminal 80 places the display image displayed on the display unit 82 in a predetermined area, the reading unit 91 may read the personal identification information included in the display image.

[0290] (Regarding the Communication Unit 92) The communication unit 92 transmits, for example, the personal identification information read by the reading unit 91 to the information processing device 500.

[0291] (Regarding the Iris Registration Unit 540) The iris registration unit 540 includes a selection unit 141 and a registration unit 143 similar to those in the first embodiment, and an identification unit 542, as shown in FIG.

[0292] The identifying unit 542 uses the personal identification information transmitted by the communication unit 92 to identify the target personal identification information from among the registered face information and personal identification information stored in association with each other in the registration storage unit 110 .

[0293] The iris registration unit 540 executes a registration process (step S540) as shown in Fig. 31. The registration process (step S540) is a process for storing iris information in the registration storage unit 110.

[0294] Step S141 is executed in the same manner as in the first embodiment.

[0295] The identifying unit 542 uses the personal identification information transmitted in step S92 to identify the target personal identification information from the registered face information and personal identification information stored in association with each other in the registration storage unit 110.

[0296] (Regarding the Identification Unit 542) The identification unit 542, for example, acquires personal identification information transmitted by the communication unit 92. For example, the identification unit 542 may identify target personal identification information from the personal identification information stored in the registration storage unit 110 based on the acquired personal identification information. The target personal identification information may be, for example, personal identification information that matches the acquired personal identification information.

[0297] (Actions and Effects) As described above, according to this embodiment, the information processing system S5 includes a storage unit 81, a display unit 82, a reading unit 91, and a communication unit 92. The storage unit 81 stores personal identification information in advance. The display unit 82 displays a display image including the stored personal identification information. The reading unit 91 reads the personal identification information from the display image. The communication unit 92 transmits the read personal identification information.

[0298] This allows the first extraction engine to select a target iris image that can be authenticated with high accuracy, and then register and authenticate the target iris image using the iris information related to the selected target iris image, thereby improving the accuracy of authentication using iris information.

[0299] Furthermore, since personal identification information is not transmitted from the information processing device 500, it is possible to reduce the possibility of personal identification information being leaked through communication and also to save the user the trouble of inputting the personal identification information. Therefore, it is possible to protect personal information and save the user the trouble of inputting the personal identification information.

[0300] [Embodiment 6] Each of the face camera and the iris camera may acquire a still image of the target, or may acquire multiple images by repeatedly capturing images of the target at predetermined time intervals. That is, the target face image and target iris image of the target transmitted by the control unit are not limited to one each, and one or both of them may be multiple. The image acquisition unit may acquire one or multiple target face images and one or multiple target iris images of the target.

[0301] In this embodiment, an example in which there are a plurality of target iris images will be described. Note that, in this embodiment, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0302] The information processing system S6 includes an image capturing device 40a including an image capturing section 50a, and an information processing device 600, as shown in FIG.

[0303] The photographing unit 50a photographs a target and acquires a target face image and a plurality of target iris images of the target.

[0304] As shown in FIG. 33, the information processing device 600 includes the same registration storage unit 110 as in the first embodiment, an image acquisition unit 620, a calculation unit 630, and an iris registration unit 640.

[0305] The image acquisition unit 620 acquires a target face image and a plurality of target iris images acquired by photographing a target.

[0306] The calculation unit 630 calculates a plurality of first affinity scores indicating the affinity between each of the plurality of target iris images and the first extraction engine based on a plurality of iris extraction information extracted from each of the plurality of target iris images using the first extraction engine.

[0307] The iris registration unit 640 stores iris information relating to at least one of the target iris images selected based on the plurality of first affinity scores in the registration storage unit 110 in further association with the registered face information and personal identification information.

[0308] The information processing system S6 executes information processing such as that shown in FIG.

[0309] The photographing unit 50a photographs a target and acquires a target face image and a plurality of target iris images of the target (step S50a).

[0310] The image acquisition unit 620 acquires a target face image and a plurality of target iris images acquired by photographing a target (step S620).

[0311] The calculation unit 630 calculates a plurality of first affinity scores indicating the affinity between each of the plurality of target iris images and the first extraction engine based on a plurality of iris extraction information extracted from each of the plurality of target iris images using the first extraction engine (step S630).

[0312] The iris registration unit 640 further associates iris information relating to at least one of the target iris images selected based on the plurality of first affinity scores with the registered face information and personal identification information and stores the information in the registration storage unit 110 (step S640).

[0313] (Regarding the Image Capturing Unit 50a) As shown in FIG. 35, the image capturing unit 50a includes a face camera 51, an iris camera 52a, and a control unit 53a, similar to those in the first embodiment.

[0314] The iris camera 52a photographs a target and acquires a plurality of target iris images including the iris of the target.

[0315] The control unit 53a transmits the target face image and the plurality of target iris images acquired by the face camera 51 and the iris camera 52a, respectively.

[0316] The photographing unit 50 executes a photographing process (step S50a) as shown in FIG. 36, for example.

[0317] Step S51 is executed in the same manner as in the first embodiment.

[0318] The iris camera 52a photographs a target and acquires a plurality of target iris images including the iris of the target (step S52a).

[0319] The control unit 53a transmits the target face image and the plurality of target iris images acquired by the face camera 51 and the iris camera 52a, respectively (step S53a).

[0320] (Regarding Iris Camera 52a) The iris camera 52a may capture an image of a target multiple times under the control of the control unit 53a, for example, to obtain multiple target iris images including the iris of the target.

[0321] (Regarding the control unit 53a) The control unit 53a transmits, for example, a target face image and a plurality of target iris images acquired by the face camera 51 and the iris camera 52a, respectively, simultaneously or sequentially to the information processing device 100. When transmitting the target face image and a plurality of target iris images sequentially, the control unit 53 may transmit any one of the target face image and the plurality of target iris images first.

[0322] (Image Acquisition Unit 620) The image acquisition unit 620 may acquire, for example, a target face image and a plurality of target iris images acquired by photographing a target with the photographing unit 50a from the photographing unit 50a or via another device.

[0323] (Regarding the Calculation Unit 630) As described above, the calculation unit 630 calculates a plurality of first affinity scores based on a plurality of pieces of iris extraction information extracted from a plurality of target iris images using the first extraction engine.

[0324] For example, the plurality of pieces of iris extraction information may be obtained by repeatedly extracting iris extraction information for each of a plurality of target iris images using a first extraction engine. The plurality of first affinity scores may correspond to the plurality of target iris images, respectively. Each of the first affinity scores may indicate an affinity between the corresponding target iris image and the first extraction engine.

[0325] (Regarding the Iris Registration Unit 640) As described above, the iris registration unit 640 stores, in the registration storage unit 110, iris information relating to at least one of the target iris images selected based on the plurality of first affinity scores.

[0326] Similar to the iris registration unit 140, the iris registration unit 640 may further associate the iris information with registered face information and personal identification information and store the information in the registration storage unit 110. When storing the information in the registration storage unit 110, the iris registration unit 640 may further associate the iris information related to the target iris image with the registered face information and personal identification information of the target corresponding to the target iris image and store the information in the registration storage unit 110.

[0327] The iris registration unit 640 includes, for example, a selection unit 641, the identification unit 142 similar to that of the first embodiment, and a registration unit 643, as shown in FIG.

[0328] The selection unit 641 selects a plurality of target iris images based on a plurality of first affinity scores.

[0329] The registration unit 643 stores the iris information relating to at least one target iris image that satisfies the first condition in the registration storage unit 110 in further association with the registered face information and personal identification information of the target.

[0330] The iris registration unit 640 executes a registration process (step S640) as shown in Fig. 38. The registration process (step S140) is a process for storing iris information in the registration storage unit 110.

[0331] The selection unit 641 selects a plurality of target iris images based on a plurality of first affinity scores (step S641).

[0332] Step S142 is executed in the same manner as in the first embodiment.

[0333] The registration unit 643 stores the iris information relating to at least one target iris image that satisfies the first condition in the registration storage unit 110 in further association with the registered face information and personal identification information of the target (step S643).

[0334] (Regarding the Selection Unit 641) The selection unit 641 determines whether each of the multiple first affinity scores calculated for each of the multiple target iris images satisfies a first condition, and then selects at least one target iris image for which a first affinity score that satisfies the first condition has been calculated as the target iris image that satisfies the first condition.

[0335] In detail, for example, the selection unit 641 may select target iris images having a first affinity score equal to or greater than a predetermined number, or having a value obtained by statistically processing a plurality of values ​​constituting the first affinity score equal to or greater than a threshold. For example, when a plurality of target iris images are selected, the selection unit 641 may select the target iris image having the largest first affinity score from the plurality of target iris images. For example, the selection unit 641 may select the target iris image having the largest first affinity score.

[0336] (Regarding the registration unit 643) As described above, the registration unit 643 further associates iris information related to at least one target iris image that satisfies the first condition with the registered face information and personal identification information of the target, and stores the information in the registration storage unit 110. Here, the registered face information and personal identification information of the target may include at least one of the registered face information and personal identification information identified by the identification unit 642.

[0337] (Actions and Effects) As described above, according to this embodiment, there are multiple target iris images. The calculation unit 630 calculates multiple first affinity scores indicating the affinity between each of the multiple target iris images and the first extraction engine based on multiple pieces of iris extraction information extracted from each of the multiple target iris images using the first extraction engine. The iris registration unit 640 stores the iris information for at least one of the target iris images selected based on the multiple first affinity scores in the registration storage unit 110, further associating it with registered face information and personal identification information.

[0338] This allows at least one target iris image that can be authenticated with high accuracy to be selected from among a plurality of target iris images using the first extraction engine, and iris information related to the at least one target iris image to be registered for authentication, thereby further improving the accuracy of authentication using iris information.

[0339] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0340] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0341] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. 1. An information processing system comprising: a registration storage means for storing registered face information related to a face and personal identification information in association with each other; a photographing means for photographing a target to obtain a target face image and a target iris image including the face and iris of the target, respectively; a calculation means for calculating a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine; and an iris registration means for storing iris information related to the target iris image selected based on the first affinity score in the registration storage means in further association with the registered face information and the personal identification information. 2. The information processing system described in 1., further comprising: an extraction means for inputting the target iris image to the first extraction engine to extract the iris extraction information; and an authentication means for authenticating the target using the iris extraction information, wherein the first affinity score is represented by a value corresponding to the accuracy of authentication of the target using the iris extraction information. 3. 2. The information processing system according to 2., further comprising a shooting environment changing means for changing a shooting environment for photographing the target, wherein the extraction means includes a first extraction means for inputting the target iris image to the first extraction engine and extracting the iris extraction information, and a second extraction means for inputting the target face image to a second extraction engine and extracting face extraction information, wherein the calculation means includes a first calculation means for calculating the first affinity score based on the iris extraction information, and a second calculation means for calculating a second affinity score indicating an affinity between the target face image and the second extraction engine based on the face extraction information, wherein the photographing means photographs the target to obtain a target face image including the face of the target, and then photographs the iris of the target to obtain a target iris image, and the shooting environment changing means generates a change instruction for changing the shooting environment for photographing the iris of the target based on the second affinity score. 4. The information processing system according to 3., wherein the authentication means authenticates the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score.5. The information processing system described in any one of 1. to 4., further comprising a communication means for transmitting personal identification information, wherein the iris registration means includes: a selection means for selecting the target iris image based on the first affinity score; a specification means for specifying personal identification information of the target based on the transmitted personal identification information; and a registration means for storing the selected iris information in the registration storage means in further association with the specified personal identification information. 6. The information processing system described in 5., further comprising: a storage means for storing personal identification information in advance; a display means for displaying a display image including the stored personal identification information; and a reading means for reading the personal identification information from the display image, wherein the communication means transmits the read personal identification information. 7. 8. The information processing system described in any one of 1. to 6., wherein there are a plurality of target iris images, and the calculation means calculates a plurality of first affinity scores indicating the affinity between each of the plurality of target iris images and the first extraction engine based on a plurality of pieces of iris extraction information extracted from each of the plurality of target iris images using the first extraction engine, and the iris registration means stores in the registration storage means iris information related to at least one of the target iris images selected based on the plurality of first affinity scores, further associated with the registered face information and the personal identification information. 9. An information processing device comprising: image acquisition means for acquiring a target face image and a target iris image obtained by photographing a target; calculation means for calculating a first affinity score indicating the affinity between the target iris image and the first extraction engine based on the iris extraction information extracted from the target iris image using the first extraction engine; and iris registration means for storing in the registration storage means iris information related to the target iris image selected based on the first affinity score. 8. The information processing device according to 8., further comprising: an extraction means that inputs the target iris image into the first extraction engine and extracts the iris extraction information; and an authentication means that authenticates the target using the iris extraction information, wherein the first affinity score is expressed as a value corresponding to accuracy of authentication of the target using the iris extraction information.10. The information processing device described in 9., further comprising a shooting environment changing means for changing a shooting environment for photographing the target, wherein the extraction means includes a first extraction means for inputting the target iris image to the first extraction engine and extracting the iris extraction information, and a second extraction means for inputting the target face image to a second extraction engine and extracting face extraction information, wherein the calculation means includes a first calculation means for calculating the first affinity score based on the iris extraction information, and a second calculation means for calculating a second affinity score indicating an affinity between the target face image and the second extraction engine based on the face extraction information, wherein the photographing means photographs the target to obtain a target face image including the face of the target, and then photographs the iris of the target to obtain a target iris image, and the environment changing means generates a change instruction for changing the shooting environment for photographing the iris of the target, based on the second affinity score. 11. The information processing device described in 10., wherein the authentication means authenticates the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score. 12. The information processing device described in any one of 8. to 11., further comprising a communication means for transmitting personal identification information, wherein the iris registration means includes: a selection means for selecting the target iris image based on the first affinity score, an identification means for identifying personal identification information of the target based on the transmitted personal identification information, and a registration means for storing the selected iris information in the registration storage means in further association with the identified personal identification information. 13. The information processing device described in 12., further comprising: a storage means for storing personal identification information in advance, a display means for displaying a display image including the stored personal identification information, and a reading means for reading the personal identification information from the display image, wherein the communication means transmits the read personal identification information.14. The information processing device described in any one of 8. to 13., wherein there are a plurality of target iris images, and the calculation means calculates a plurality of first affinity scores indicating the affinity between each of the plurality of target iris images and the first extraction engine based on a plurality of pieces of iris extraction information extracted from each of the plurality of target iris images using the first extraction engine, and the iris registration means stores iris information related to at least one of the target iris images selected based on the plurality of first affinity scores in the registration storage means, further associating it with the registration face information and the personal identification information. 15. An information processing method, wherein one or more computers acquire target face images and target iris images obtained by photographing a target, calculate a first affinity score indicating the affinity between the target iris image and the first extraction engine based on the iris extraction information extracted from the target iris image using a first extraction engine, and store iris information related to the target iris image selected based on the first affinity score in the registration storage unit. 16. The information processing method described in 15. further includes inputting the target iris image into the first extraction engine to extract the iris extraction information, and authenticating the target using the iris extraction information, wherein the first affinity score is expressed as a value corresponding to accuracy of authenticating the target using the iris extraction information. 17. The information processing method described in 16. further includes inputting the target face image into a second extraction engine to extract face extraction information, calculating a second affinity score indicating the affinity between the target face image and the second extraction engine based on the face extraction information, and changing a shooting environment for photographing the target, wherein acquiring the target face image and the target iris image includes photographing the target to obtain a target face image including the face of the target, and then photographing the iris of the target to obtain the target iris image, and changing the shooting environment includes generating a change instruction for changing the shooting environment for photographing the iris of the target based on the second affinity score. 18. 17. The information processing method according to 17., wherein authenticating the target includes authenticating the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score.19. The information processing method according to any one of 15. to 18., further comprising transmitting personal identification information, wherein storing the iris information comprises: selecting the target iris image based on the first affinity score; identifying personal identification information of the target based on the transmitted personal identification information; and storing the selected iris information in the registration storage means in further association with the identified personal identification information. 20. The information processing method according to 19., wherein the one or more computers comprise: storage means for pre-storing the personal identification information; display means for displaying a display image including the stored personal identification information; and reading means for reading the personal identification information from the display image, and wherein transmitting the personal identification information comprises transmitting the read personal identification information. 21. The information processing method described in any one of 15. to 20., wherein the target iris images are multiple, and calculating the first affinity score calculates multiple first affinity scores indicating affinity between each of the multiple target iris images and the first extraction engine based on multiple pieces of iris extraction information extracted from each of the multiple target iris images using the first extraction engine, and storing the iris information causes iris information for at least one of the target iris images selected based on the multiple first affinity scores to be further associated with the registered face information and the personal identification information and stored in the registration storage unit. 22. A program for causing one or more computers to execute the information processing method described in any one of 15. to 20.. 23. A recording medium having recorded thereon a program for causing one or more computers to execute the information processing method described in any one of 15. to 20.

[0342] S1 to S6 Information processing system 40, 340, 40a Photography device 50, 350, 50a Photography unit 53, 353, 53a Control unit 60 Communication terminal 70 Communication unit 80 First communication terminal 81 Holding unit 82 Display unit 90 Second communication terminal 91 Reading unit 92 Communication unit 100, 200, 300, 400, 500, 600 Information processing device 110 Registration storage unit 120, 320, 620 Image acquisition unit 130, 330, 630 Calculation unit 140, 440, 540, 640 Iris registration unit 141, 641 Selection unit 142, 442, 542, 642 Identification unit 143, 643 Registration unit 260, 360 Extraction unit 270, 370 Authentication unit 330a First calculation unit 330b Second calculation unit 360a First extraction unit 360b Second extraction unit 380 Shooting environment change unit 450 Communication unit

Claims

1. An information processing system comprising: a registration storage means for storing registered face information relating to a face in association with personal identification information; a photographing means for photographing a subject to obtain a target face image and a target iris image including the subject's face and iris, respectively; a calculation means for calculating a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine; and an iris registration means for storing iris information relating to the target iris image selected based on the first affinity score in the registration storage means in further association with the registered face information and the personal identification information.

2. The information processing system of claim 1, further comprising: an extraction means for inputting the target iris image into the first extraction engine to extract the iris extraction information; and an authentication means for authenticating the target using the iris extraction information, wherein the first affinity score is expressed as a value corresponding to the accuracy of authentication of the target using the iris extraction information.

3. The information processing system according to claim 2, further comprising a shooting environment changing means for changing the shooting environment for photographing the subject, wherein the extraction means includes a first extraction means for inputting the subject iris image into the first extraction engine and extracting the iris extraction information, and a second extraction means for inputting the subject face image into a second extraction engine and extracting face extraction information, wherein the calculation means includes a first calculation means for calculating the first affinity score based on the iris extraction information, and a second calculation means for calculating a second affinity score indicating the affinity between the subject face image and the second extraction engine based on the face extraction information, wherein the photographing means photographs the subject to obtain a target face image including the subject's face, and then photographs the subject's iris to obtain a target iris image, and wherein the photographing environment changing means generates a change instruction regarding the change of the shooting environment for photographing the subject's iris based on the second affinity score.

4. The information processing system described in claim 3, wherein the authentication means authenticates the target using the target iris image selected based on the first affinity score and the target face image selected based on the second affinity score.

5. An information processing system as described in any one of claims 1 to 4, further comprising a communication means for transmitting personal identification information, wherein the iris registration means includes: a selection means for selecting the target iris image based on the first affinity score; an identification means for identifying the personal identification information of the target based on the transmitted personal identification information; and a registration means for further associating the selected iris information with the identified personal identification information and storing it in the registration storage means.

6. An information processing system as described in claim 5, further comprising: a storage means for storing personal identification information in advance; a display means for displaying a display image including the stored personal identification information; and a reading means for reading the personal identification information from the display image, wherein the communication means transmits the read personal identification information.

7. An information processing system as described in any one of claims 1 to 6, wherein there are multiple target iris images, the calculation means calculates multiple first affinity scores indicating the affinity between each of the multiple target iris images and the first extraction engine based on multiple iris extraction information extracted from each of the multiple target iris images using the first extraction engine, and the iris registration means further associates the iris information regarding at least one of the target iris images selected based on the multiple first affinity scores with the registered face information and the personal identification information and stores it in the registration storage means.

8. An information processing device comprising: an image acquisition means for acquiring a target face image and a target iris image obtained by photographing a target; a calculation means for calculating a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine; and an iris registration means for storing iris information related to the target iris image selected based on the first affinity score in a registration storage means.

9. An information processing method in which one or more computers acquire a target face image and a target iris image obtained by photographing a target, calculate a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine, and store iris information related to the target iris image selected based on the first affinity score in a registration memory unit.

10. A recording medium having recorded thereon a program for causing one or more computers to acquire a target face image and a target iris image obtained by photographing a target, calculate a first affinity score indicating the affinity between the target iris image and a first extraction engine based on iris extraction information extracted from the target iris image using a first extraction engine, and store the iris information related to the target iris image selected based on the first affinity score in a registration memory unit.

Images