Information processing device, information processing method, and computer program

Abstract

In the present invention, an information processing device 1 comprises: a learning data generation unit 11 that generates second learning data on the basis of first learning data including an eye image in which the eye of a subject appears and information related to an article mounted on the subject appearing in the eye image included in the first learning data; a model generation unit 12 that generates an authentication model using the second learning data; and an authentication unit 13 that authenticates the subject using the authentication model.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Patent Literature 1 describes a technology for acquiring a target iris image, which is an iris image to be processed, searching a registered iris image for one or more similar registered iris images that are similar to the target iris image, and determining that the target iris image and the similar registered iris image are iris images wearing color contact lenses if the person corresponding to the target iris image and the person corresponding to the similar registered iris image are different people. Patent Literature 2 describes a technology for acquiring an image of a subject's eye, comparing the eye image with a reference image to identify the color pattern of the color contact lenses worn by the subject, and identifying the subject using features of the iris region of the eye other than the colored region of the color pattern. Patent Literature 3 describes a method for detecting iris edges in an image, acquiring texture from the image, combining the edges and texture to generate inner and outer iris boundaries, and selecting between an ellipse model and a circle model to improve iris boundary detection accuracy. Patent Literature 4 describes a technology for extracting the iris from an image by using a dome model to obtain a mask image and removing occlusions by eyelids in the unwrapped image. Patent Literature 4 describes a technology in which a captured image is input into a first learning model and whether iris information can be extracted based on the output from the first learning model is determined. The first learning model is constructed by machine learning using training data in which multiple eye images are associated with labels indicating whether irises can be extracted. Patent Literature 5 describes a technology in which different iris images corresponding to the same person are extracted as cosmetic lens candidates, the iris feature values ​​of the cosmetic lens candidate are compared with the iris feature values ​​of other cosmetic lens candidates, a reliability indicating the likelihood of the candidate being a cosmetic lens is calculated based on the comparison results, and if the calculated reliability is equal to or greater than a predetermined threshold, the cosmetic lens candidate is determined to be a cosmetic lens. Non-Patent Literature 1 describes a technology in which an image is divided into patches, a weight is calculated based on the degree of occlusion of each patch, and a weight is applied to areas with less occlusion to estimate facial expression.

[0003] International Publication No. 2022 / 014001 International Publication No. 2020 / 065935 Special Publication No. 2009-523266 International Publication No. 2021 / 192311 International Publication No. 2021 / 245932

[0004] Y. Li, et al. Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 5, MAY 2019

[0005] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents.

[0006] One aspect of the information processing device includes a training data generation means for generating second training data based on first training data including an eye image showing the eyes of a subject and information relating to an accessory of the subject that is shown in the eye image included in the first training data, a model generation means for generating an authentication model using the second training data, and an authentication means for authenticating the subject using the authentication model.

[0007] One aspect of the information processing method generates second training data based on first training data including an eye image showing the subject's eyes and information about the subject's accessories that appear in the eye image included in the first training data, generates an authentication model using the second training data, and authenticates the subject using the authentication model.

[0008] One aspect of the recording medium has recorded thereon a computer program for causing a computer to execute an information processing method that generates second training data based on first training data including an eye image showing the eyes of a subject and information about the subject's accessories that are shown in the eye image included in the first training data, generates an authentication model using the second training data, and authenticates the subject using the authentication model.

[0009] FIG. 1 is a block diagram showing the configuration of an information processing device in a first embodiment. FIG. 2 is a block diagram showing the configuration of an information processing device in a second embodiment. FIG. 3 is a conceptual diagram showing the flow of information processing operations of the information processing device in the second embodiment. FIG. 4 is a block diagram showing the configuration of an information processing device in a third embodiment. FIG. 5 is a conceptual diagram showing the flow of information processing operations of the information processing device in the third embodiment. FIG. 6 is a block diagram showing the configuration of an information processing device in a fourth embodiment. FIG. 7 is a conceptual diagram showing the flow of information processing operations of the information processing device in the fourth embodiment. FIG. 8 is a block diagram showing the configuration of an information processing device in a fifth embodiment. FIG. 9 is a conceptual diagram showing the flow of information processing operations of the information processing device in the fifth embodiment. FIG. 10 is a block diagram showing the configuration of an information processing device in a sixth embodiment. FIG. 11 is a conceptual diagram showing the flow of information processing operations of the information processing device in the sixth embodiment. FIG. 12 is a block diagram showing the configuration of an information processing device in a seventh embodiment. FIG. 13 is a conceptual diagram showing the flow of information processing operations of the information processing device in the seventh embodiment. FIG. 14 is a block diagram showing the configuration of an information processing device in an eighth embodiment. FIG. 15 is a block diagram showing the configuration of an information processing device in a ninth embodiment. FIG. 16 is a conceptual diagram showing the flow of information processing operations of the information processing device in the ninth embodiment.

[0010] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings. [1: First Embodiment]

[0011] A first embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the first embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 1 to which the first embodiment of the information processing device, the information processing method, and the recording medium is applied. [1-1: Configuration of Information Processing Device 1]

[0012] 1 is a block diagram showing the configuration of an information processing device 1 according to the first embodiment. As shown in FIG. 1, the information processing device 1 includes a training data generation unit 11, a model generation unit 12, and an authentication unit 13.

[0013] The training data generation unit 11 generates second training data based on first training data including eye images of the target's eyes and information about the target's accessories that appear in the eye images included in the first training data. The model generation unit 12 generates an authentication model using the second training data. The authentication unit 13 authenticates the target person using the authentication model. [1-2: Technical Effects of the Information Processing Device 1]

[0014] The information processing device 1 in the first embodiment generates second training data based on information about the subject's accessories shown in the eye image. The information processing device 1 authenticates the subject using an authentication model generated using this second training data, so that highly accurate authentication can be performed regardless of whether the subject is wearing accessories. [2: Second Embodiment]

[0015] Next, a second embodiment of the information processing device, the information processing method, and the recording medium will be described. Hereinafter, the second embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 2 to which the second embodiment of the information processing device, the information processing method, and the recording medium is applied. [2-1: Configuration of the information processing device 2]

[0016] 2 is a block diagram showing the configuration of an information processing device 2 in the second embodiment. As shown in FIG. 2, the information processing device 2 includes a calculation device 21 and a storage device 22. The information processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not necessarily have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is included in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing the operations to be performed by the information processing device 2 are realized within the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) to be performed by the information processing device 2.

[0018] 2 shows an example of logical functional blocks implemented within the computing device 21 to perform information processing operations. As shown in FIG. 2, the computing device 21 includes a learning data generation unit 211, which is a specific example of a "learning data generation means" described in the appendix, a model generation unit 212, which is a specific example of a "model generation means" described in the appendix, an authentication unit 213, which is a specific example of an "authentication means" described in the appendix, a learning data acquisition unit 214, which is a specific example of a "learning data acquisition means" described in the appendix, an accessory detection unit 215, which is a specific example of an "accessory detection means" described in the appendix, and an eye image acquisition unit 216. However, at least one of the learning data acquisition unit 214, the accessory detection unit 215, and the eye image acquisition unit 216 does not necessarily have to be implemented within the computing device 21. The operations of the learning data generation unit 211, model generation unit 212, authentication unit 213, learning data acquisition unit 214, accessory detection unit 215, and eye image acquisition unit 216 will be described in detail later with reference to FIG.

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores long-term. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0020] The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or USB (Universal Serial Bus).

[0021] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0022] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-2: Information Processing Operation Performed by the Information Processing Device 2]

[0023] The information processing operation performed by the information processing device 2 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the information processing operation performed by the information processing device 2. Fig. 3(a) is a flowchart showing the flow of the learning operation performed by the information processing device 2, and Fig. 3(b) is a flowchart showing the flow of the authentication operation performed by the information processing device 2. [2-2-1: Learning Operation]

[0024] As shown in FIG. 3A, the training data acquisition unit 214 acquires first training data including eye images showing the eyes of a target (step S20). In this embodiment, the target includes a person. Also, in this embodiment, the target includes animals such as dogs and snakes. The following description will be given using an example in which the target is a person. The first training data includes eye images of a person not wearing an accessory and eye images of a person wearing an accessory. Furthermore, the eye images of a person wearing an accessory include eye images of people wearing various types of accessory. In this embodiment, the accessory refers to, for example, eyeglasses, contact lenses, masks, and other accessories worn around the eyes. The accessory is an object that affects authentication processing, matching processing, and the like, using the area of ​​a person's eyes included in the eye image, particularly the area of ​​the person's iris.

[0025] The first learning data may be stored in the storage device 22. In this case, the learning data acquisition unit 214 may acquire the first learning data from the storage device 22. Alternatively, the learning data acquisition unit 214 may acquire the first learning data from a device external to the information processing device 2 via the communication device 23.

[0026] The accessory detection unit 215 detects accessories of the person appearing in the eye image from the eye image included in the first learning data (step S21). In this embodiment, the type of accessory may be predetermined. That is, the accessory detection unit 215 may detect whether the person appearing in the eye image is wearing an accessory, and if so, what type of accessory the person is wearing.

[0027] The type of attachment may be classified according to the effect that the attachment has on the eye area of ​​the person included in the eye image. The type of attachment may be classified according to the manner in which the eye area of ​​the person included in the eye image is hidden by the attachment. The type of attachment may be classified according to the effect that the eye area of ​​the person included in the eye image is hidden by the attachment. The type of attachment may be classified according to the effect on the authentication result that the eye area of ​​the person included in the eye image is hidden by the attachment. Furthermore, for example, eyeglasses as attachments may be classified into multiple types according to the type of frame, the type of lens, etc.

[0028] For example, suppose N types of attachments are defined. In this case, the attachment detection unit 215 may detect whether the person in the eye image is (0) not wearing any attachments, (1) wearing a first type of attachment, (2) wearing a second type of attachment, ..., or (N) wearing an Nth type of attachment. The attachment detection unit 215 may calculate the likelihood that the person is not wearing any attachments and, if the person is wearing attachments, the likelihood that the person is wearing each type of attachment. The attachment detection unit 215 may detect which of the above cases (0) to (N) is most likely. Furthermore, the person in the eye image may be wearing multiple attachments (e.g., a first type of attachment and a second type of attachment). In this case, the attachment detection unit 215 may detect the above cases (1) and (2). Below, this embodiment will be described using an example in which N types of attachments are defined.

[0029] Each of the above (0) to (N) may also be referred to as an attachment class WC. An eye image in which a person is not wearing any attachment may be said to belong to the 0th attachment class WC0. An eye image in which a person is wearing the first type of attachment may be said to belong to the first attachment class WC1. ...An eye image in which a person is wearing the Nth type of attachment may be said to belong to the Nth attachment class WCN.

[0030] The training data generation unit 211 generates second training data from the first training data (step S22). In this embodiment, the second training data may be training data generated for learning iris characteristics. The second training data may be training data having desired characteristics related to an attachment. The second training data may be training data having desired iris characteristics related to an attachment. The second training data may be training data generated in particular for learning the effect on the iris when a person wears an attachment.

[0031] The training data generation unit 211 generates second training data based on the first training data and information about accessories worn by people appearing in eye images included in the first training data. The training data generation unit 211 generates the second training data from the first training data in accordance with the detection result by the accessory detection unit 215.

[0032] Based on the detection results by the accessory detection unit 215, the learning data generation unit 211 may classify the eye images included in the first learning data into one of the following: an 0th accessory class WC0 in which the person is not wearing any accessory, a 1st accessory class WC1 in which the person is wearing a 1st type of accessory, a 2nd accessory class WC2 in which the person is wearing a 2nd type of accessory, ..., an Nth accessory class WCN in which the person is wearing an Nth type of accessory.

[0033] The training data generation unit 211 may generate the second training data so that the ratio of the number of eye images belonging to each accessory class WC is a desired ratio. The training data generation unit 211 may generate the second training data by adjusting the second training data so that there are no accessory classes WC with too few eye images or no accessory classes WC with too many eye images. For example, the training data generation unit 211 may perform undersampling to select necessary eye images and oversampling to pad out necessary eye images. Oversampling may be achieved by copying the corresponding eye images.

[0034] The training data generation unit 211 may generate the second training data so that the number of eye images belonging to each accessory class WC is the same. Alternatively, the training data generation unit 211 may generate the second training data based on the detection result of the accessory detection unit 215 so that the ratio of eye images belonging to each accessory class WC is a desired ratio. In other words, the number of eye images belonging to each accessory class WC does not have to be the same. The training data generation unit 211 may adjust the number of eye images belonging to each accessory class WC according to the number of eye images belonging to each accessory class WC included in the first training data, starting from a state in which the number of eye images belonging to each accessory class WC is constant.

[0035] The second training data may be training data including an adjusted number of eye images belonging to each attachment class WC. In other words, the training data generating unit 211 adjusts the number of eye images included in the first training data to perform desired iris authentication, and generates the second training data.

[0036] The model generation unit 212 generates an authentication model using the second training data (step S23). In this embodiment, the authentication model is an iris authentication model. The iris authentication model may be a model that outputs an iris authentication result when an eye image including an iris is input.

[0037] It is not necessary to detect the attached item within the information processing device 2. In this case, for example, the eye image included in the acquired first learning data may have information about the attached item of the person appearing in the eye image. [2-2-2: Authentication Operation]

[0038] 3B, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The authentication unit 213 performs iris authentication of the subject using an iris authentication model as an authentication model (step S25). [2-3: Technical Effects of Information Processing Device 2]

[0039] In iris authentication using the iris region of a subject's eye image, identity verification is performed using the subject's iris region and a pre-stored iris region. Even if the subject's iris region and the pre-registered iris region belong to the same person, there may be differences. For example, if the subject is wearing glasses or other accessories when an iris photograph is taken, the effects of the glasses or other accessories may be reflected in the subject's iris region image, potentially affecting the accuracy of authentication.

[0040] Furthermore, if the number of eye images belonging to each accessory class WC included in the training data used to build the authentication model varies greatly, training may be performed with less emphasis on accessory classes WC with fewer eye images. For example, an authentication model may be generated that is built by performing heavy training on the 0th accessory class WC0, which has the largest number of eye images, but light training on accessory classes WC with fewer eye images.

[0041] The information processing device 2 in the second embodiment generates second training data for generating an authentication model from first training data obtained by detecting and classifying accessories worn by a person captured in an eye image. Therefore, it is possible to generate second training data with a balanced ratio of accessory classes, suitable for training according to the person's accessories. Since the information processing device 2 generates an authentication model using the second training data, it is possible to perform authentication training not only for the iris region but also for the accessories. The information processing device 2 can build an authentication model that is robust against occlusion, including the accessories. Since the information processing device 2 authenticates the subject using the authentication model, it is possible to perform accurate authentication regardless of whether the subject is wearing accessories. In other words, the information processing device 2 can perform accurate iris authentication even when the subject is wearing accessories. [3: Third Embodiment]

[0042] Next, a third embodiment of the information processing device, the information processing method, and the recording medium will be described. Hereinafter, the third embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 3 to which the third embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0043] 4 is a block diagram showing the configuration of an information processing device 3 according to the third embodiment. The information processing device 3 according to the third embodiment differs from the information processing device 2 according to the second embodiment in the operation of a training data generation unit 311, the operation of a model generation unit 312, and the operation of an authentication unit 313. [3-1: Information Processing Operation Performed by Information Processing Device 3]

[0044] The flow of information processing operations performed by the information processing device 3 in the third embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of information processing operations performed by the information processing device 3 in the third embodiment. Fig. 5(a) is a flowchart showing the flow of learning operations performed by the information processing device 3, and Fig. 5(b) is a flowchart showing the flow of authentication operations performed by the information processing device 3. [3-1-1: Learning Operation]

[0045] 5A, the training data acquisition unit 214 acquires first training data including eye images showing the eyes of a person (step S20). The accessory detection unit 215 detects accessories of the person shown in the eye images from the eye images included in the first training data (step S21).

[0046] Based on the detection results of the accessory detection unit 215, the training data generation unit 311 generates second training data including classified training data CD obtained by classifying the first training data based on whether the person depicted in the eye image is wearing an accessory and, if so, what type of accessory the person is wearing (step S30). In other words, the second training data generated by the training data generation unit 311 includes 0th classification training data CD0 consisting of eye images belonging to the 0th accessory class WC0, 1st classification training data CD1, ... consisting of eye images belonging to the 1st accessory class WC1, and Nth classification training data CDN consisting of eye images belonging to the Nth accessory class WCN. The training data generation unit 311 in the third embodiment may appropriately adjust the number of eye images belonging to each accessory class WC, as described in the second embodiment.

[0047] The model generation unit 312 uses each piece of categorized training data CD to generate each categorized authentication model CM according to the category (step S31). For example, the model generation unit 312 uses the 0th category training data CD0 to generate a 0th category authentication model CM0 according to the 0th accessory class WC0. The model generation unit 312 uses the 1st category training data CD1 to generate a 1st category authentication model CM1 according to the 1st accessory class WC1. ...The model generation unit 312 uses the Nth category training data CDN to generate an Nth category authentication model CMN according to the Nth accessory class WCN. Note that in other embodiments, when the model generation unit 312 is implemented within a computing device, the information processing device generates each categorized authentication model CM according to the category. [3-1-2: Authentication Operation]

[0048] As shown in FIG. 5B, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The authentication unit 313 performs iris authentication of the subject using the classification authentication model CM (step S32). In the third embodiment, the authentication unit 313 may perform iris authentication using at least one classification authentication model CM from the 0th classification authentication model CM0 to the Nth classification authentication model CMN. [3-2: Technical Effects of Information Processing Device 3]

[0049] The information processing device 3 in the third embodiment generates second learning data including classification learning data obtained by classifying the first learning data based on the detection result of the person's accessories in the eye image, according to whether the person is wearing accessories and, if so, what type of accessories the person is wearing. Therefore, it is possible to acquire learning data suitable for the accessory class. Since the information processing device 3 can perform learning using the learning data suitable for the accessory class, it is possible to generate each of the classification authentication models CM suitable for the accessory class. [4: Fourth Embodiment]

[0050] A fourth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the fourth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 4 to which the fourth embodiment of the information processing device, the information processing method, and the recording medium is applied. [4-1: Configuration of Information Processing Device 4]

[0051] The configuration of the information processing device 4 in the fourth embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the configuration of the information processing device 4 in the fourth embodiment.

[0052] As shown in FIG. 6, the information processing device 4 in the fourth embodiment differs from the information processing device 2 in the second embodiment and the information processing device 3 in the third embodiment in that an authentication model selection unit 417 is implemented in the calculation device 21. Also, in the information processing device 4, as in the information processing device 3, the model generation unit 312 generates each of the categorized authentication models CM according to the categories. Other features of the information processing device 4 may be the same as other features of at least one of the information processing device 2 and the information processing device 3. Therefore, hereinafter, differences from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [4-2: Information Processing Operation Performed by Information Processing Device 4]

[0053] The flow of information processing operations performed by the information processing device 4 in the fourth embodiment will be described with reference to Fig. 7. The information processing operations performed by the information processing device 4 in the fourth embodiment differ in authentication operations from the information processing operations performed by the information processing device 3 in the third embodiment. Fig. 7 is a flowchart showing the flow of authentication operations performed by the information processing device 4 in the fourth embodiment.

[0054] As shown in FIG. 7 , the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The accessory detection unit 415 detects the accessory worn by the subject from the eye image of the subject (step S40). The authentication model selection unit 417 selects which of the categorized authentication models CM the authentication unit 413 will use based on the detection result of the accessory detection unit 415 (step S41). The authentication model selection unit 417 uses the accessory detection result to select a categorized authentication model CM suitable for the accessory worn by the subject. The authentication unit 413 performs iris authentication of the subject using the selected categorized authentication model CM (step S42). [4-3: Technical Effects of Information Processing Device 4]

[0055] The information processing device 4 in the fourth embodiment selects a categorized authentication model CM according to what the subject wears, and authenticates the subject using the selected categorized authentication model CM, thereby enabling highly accurate authentication.

[0056] When iris authentication is performed using the same authentication model, the authentication result may differ depending on whether the subject is wearing an accessory or not. The information processing device 3 uses different classification authentication models CM depending on the accessory class WC, so the authentication result is stable. [5: Fifth Embodiment]

[0057] A fifth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the fifth embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 5 to which the fifth embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0058] 8 is a block diagram showing the configuration of an information processing device 5 according to the fifth embodiment. The information processing device 5 according to the fifth embodiment differs from the information processing device 2 according to the second embodiment to the information processing device 4 according to the fourth embodiment in the operation of the authentication unit 513. [5-1: Information Processing Operation Performed by Information Processing Device 5]

[0059] The flow of information processing operations performed by the information processing device 5 in the fifth embodiment will be described with reference to Fig. 9 . The information processing operations performed by the information processing device 5 in the fifth embodiment differ in authentication operations from the information processing operations performed by at least one of the information processing device 3 in the third embodiment and the information processing device 4 in the fourth embodiment. Also in the information processing device 5, similar to the information processing device 3, the model generation unit 312 generates each of the categorized authentication models CM according to the categories. Fig. 9 is a flowchart showing the flow of authentication operations performed by the information processing device 5 in the fifth embodiment.

[0060] As shown in FIG. 9 , the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The authentication unit 513 performs iris authentication of the subject using each of the categorized authentication models CM (step S50). The authentication unit 513 determines whether or not the subject's iris can be authenticated based on each of the authentication results from each of the categorized authentication models CM (step S51). For example, the authentication unit 513 may perform matching using all of the 0th category authentication model CM0 to the Nth category authentication model CMN, and determine whether or not authentication can be performed using the highest matching score among the respective matching scores. Alternatively, the authentication unit 513 may perform matching using all of the 0th category authentication model CM0 to the Nth category authentication model CMN, calculate an average score by averaging the respective matching scores, and determine whether or not authentication can be performed using the average score. [5-2: Technical Effects of Information Processing Device 5]

[0061] The information processing device 5 in the fifth embodiment performs authentication using each of the categorized authentication models CM, and therefore can perform appropriate authentication regardless of whether the subject is wearing an attachment. [6: Sixth Embodiment]

[0062] A sixth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the sixth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 6 to which the sixth embodiment of the information processing device, the information processing method, and the recording medium is applied. [6-1: Configuration of Information Processing Device 6]

[0063] The configuration of the information processing device 6 in the sixth embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the information processing device 6 in the sixth embodiment.

[0064] As shown in FIG. 10 , the information processing device 6 in the sixth embodiment differs from the information processing device 2 in the second embodiment to the information processing device 5 in the fifth embodiment in that a categorized registration data generation unit 618 and a categorized registration data selection unit 619 are implemented in the calculation device 21. It also differs in that a registration data holding unit 621 is implemented in the storage device 22. Similarly to the information processing device 3, the model generation unit 312 in the information processing device 6 generates each of the categorized authentication models CM according to the classification. Other features of the information processing device 6 may be the same as at least one of the other features of the information processing device 2 to the information processing device 5. Therefore, the following will describe in detail the parts that differ from the already-described embodiments, and will omit a description of other overlapping parts as appropriate. [6-2: Information Processing Operation Performed by Information Processing Device 6]

[0065] The flow of information processing operations performed by the information processing device 6 in the sixth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of information processing operations performed by the information processing device 6 in the sixth embodiment. Fig. 11(a) is a flowchart showing the flow of registration data generation operations performed by the information processing device 6, and Fig. 11(b) is a flowchart showing the flow of authentication operations performed by the information processing device 6.

[0066] The enrollment data storage unit 621 in the sixth embodiment stores enrolled eye images. The enrollment data storage unit 621 stores enrollment data including enrolled eye images showing a person's eyes. The enrollment data may include an eye image of only one eye, or may include eye images of both eyes. [6-2-1: Enrollment Data Generation Operation]

[0067] As shown in FIG. 11A, the categorized registration data generation unit 618 acquires registration data (step S60). The accessory detection unit 615 detects accessories of the person appearing in the eye image from the eye image included in the registration data (step S61). Based on the detection result of the accessory detection unit 615, the categorized registration data generation unit 618 generates categorized registration data CRD by classifying the registration data according to whether the person appearing in the eye image is wearing accessories and, if so, what type of accessories the person is wearing (step S62). In other words, the categorized registration data generation unit 618 generates 0th categorized registration data CRD0 consisting of eye images belonging to the 0th accessory class WC0, 1st categorized registration data CRD1, ... consisting of eye images belonging to the 1st accessory class WC1, and Nth categorized registration data CRDN consisting of eye images belonging to the Nth accessory class WCN. The categorized registration data generating unit 618 may generate registration data including the 0th categorized registration data CRD0, the 1st categorized registration data CRD1, . . . , and the Nth categorized registration data CRDN, and store the generated registration data in the registration data storing unit 621.

[0068] 11B, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The attachment detection unit 615 detects the subject's attachment from the eye image of the subject (step S63). The categorized registration data selection unit 619 selects which of the categorized registration data CRD the authentication unit 613 will use based on the detection result of the attachment detection unit 615 (step S64). The authentication unit 613 performs iris authentication of the subject using the categorized registration data CRD selected by the categorized registration data selection unit 619 (step S65). That is, in the sixth embodiment, different registration data is used depending on the subject's attachment. Furthermore, the authentication unit 613 may select the categorized authentication model CM to use depending on the attachment detection result. 6-3: Technical Effects of Information Processing Device 6

[0069] The information processing device 6 in the sixth embodiment classifies the registration data based on whether the person in the eye image is wearing an accessory and, if so, what type of accessory the person is wearing, and selects the categorized registration data CRD to be used for authentication based on the accessory of the subject, thereby enabling more accurate authentication. [7: Seventh Embodiment]

[0070] A seventh embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the seventh embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 7 to which the seventh embodiment of the information processing device, the information processing method, and the recording medium is applied. [7-1: Configuration of Information Processing Device 7]

[0071] The configuration of the information processing device 7 in the seventh embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing the configuration of the information processing device 7 in the seventh embodiment.

[0072] As shown in FIG. 12 , the information processing device 7 in the seventh embodiment differs from the information processing device 2 in the second embodiment to the information processing device 6 in the sixth embodiment in that the authentication unit 713 has a matching unit 7131, a weighting unit 7132, and an appropriateness determination unit 7133, and the attachment detection unit 715 has a calculation unit 7151 and a determination unit 7152. Also, in the information processing device 7, similar to the information processing device 3, the model generation unit 312 generates each of the categorized authentication models CM according to the category. Other features of the information processing device 7 may be the same as at least one other feature of the information processing device 2 to the information processing device 6. Therefore, hereinafter, differences from the already-described embodiments will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [7-2: Information Processing Operation Performed by Information Processing Device 7]

[0073] The flow of information processing operations performed by the information processing device 7 in the seventh embodiment will be described with reference to Fig. 13. The information processing operations performed by the information processing device 7 in the seventh embodiment differ in authentication operations from the information processing operations performed by at least one of the information processing device 3 in the third embodiment to the information processing device 6 in the sixth embodiment. Fig. 13 is a flowchart showing the flow of authentication operations performed by the information processing device 7 in the seventh embodiment.

[0074] As shown in Fig. 13, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The calculation unit 7151 calculates the likelihood that the subject is not wearing an accessory and, if the subject is wearing an accessory, the likelihood that the subject is wearing each type of accessory (step S70). The determination unit 7152 determines a weight for each accessory class WC according to the likelihood of each accessory class WC (step S71). The determination unit 7152 may use the likelihood value as the weight directly.

[0075] The matching unit 7131 performs iris authentication of the subject using each of the categorized authentication models CM (step S72). The weighting unit 7132 weights each authentication result using the weight of the attachment class WC corresponding to the categorized authentication model CM (step S73). For example, the weighting unit 7132 may calculate a weighted average score by weighting the scores output by each of the categorized authentication models CM using weights according to likelihood. The possibility determination unit 7133 determines whether or not the subject's iris authentication is possible based on each of the weighted authentication results (step S74). [7-3: Technical Effects of the Information Processing Device 7]

[0076] The information processing device 7 in the seventh embodiment determines whether or not to authenticate the subject based on each of the authentication results weighted according to the likelihood of the attachment, thereby enabling highly accurate authentication. [8: Eighth Embodiment]

[0077] An eighth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the eighth embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 8 to which the eighth embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0078] 14 is a block diagram showing the configuration of an information processing device 8 according to the eighth embodiment. The information processing device 8 according to the eighth embodiment differs from the information processing device 2 according to the second embodiment to the information processing device 7 according to the seventh embodiment in the operation of a model generation unit 812 and the operation of an authentication unit 813. [8-1: Information Processing Operation Performed by Information Processing Device 8]

[0079] The authentication model has a function of outputting features of an eye image when an eye image is input. When eye images of the same person are input, the model generation unit 812 generates an authentication model so that features similar to or more than a predetermined level are output when an eye image of a person not wearing any accessory is input, regardless of whether the person in the eye image is wearing an accessory. The model generation unit 812 generates an authentication model so that features similar to or more than a predetermined level are output when an eye image of the person wearing any accessory is input and features similar to or more than a predetermined level are output when an eye image of the person not wearing any accessory is input.

[0080] The authentication unit 813 in the eighth embodiment performs iris authentication using an eye image without any attachment as enrollment data. Unlike the sixth embodiment described above, the authentication unit 813 in the eighth embodiment can extract features similar to those without any attachment, so the authentication unit 813 can perform iris authentication by preparing only enrollment data without any attachment. In other words, in the eighth embodiment, it is sufficient to enroll only an eye image without any attachment as enrollment data. [8-2: Technical Effects of the Information Processing Device 8]

[0081] The information processing device 8 in the eighth embodiment can perform authentication by registering an eye image when the user is not wearing an attachment. [9: Ninth embodiment]

[0082] A ninth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the ninth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 9 to which the ninth embodiment of the information processing device, the information processing method, and the recording medium is applied. [9-1: Eye Surrounding Authentication]

[0083] In the ninth embodiment, eye-surrounding authentication is performed in addition to or instead of iris authentication. Eye-surrounding authentication may be performed by focusing on the area around the eyes, extracting features from the area around the eyes, and comparing the extracted results with registered data around the eyes. The features around the eyes may include the position and shape of the corners and corners of the eyes, the shape of the eyelids, etc. [9-2: Information processing operation performed by the information processing device 9]

[0084] The flow of information processing operations performed by the information processing device 9 in the ninth embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of information processing operations performed by the information processing device 9 in the ninth embodiment. [9-2-1: Learning Operation]

[0085] 16A, the training data acquisition unit 214 acquires first training data including an eye image showing a person's eyes (step S20). In the ninth embodiment, the first training data includes an eye-periphery image including the area around the eye. Note that the first training data used in the ninth embodiment may be the same as the first training data used in the second to eighth embodiments.

[0086] The covering object detection unit 915 detects coverings covering the areas around the eyes from the eye-periphery images included in the first learning data (step S91). The covering object detection unit 915 may detect clothing worn by a person appearing in the eye-periphery images from the eye-periphery images included in the first learning data. The covering object detection unit 915 may detect coverings other than clothing worn by a person appearing in the eye-periphery images from the eye-periphery images included in the first learning data. Coverings other than clothing may include, for example, makeup or other items that cover the area around the eyes. The covering object detection unit 915 may detect factors that affect the features around the eyes from the eye-periphery images. The covering object detection unit 915 may detect conditions around the eyes, such as occlusion or distortion due to physical condition or the like. The covering object detection unit 915 may detect which covering object class CC the eye-periphery image is classified into based on the effect it has on the area around the eyes. Furthermore, the covering object detection unit 915 may not detect as a covering an object that affects the iris but not the features around the eyes. That is, the detection operation by the attachment detection unit 215 and the detection operation by the covering detection unit 915 may be different.

[0087] The training data generation unit 911 generates third training data from the first training data based on the detection result of the covering object detection unit 915 (step S92). In this embodiment, the third training data may be training data generated for learning features of the area around the eyes. The third training data may be training data including desired features related to the covering. The third training data may be training data including desired features related to the area around the eyes. The third training data may particularly be training data generated for learning the effect on the area around the eyes when the area around the eyes is covered by a covering. The training data generation unit 911 may generate the third training data by adjusting the number of objects belonging to each covering object, as described in the second embodiment, for example. Alternatively, the training data generation unit 911 may generate the third training data including classified training data classified for each covering object, as described in the third embodiment, for example.

[0088] The model generation unit 912 generates an eye-periphery authentication model using the third training data (step S93). The model generation unit 912 may generate an eye-periphery authentication model that corresponds to any covering, as described in the second embodiment, for example. Alternatively, the model generation unit 912 may generate an eye-periphery authentication model that is specialized for each covering, as described in the third embodiment, for example. [9-2-2: Authentication Operation]

[0089] As shown in FIG. 16B, the eye-periphery image acquisition unit 916 acquires an eye-periphery image of the subject (step S94). The authentication unit 913 performs iris authentication of the subject using the iris authentication model and the iris region included in the eye-periphery image of the subject (step S95). The authentication unit 913 performs eye-periphery authentication of the subject using the eye-periphery authentication model and the eye-periphery image of the subject (step S96). That is, the authentication unit 913 performs two-factor authentication by performing iris authentication of the subject using the iris authentication model and eye-periphery authentication of the subject using the eye-periphery authentication model. The authentication unit 913 may calculate a first score using the iris authentication model, calculate a second score using the eye-periphery authentication model, and authenticate the subject using the first score and the second score. [9-3: Technical Effects of Information Processing Device 9]

[0090] If the area around the eyes of the subject has changed since the registration data was registered, this may affect the accuracy of authentication. The information processing device 9 in the ninth embodiment learns the area around the eyes to perform eye area authentication with high accuracy even when the condition of the area around the eyes has changed, such as when there is an area around the eyes that is obscured. The information processing device 9 can perform eye area authentication with high accuracy even when there is an area around the eyes that is obscured. Furthermore, the information processing device 9 can perform two-factor authentication, so it can authenticate the subject with higher accuracy. [10: Supplementary Note]

[0091] The following supplementary notes are further disclosed with respect to the above-described embodiments. [Supplementary Note 1] An information processing device comprising: training data generation means for generating second training data based on first training data including eye images of a target's eyes and information on an accessory of the target that appears in the eye images included in the first training data; model generation means for generating an authentication model using the second training data; and authentication means for authenticating the target person using the authentication model. [Supplementary Note 2] The information processing device according to Supplementary Note 1 further comprises: training data acquisition means for acquiring the first training data; and accessory detection means for detecting an accessory of the target that appears in the eye images from the eye images included in the first training data, wherein the training data generation means generates the second training data from the first training data in accordance with a detection result by the accessory detection means. [Supplementary Note 3] The information processing device according to Supplementary Note 2, wherein the training data generation means generates the second training data including categorized training data obtained by classifying the first training data based on detection results by the accessory detection means, based on whether the subject appearing in the eye image is wearing an accessory and, if so, what type of accessory the subject is wearing. [Supplementary Note 4] The information processing device according to Supplementary Note 3, wherein the model generation means generates categorized authentication models according to the classifications using each of the categorized training data. [Supplementary Note 5] The information processing device according to Supplementary Note 4, wherein the accessory detection means detects an accessory worn by the subject from the eye image of the subject, and further comprises authentication model selection means for selecting which of the categorized authentication models the authentication means will use based on detection results by the accessory detection means. [Supplementary Note 6] The information processing device according to Supplementary Note 4, wherein the authentication means authenticates the subject using each of the categorized authentication models, and determines whether or not the subject can be authenticated based on authentication results from each of the categorized authentication models.[Supplementary Note 7] The information processing device according to Supplementary Note 2 further comprises: a classification means for generating categorized registration data by classifying the registration data based on whether or not the subject shown in the eye image is wearing an accessory and, if so, what type of accessory the subject is wearing, from an eye image included in registration data including an eye image showing the subject's eyes detected by the accessory detection means; and a registration data selection means for selecting which of the categorized registration data the authentication means will use based on the accessory of the subject detected from the eye image of the subject by the accessory detection means. [Supplementary Note 8] The information processing device according to Supplementary Note 6, wherein the attachment detection means includes a calculation means that calculates the likelihood that the subject is not wearing attachments and the likelihood that the subject is wearing each type of attachment if the subject is wearing attachments, and calculates each of the weights of the categories according to the likelihoods, and the authentication means authenticates the subject using each of the category authentication models, weights each authentication result using the weight of the category corresponding to the category authentication model, and determines whether or not the subject can be authenticated based on each of the weighted authentication results. [Supplementary Note 9] The information processing device according to Supplementary Note 1 or 2, wherein the authentication model has a function of outputting features of the eye image when the eye image is input, and the model generation means generates the authentication model when eye images of the same subject are input, so as to output features that are similar to or more than a predetermined level to features that would be output if an eye image of the subject not wearing attachments was input, regardless of whether the subject shown in the eye image is wearing attachments. [Supplementary Note 10] The information processing device described in Supplementary Note 2, wherein the first training data includes an eye-periphery image including the area around the eyes, the accessory detection means detects an accessory of a target appearing in the eye-periphery image from the eye-periphery image included in the first training data, the training data generation means generates third training data from the first training data based on the detection result by the accessory detection means, and the model generation means generates an eye-periphery authentication model using the third training data.[Supplementary Note 11] The information processing device according to claim 10, wherein the authentication using the authentication model is iris authentication, and wherein the authentication means performs eye-periphery authentication of the subject using the eye-periphery authentication model in addition to iris authentication of the subject using the authentication model. [Supplementary Note 12] An information processing method comprising: generating second training data based on first training data including eye images of the subject's eyes and information about an article of clothing of the subject that appears in the eye images included in the first training data, generating an authentication model using the second training data, and authenticating the subject using the authentication model. [Supplementary Note 13] A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method comprising: generating second training data based on first training data including eye images of the subject's eyes and information about an article of clothing of the subject that appears in the eye images included in the first training data, generating an authentication model using the second training data, and authenticating the subject using the authentication model.

[0092] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0093] 1, 2, 3, 4, 5, 6, 7, 8, 9 Information processing device 11, 211, 311, 911 Learning data generation unit 12, 212, 312, 812, 912 Model generation unit 13, 213, 313, 413, 513, 613, 713, 813, 913 Authentication unit 214 Learning data acquisition unit 215, 415, 615, 715 Wearing object detection unit 216 Eye image acquisition unit 417 Authentication model selection unit 618 Classification registration data generation unit 619 Classification registration data selection unit 621 Registration data holding unit 7131 Matching unit 7132 Weighting unit 7133 Possibility determination unit 7151 Calculation unit 7152 Decision unit 915 Covering object detection unit 916 Eye surroundings image acquisition unit

Claims

1. a training data generating means for generating second training data based on first training data including an eye image showing the eye of a target and information about an attachment of the target shown in the eye image included in the first training data; a model generation means for generating an authentication model using the second training data; an authentication means for authenticating a subject using the authentication model; An information processing device comprising:

2. learning data acquisition means for acquiring the first learning data; an attachment detection means for detecting an attachment of a target appearing in an eye image included in the first learning data; Further provided with The training data generating means generates the second training data from the first training data in accordance with a detection result by the attachment detecting means. The information processing device according to claim 1 .

3. The learning data generation means generates the second learning data including classification learning data obtained by classifying the first learning data based on the detection result by the accessory detection means, based on whether the object shown in the eye image is wearing an accessory and, if so, what type of accessory the object is wearing. The information processing device according to claim 2 .

4. The model generation means generates each of the classification authentication models according to the classification using each of the classification training data. The information processing device according to claim 3 .

5. the attachment detection means detects an attachment of the subject from the eye image of the subject; The authentication device further includes an authentication model selection unit that selects which of the classification authentication models the authentication device will use based on the detection result by the attachment detection unit. The information processing device according to claim 4 .

6. The authentication means authenticating the subject using each of the classification authentication models; Based on the authentication results of each of the classification authentication models, it is determined whether or not the subject is authenticated. The information processing device according to claim 4 .

7. a classification means for generating classified registration data by classifying the registration data based on the attachment of the subject shown in the eye image detected by the attachment detection means from the eye image included in the registration data including the eye image showing the subject's eyes, whether the subject shown in the eye image is wearing an attachment and, if so, what type of attachment the subject is wearing; and a registration data selection means for selecting which of the categorized registration data the authentication means will use based on the attachment of the subject detected from the eye image of the subject by the attachment detection means. The information processing device according to claim 2 .

8. the attachment detection means includes a calculation means for calculating a likelihood that the subject is not wearing an attachment and a likelihood that the subject is wearing each type of attachment when the subject is wearing an attachment, and calculating each of the weights of the classifications in accordance with the likelihoods; The authentication means authenticating the subject using each of the classification authentication models; weighting each authentication result using the classification weights corresponding to the classification authentication model; Based on each of the weighted authentication results, it is determined whether or not the subject can be authenticated. The information processing device according to claim 6 .

9. generating second learning data based on first learning data including an eye image showing the eye of the target and information about an attachment of the target shown in the eye image included in the first learning data; generating an authentication model using the second training data; Using the authentication model, authenticate the subject. A computer-implemented information processing method.

10. On the computer, generating second learning data based on first learning data including an eye image showing the eye of the target and information about an attachment of the target shown in the eye image included in the first learning data; generating an authentication model using the second training data; Using the authentication model, authenticate the subject. A computer program for executing an information processing method.