Information processing system, information processing device, information processing method, and program

Abstract

An information processing system according to one embodiment comprises an acquisition means for acquiring an image including an iris of a subject, an identification means for identifying the position of a shadow cast on the iris on the basis of the image, and a setting means for setting imaging conditions for the subject on the basis of the position of the shadow.

Description

Information processing system, information processing device, information processing method and program

[0001] The present disclosure relates to an information processing system, an information processing device, an information processing method, and a program.

[0002] Technology related to iris recognition is advancing.

[0003] For example, Patent Literature 1 discloses an information providing device for reducing unnecessary actions and unnecessary processes that may occur in a scene where data of the iris region (hereinafter also referred to as iris data) is acquired. The information providing device determines whether a person depicted in a facial image is unsuitable for acquiring iris data based on the facial image. The information providing device may determine that the subject is unsuitable by detecting an obstacle that partially or entirely obscures the iris region, and detecting that the obstacle is an eyelash and that the degree to which the eyelashes obscure the iris region is unacceptable. Based on the determination result, the information providing device may display a message offering advice for acquiring more appropriate iris data.

[0004] International Publication No. 2020 / 066776

[0005] This disclosure aims to improve upon the techniques described in the prior art documents.

[0006] An information processing system according to one aspect includes: an acquisition means for acquiring a first image including an iris of a subject; an identification means for identifying the position of a shadow on the iris based on the first image; and a setting means for setting photographing conditions for the subject based on the position of the shadow.

[0007] An information processing device according to one aspect includes: an acquisition means for acquiring a first image including an iris of a subject; an identification means for identifying the position of a shadow on the iris based on the first image; and a setting means for setting photographing conditions for the subject based on the position of the shadow.

[0008] An information processing method according to one aspect is an information processing method executed by a computer, comprising: acquiring a first image including an iris of a subject; identifying a position of a shadow on the iris based on the first image; and setting a photographing condition for the subject based on the position of the shadow.

[0009] According to one aspect, the program causes a computer to acquire a first image including an iris of a subject, identify the position of a shadow on the iris based on the first image, and set shooting conditions for the subject based on the position of the shadow.

[0010] 1 is a block diagram showing an example of an information processing system according to the present disclosure. FIG. 2 is a flowchart showing an example of a representative process of the information processing system. FIG. 3 is a block diagram showing an example of an authentication system according to the present disclosure. FIG. 4 is a diagram showing an example of an external appearance of an imaging device. FIG. 5 is a block diagram showing an example of an evaluation unit and a setting unit. FIG. 6 is a diagram showing an example of a partial region of an iris. FIG. 7 is a flowchart showing an example of a representative process of the authentication system. FIG. 8 is a block diagram showing another example of an authentication system according to the present disclosure. FIG. 9 is a flowchart showing an example of a representative process of the authentication system. FIG. 10 is a block diagram showing an example of an evaluation unit. FIG. 11 is a flowchart showing an example of a representative process of the authentication system. FIG. 12 is a block diagram showing an example of a representative process of the authentication system. FIG. 13 is a block diagram showing an example of a hardware configuration of an information processing device in which the process of the system is executed.

[0011] Hereinafter, embodiments of this disclosure will be described with reference to the drawings. Note that the following descriptions and drawings in the embodiments have been omitted or simplified as appropriate for clarity of explanation. Furthermore, in this disclosure, unless otherwise specified, when multiple items are defined as "at least one of multiple items," the definition may mean any one item, or any multiple items including all items.

[0012] Each drawing referenced in the embodiments is merely an example for describing one or more embodiments. Each drawing is not related to only one particular embodiment, but may also be related to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessarily required to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0013] 1 is a block diagram showing an example of an information processing system according to the present disclosure. The information processing system 10 includes an acquisition unit 102, an identification unit 104, and a setting unit 106. The information processing system 10 constitutes, for example, a part of a system that authenticates an object, but the system in which the information processing system 10 is installed is not limited to this.

[0014] The acquisition unit 102 acquires an image including the iris of a target. Here, "target" refers to an object to be imaged, and may be, for example, a human being to be authenticated. The acquisition unit 102 may acquire the image from an imaging means that images the target, or may acquire the image from a storage unit inside the information processing system 10 or an external network. The imaging means is any element capable of performing imaging, or a device including such an element (e.g., a camera). The element is, for example, a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image may be an image captured using visible light, or an image captured using invisible light such as infrared light or ultraviolet light. The image may be a still image or a video.

[0015] The identification unit 104 identifies the position of the shadow on the iris based on the image (hereinafter also referred to as the acquired image) acquired by the acquisition unit 102. Here, identifying the position of the shadow on the iris by the identification unit 104 also includes determining whether or not a shadow is cast on at least one of a predetermined region that is a part of the iris region, or a region of the iris region that occupies a predetermined proportion or more of the iris region. If it is determined that a shadow is cast, the identification unit 104 identifies the position of the shadow in the iris region.

[0016] To identify the position of the shadow on the iris, the identification unit 104 may analyze an image of the iris region in the acquired image, or may analyze an image of a region other than the iris region in the acquired image. When analyzing the image of the iris region in the acquired image, the identification unit 104 may identify the position of the shadow on the iris by comparing images of each of the partial regions into which the iris region is divided. This will be described in detail later.

[0017] The setting unit 106 sets the photographing conditions of the target based on the position of the shadow identified by the identifying unit 104. The photographing conditions of the target to be set include, for example, at least one of the settings of lighting that illuminates the target during photographing, the settings of the imaging means that captured the image acquired by the acquiring unit 102, and guidance for the target, but the photographing conditions to be set are not limited to these.

[0018] Illumination is any light source that irradiates an object with light. The irradiated light may be any light, such as visible light or invisible light such as infrared light or ultraviolet light. Illumination settings include, for example, at least one of the illumination intensity (i.e., light intensity) and the illumination irradiation time, but the set conditions are not limited to these.

[0019] For example, if the imaging means is a camera, the setting of the imaging means may be at least one of the gain, exposure time, and aperture of the camera. By changing this setting, the brightness value of the imaging means is changed.

[0020] Guidance of a target means guiding the target to perform a predetermined action. Guidance can be achieved by visual display on a display unit such as a display or touch panel, or by audio output from a speaker or the like.

[0021] If the identification unit 104 identifies a predetermined area of ​​the iris region or a predetermined percentage or more of the iris region as being shaded, the setting unit 106 sets the imaging conditions to reduce the influence of the shadow on the iris region of the image. Here, "reducing the influence of the shadow" includes reducing the area of ​​the shadow, eliminating the shadow on the iris region (i.e., reducing the area of ​​the shadow to zero), and reducing the difference in shading between the shadowed and non-shadowed areas of the iris region. For example, if a predetermined area of ​​iris data suitable for iris authentication is shaded in the acquired image, the setting unit 106 sets the imaging conditions to reduce the influence of the shadow on that predetermined area. Examples of setting the imaging conditions include increasing or decreasing the illumination intensity of the lighting, increasing or decreasing the luminance value of the imaging means, or guiding the person to perform a predetermined action to reduce the area of ​​the shadow on the iris of the target or to eliminate the shadow. In addition, the difference in shading between the shadows caused by lighting (such as ceiling lights or sunlight) other than the lighting provided by the acquisition unit 102 and the areas illuminated by light can be reduced by increasing the intensity of the lighting provided by the acquisition unit 102 or by reducing the exposure of the camera. Examples of the predetermined action include changing at least one of the position or orientation of the subject's face (for example, facing the face directly toward the imaging means), removing glasses, etc. Details of this will be described later.

[0022] If the specifying unit 104 specifies that there is no shadow on the iris, the setting unit 106 does not need to change the photographing conditions.

[0023] The image acquired by the acquisition unit 102 may be used for authentication, etc. The imaging unit may be included in the information processing system 10 or may be provided outside the information processing system 10. The lighting, display unit, and speaker may also be included in the information processing system 10 or may be provided outside the information processing system 10.

[0024] [Explanation of Processing Flow] Fig. 2 is a flowchart showing an example of a typical process of the information processing system 10, and the flowchart in Fig. 2 explains an overview of the process of the information processing system 10. Note that the details of each process are as described above, and therefore will not be explained as appropriate.

[0025] First, the acquisition unit 102 acquires an image including the iris of the subject (step S12). The identification unit 104 identifies the position of a shadow on the iris based on the acquired image (step S14). The setting unit 106 sets the photographing conditions for the subject based on the identified position of the shadow (step S16).

[0026] [Explanation of Effect] When an iris is photographed, even if the entire iris region is captured in the photographed image, if a shadow is cast over part or all of the iris region, the shadowed area will appear black in the image. In this case, the iris data will be substantially damaged, which may make it difficult to use the iris data for a desired purpose (for example, iris authentication).

[0027] The information providing device disclosed in Patent Document 1 displays a message offering advice on how to acquire more appropriate iris data when the iris area of ​​a person is hidden. However, even when the iris area is not hidden, it may be difficult to acquire appropriate iris data due to, for example, a shadow cast by light on the iris surface.

[0028] However, as described above, the information processing system 10 identifies the position of the shadow on the iris and sets the photographing conditions of the subject based on the position of the shadow. This makes it possible to capture an image in which the influence of the shadow on the iris area is reduced. Therefore, the information processing system 10 can assist in acquiring more appropriate iris data even when a shadow is cast on the iris. However, the effects of the system according to the present disclosure are not limited to this.

[0029] The information processing system 10 is applicable not only to iris authentication of a person, but also to authentication of a person's face, fingerprint, or palm print. In this case, the acquisition unit 102 acquires an image including the target's face, fingerprint, or palm print. The identification unit 104 identifies the position of a shadow cast on the face, fingerprint, or palm print based on the acquired image. The setting unit 106 sets the target's photographing conditions based on the identified position of the shadow. Details of the processing performed by each unit are as described above.

[0030] For example, in face authentication, if a shadow from an accessory such as a hat or eyelashes casts on the face, it may be difficult to perform face authentication based on the captured image even if the photographing device captures the face. Similarly, in fingerprint or palm print authentication, if a shadow from a nail or scratch on the finger, or from dirt between the photographing device and the hand casts on the finger or palm, it may be difficult to perform authentication based on the captured image even if the photographing device captures the finger or palm. The reasons for this difficulty are similar to those described above. Even in such cases, the information processing system 10 can assist in obtaining more appropriate face, fingerprint, or palm print data.

[0031] The information processing system 10 may be configured as a single computer device (information processing device) or as a distributed system having multiple computer devices. In a distributed system, the processing performed by the information processing system 10 can be shared and executed by multiple computer devices. In other words, the information processing system 10 may be realized by distributing the acquisition unit 102 to the setting unit 106 across two or more computer devices, and enabling the two or more computer devices to communicate with each other.

[0032] Some or all of the components of the information processing system 10 may be provided on a cloud server built on a cloud, or on other types of virtualized servers generated using virtualization technology, etc. Functions other than those provided on servers such as cloud servers or virtualized servers are placed on edges. For example, in a system that acquires video footage captured on-site via a network and monitors the footage, edges are devices placed in a specific area or near the site, and are also devices that are close to the terminals in terms of the network hierarchy.

[0033] In each of the following embodiments, a specific example of the information processing system 10 described in embodiment 1 will be disclosed. However, the specific example of the information processing system 10 described in embodiment 1 is not limited to the one shown below. Furthermore, the configurations and processes described below are examples and are not limited to these.

[0034] Embodiment 2 (2A) [Configuration Description] Fig. 3 is a block diagram showing an example of an authentication system according to this disclosure. The authentication system 20 may be used for security checks at airports, laboratories, etc., for payments, and for user authentication at machines such as ATMs (automatic teller machines). The authentication system 20 is provided as a multimodal authentication device that performs multiple biometric authentication methods.

[0035] The authentication system 20 is connected to an imaging device 30 that captures an image of a person to be authenticated. The imaging device 30 has an infrared camera 302, an infrared illuminator 304, and a display unit 306. The infrared camera 302 is a camera capable of capturing images in the near-infrared band, and captures an image of the person to be authenticated by capturing an image of a predetermined spatial region. The image captured by the infrared camera 302 is output to the authentication system 20. The infrared illuminator 304 illuminates the predetermined spatial region captured by the infrared camera 302 with near-infrared light (e.g., a wavelength of approximately 850 nm). This allows the infrared camera 302 to capture an image that enables detection or authentication of the person. The display unit 306 displays a screen that guides the person to perform a predetermined action.

[0036] It should be noted that only one infrared illuminator 304 may be provided, or multiple infrared illuminators 304 may be provided. For example, the infrared illuminator 304 may include an infrared illuminator (infrared illuminator 304R) that illuminates the person to be imaged from the right direction as viewed from the imaging device 30, and an infrared illuminator (infrared illuminator 304L) that illuminates the person to be imaged from the left direction as viewed from the imaging device 30. As another example, the infrared illuminator 304 may include an infrared illuminator (infrared illuminator 304U) that illuminates the person to be imaged from above as viewed from the imaging device 30, and an infrared illuminator (infrared illuminator 304D) that illuminates the person to be imaged from below as viewed from the imaging device 30. Furthermore, three or more infrared illuminators 304 may be provided.

[0037] 4 shows an example of the appearance of the imaging device 30. An infrared illuminator 304U is provided at the top of the imaging device 30 to illuminate the person T (particularly the face of the person T) to be imaged from above as viewed from the imaging device 30, and an infrared illuminator 304D is provided at the bottom of the imaging device 30 to illuminate the person T from below as viewed from the imaging device 30. An infrared illuminator 304R is provided at the center of the imaging device 30 to illuminate the person T from the right as viewed from the imaging device 30, and an infrared illuminator 304L is provided at the center of the imaging device 30 to illuminate the person T from the left as viewed from the imaging device 30. An infrared camera 302 and a display unit 306 are also provided at the center of the imaging device 30. The person T can view the screen of the display unit 306 and perform predetermined operations.

[0038] The illumination intensity of the infrared illuminator 304 is set by a setting unit 214, which will be described later. When a plurality of infrared illuminators 304 are provided, the illumination intensity is set individually for each infrared illuminator 304 by the setting unit 214.

[0039] 3, the authentication system 20 includes a storage unit 202, an acquisition unit 204, a face detection unit 206, a face authentication unit 208, an iris authentication unit 210, an evaluation unit 212, and a setting unit 214. Each unit of the authentication system 20 will be described below.

[0040] The storage unit 202 stores information on facial features and iris features that can identify a user to be authenticated. The facial feature information is information used for face authentication, and for example, the positions of feature points such as the eyes, nose, and mouth in a face image and the distance between feature points are defined as "features." The iris feature information is information used for iris authentication, and is, for example, information on the iris pattern. The storage unit 202 stores information on iris features for each of the left and right eyes. However, information on only one eye may be stored as iris features. The storage unit 202 may store information on the facial features and iris features of each person. The storage unit 202 may also store information indicating the position and intensity of the infrared light 304.

[0041] The acquisition unit 204 acquires an image (hereinafter also referred to as an acquired image) captured by the infrared camera 302. The acquired image includes, for example, the face of the person to be authenticated.

[0042] The face detection unit 206 analyzes the acquired image to detect whether or not a human face is present in the image. Furthermore, when the face detection unit 206 detects a human face image (i.e., information related to a face), it also detects the shape of the person's eyes in the image.

[0043] When the face detection unit 206 detects a person's face in the acquired image, the face authentication unit 208 performs face authentication of the person. Specifically, the face authentication unit 208 calculates information on the person's facial features based on the person's facial image. The face authentication unit 208 compares the calculated information on the facial features with the information on the facial features stored in the storage unit 202.

[0044] For example, the face authentication unit 208 calculates the similarity between the calculated feature amount and the facial feature amount stored in the storage unit 202. If there is a facial feature amount on the storage unit 202 side with a similarity greater than a threshold, the face authentication unit 208 determines that face authentication has been successful. Furthermore, if the feature amount is a feature vector, the face authentication unit 208 calculates the distance between the calculated feature vector and the facial feature vector on the storage unit 202 side. If there is a facial feature vector on the storage unit 202 side with a distance smaller than the threshold, the face authentication unit 208 determines that the feature vectors match and that face authentication has been successful. On the other hand, if there is no facial feature amount on the storage unit 202 side with a similarity greater than the threshold or if there is no facial feature vector on the storage unit 202 side with a distance smaller than the threshold, the face authentication unit 208 determines that face authentication has failed.

[0045] The iris authentication unit 210 detects the irises of the eyes in the image acquired by the acquisition unit 204. Then, the iris authentication unit 210 performs iris authentication on the detected irises. In detail, the iris authentication unit 210 calculates iris feature information based on the images of the right and left eyes (i.e., information about the eyes) captured in the captured image. The iris authentication unit 210 compares the calculated iris feature information with the iris feature information stored in the storage unit 202 for each of the left and right eyes. Note that the iris authentication unit 210 may compare the iris feature information of a person whose face authentication has been determined to be successful by the face authentication unit 208, among the iris feature information stored in the storage unit 202, with the calculated iris feature information.

[0046] For example, the iris authentication unit 210 calculates the similarity between the calculated feature amount and the iris feature amount stored in the storage unit 202 for each of the left and right eyes. If there are iris feature amounts on the storage unit 202 for both the left and right eyes that have a similarity greater than a threshold, the iris authentication unit 210 determines that iris authentication has been successful. Furthermore, if the feature amount is a feature vector, the iris authentication unit 210 calculates the distance between the calculated feature vector and the iris feature vector on the storage unit 202 for each of the left and right eyes. If there are iris feature vectors on the storage unit 202 for both the left and right eyes that have a distance smaller than the threshold, the iris authentication unit 210 determines that the feature vectors match and that iris authentication has been successful. On the other hand, if there are no iris feature amounts on the storage unit 202 for at least either the left or right eyes that have a similarity greater than the threshold, or if there are no iris feature vectors on the storage unit 202 for which the distance is smaller than the threshold, the iris authentication unit 210 determines that iris authentication has failed.

[0047] 5 is a block diagram showing an example of the evaluation unit 212 and the setting unit 214. The evaluation unit 212 has a shadow detection unit 222 and a factor estimation unit 224. The setting unit 214 has a hardware control unit 232 and a UI control unit 234. Each unit will be described below.

[0048] The shadow detection unit 222 detects the position of a shadow cast on the iris in the image by analyzing the acquired image. Here, detecting the position of a shadow cast on the iris by the shadow detection unit 222 also includes determining whether or not a shadow is cast on at least a portion of the iris region. If a shadow is cast on at least a portion of the iris region, the shadow detection unit 222 identifies the position of the shadow in the iris region.

[0049] Specifically, the shadow detection unit 222 identifies the position of the shadow on the iris by comparing images of each of the partial regions obtained by dividing the iris region into a plurality of regions. The partial regions are set using at least one of the following methods.

[0050] The shadow detection unit 222 may change the number of partial regions depending on the number of illuminators. For example, if there are two infrared illuminators 304, the iris region may be divided into two and two partial regions may be provided. If there are four infrared illuminators 304, the iris region may be divided into four and four partial regions may be provided. In this way, the number of partial regions may increase as the number of infrared illuminators 304 increases. However, the number of infrared illuminators 304 and the number of partial regions do not have to be the same.

[0051] The shadow detection unit 222 may change the division method of the partial regions (i.e., the position of the partial region) depending on the position of the illumination. For example, when the above-described infrared illumination 304R and infrared illumination 304L are provided as the infrared illumination 304, the shadow detection unit 222 may divide the iris region into a right half partial region and a left half partial region. When the above-described infrared illumination 304U and infrared illumination 304D are provided as the infrared illumination 304, the shadow detection unit 222 may divide the iris region into an upper half partial region and a lower half partial region.

[0052] FIG. 6 is a diagram illustrating an example of iris partial regions. FIG. 6 shows a person's eye and its surrounding region E (particularly, iris region I) as viewed from the imaging device 30. In FIG. 6, the iris region I is divided into four regions based on its center O. In FIG. 6, the partial regions include region U located above the iris region I, region D located below, region R located to the right, and region L located to the left. The number of partial regions and the division method correspond to the infrared illuminators 304RU, 304LU, 304RD, and 304LD shown in FIG. 4. In other words, each infrared illuminator shown in FIG. 4 specifically illuminates each partial region in FIG. 6. The sizes of the partial regions obtained by dividing the iris region may be different from each other or may be approximately the same. In this way, the shadow detection unit 222 may change the division method for the partial regions depending on the number and positions of the infrared illuminators 304.

[0053] As another example, when infrared illuminators 304R and 304L are provided as infrared illuminators 304, iris region I may be divided into two, upper and lower, with center O as the reference point. Furthermore, when infrared illuminators 304U and 304D are provided as infrared illuminators 304, iris region I may be divided into two, left and right, with center O as the reference point.

[0054] The shadow detection unit 222 may change at least one of the number or position of partial regions or the size of each partial region, depending on the shape of the person's eyes detected by the face detection unit 206. For example, if the person's eyes are not open (area) closely (i.e., the eyes are narrow), when the iris region I is divided into four parts with the center O as the reference, the regions U and D in FIG. 6 will be smaller than the regions L and R. Therefore, the shadow detection unit 222 sets the regions U and D to be larger than the original regions, and the regions L and R to be smaller than the original regions.

[0055] However, the shadow detection unit 222 may use a method other than the above to detect a shadow in the iris region (for example, a method by which the shadow detection unit 222 sets a partial region).

[0056] The shadow detection unit 222 performs image quality evaluation on the image of each partial region set as described above. For example, the shadow detection unit 222 calculates the luminance of each pixel in the image of each partial region. Then, based on the calculated luminance, it calculates the contrast of each partial region. By comparing the calculated contrast of each partial region, it determines whether there is a particularly dark area among the multiple partial regions. For example, if there is a partial region with the highest contrast ratio and the contrast ratio of that partial region is higher than a predetermined value, the shadow detection unit 222 determines that that partial region is shaded. In this case, the shadow detection unit 222 may further determine where the shadow is cast in that partial region based on the luminance of each pixel in that partial region. Furthermore, if the shadow detection unit 222 determines that a shadow is cast in a certain partial region, it may calculate the density (i.e., light intensity) of the shadow based on the luminance of the shaded portion. In this manner, the shadow detection unit 222 determines the position of the shadow cast in the iris region.

[0057] Returning to FIG. 5 , the explanation continues. The factor estimation unit 224 estimates the cause of the shadow based on the position of the shadow detected by the shadow detection unit 222. For example, if the shadow detection unit 222 detects that a shadow is located in region U in FIG. 6 , the factor estimation unit 224 may estimate at least one of the following as the cause of the shadow: The position of the person's face is below a predetermined height; The orientation of the person's face is tilted at a predetermined angle or more with respect to the vertical direction (i.e., the person is looking down); The presence of the person's eyeglass frames; The illumination intensity of the infrared illuminators 304U and 304D, or the ratio between the illumination intensities of the two; and The exposure value of the infrared camera 302 is above a predetermined value. For example, if the position of the person's face is below a predetermined height, it is possible that a shadow of the upper eyelid will be cast on the upper region of the person's iris due to illumination from above. The exposure value is a value indicating the degree of exposure, determined based on the aperture value and exposure time of the infrared camera 302. When the exposure value of the infrared camera 302 is equal to or greater than a predetermined value, the influence of external light other than that from the infrared illumination 304 becomes relatively high in the image, which may cause a shadow to appear on the iris of the person. Note that the cause estimation unit 224 may estimate the cause of the shadow based not only on the position of the shadow but also on the detailed shape of the shadow.

[0058] The factor estimation unit 224 may estimate the cause of the shadow not only by using the information on the position of the shadow but also by using at least one of information on the position, orientation, or shape of the person's face, the presence or absence or shape of an accessory worn by the person, the position or intensity of the infrared illuminator 304 illuminating the person, or the luminance value of the infrared camera 302 that captured the acquired image. The cause of the shadow is at least one of the position, orientation, or shape of the person's face, the shape of an accessory worn by the person, or the position or intensity of the infrared illuminator 304 illuminating the person. An accessory is, for example, an item worn on the face or head, and includes glasses, a mask, a hat, etc. The factor estimation unit 224 determines the position, orientation, or shape of the person's face and the presence or shape of an accessory worn by the person based on the acquired image.

[0059] For example, the factor estimation unit 224 can estimate how the infrared illuminator 304 will illuminate the iris of the person, based on information on the position of the shadow, the position, orientation, and shape of the person's face, and the position of the infrared illuminator 304. Based on the result of this estimation and the actually measured position of the shadow, the factor estimation unit 224 may estimate at least one of the following as the cause of the shadow. - The position of the person's face is significantly deviated from the front of the infrared illuminator 304, which is the light source, or the position of the infrared illuminator 304 is significantly deviated from the front of the person (for example, when the normal line from the image capture device 30 facing the front direction of the infrared illuminator 304 is taken as the reference line, the person's face is not within a range of a predetermined angle from the reference line in the horizontal direction). - The direction of the person's face is significantly deviated from the direction of the infrared illuminator 304 (for example, the normal line from the person's face facing the front direction forms an angle of a predetermined angle or more with the normal line from the image capture device 30). - The shape of the face is likely to produce shadows (for example, the person has characteristics such as deeply defined eyes or thick eyelids). - The shape of the accessory worn by the person is likely to block the light from the light source.

[0060] Furthermore, the factor estimation unit 224 may further use information on the density of the shadow in addition to the above information. As a result, the factor estimation unit 224 may estimate, for example, the illumination intensity of the infrared illumination unit 304 as the cause of the shadow. For example, when the density of the shadow cast on the iris region is high, i.e., when the difference in luminance between the iris region in the shadow region and the iris region other than the shadow region is equal to or greater than a predetermined value, the factor estimation unit 224 estimates the illumination intensity of the infrared illumination unit 304 as the cause of the shadow. In this way, the factor estimation unit 224 can identify the intensity of the infrared illumination unit 304 being too strong or too weak (for example, the intensity being equal to or greater than a predetermined value or equal to or less than a predetermined value) as the cause of the shadow.

[0061] Furthermore, the factor estimation unit 224 may use the acquired image in addition to the above information to determine whether the person is wearing eyeglasses with frames that are equal to or wider than a predetermined width. Then, the factor estimation unit 224 may estimate that the eyeglass frames are the cause of the shadow based on the determination result and the actually measured position of the shadow.

[0062] The setting unit 214 reduces the area of ​​the shadow falling on the iris region by setting the photographing conditions for the person based on the shadow generation factor estimated by the factor estimation unit 224. The photographing conditions include at least one of the setting of the infrared illuminator 304 that illuminates the person, the setting of the infrared illuminator 304 that captured the person, and guidance for the person.

[0063] The hardware control unit 232 controls the shooting conditions related to the hardware of the image capture device 30 (i.e., the shooting conditions of at least one of the infrared camera 302 and the infrared illuminator 304) based on the settings of the evaluation unit 212. Below, specific examples of factors that cause shadows and shooting conditions that are set accordingly will be described.

[0064] For example, assume that infrared illuminators 304U and 304D are provided as the infrared illuminators 304, and that infrared illuminator 304D is turned on while infrared illuminator 304U is turned off when capturing an acquired image. If the cause estimation unit 224 estimates that the high position of the person's face is the cause of the shadow, the hardware control unit 232 controls the infrared illuminators 304 so that not only infrared illuminator 304D but also infrared illuminator 304U is turned on when capturing an acquired image. Alternatively, if the illumination intensity of infrared illuminator 304U when capturing an acquired image is equal to or lower than a predetermined threshold, the hardware control unit 232 controls the infrared illuminator 304 so that the illumination intensity of infrared illuminator 304U is increased above the predetermined threshold.

[0065] Assume that the cause estimation unit 224 estimates that the cause of the shadow is that the infrared illuminator 304U is turned on while the infrared illuminator 304D is turned off. Here, the hardware control unit 232 controls the infrared illuminators 304 so that not only the infrared illuminator 304U but also the infrared illuminator 304D is turned on during image capture.

[0066] When the cause estimation unit 224 estimates that the illumination intensity of the infrared illuminator 304 is the cause of the shadow, the hardware control unit 232 controls the infrared camera 302 and the infrared illuminator 304. For example, assume that the illumination intensity of the infrared illuminator 304 is estimated to be equal to or greater than a predetermined value as the cause of the shadow. In this case, the hardware control unit 232 sets the illumination intensity of the infrared illuminator 304 to be reduced.

[0067] Meanwhile, the UI control unit 234 controls the display unit 306 to display a display screen that guides the person to perform a predetermined action. For example, assume that the cause estimation unit 224 estimates that the person is wearing glasses with frames that are at least a predetermined width as a cause of the shadow. Here, the UI control unit 234 causes the display unit 306 to display information that prompts the person to at least one of change the direction of their face or remove their glasses.

[0068] Furthermore, the controls of the hardware control unit 232 and the UI control unit 234 may be executed in combination. For example, assume that the cause estimation unit 224 estimates that the cause of a shadow is that the infrared illuminator 304U is on, the infrared illuminator 304D is off, and the person is wearing eyeglasses with frames wider than a predetermined width. In this case, the hardware control unit 232 controls the infrared illuminators 304 so that not only the infrared illuminator 304U but also the infrared illuminator 304D is turned on during image capture. Furthermore, the UI control unit 234 causes the display unit 306 to display information that prompts the person to at least one of change the direction of their face or remove their eyeglasses.

[0069] The factor estimation unit 224 can estimate the cause of a shadow by using, for example, an arbitrary algorithm or a pre-trained AI (artificial intelligence) model. The AI ​​model is trained by inputting training data including sample information and a correct answer label indicating the cause of the shadow corresponding to the sample information. The sample information may include information on the position of the shadow, or may include not only the position of the shadow but also information on the density of the shadow, or part or all of the acquired image. The sample information may also include at least one of the position, orientation, or shape of the person's face, the shape of an accessory worn by the person, the position or intensity of the infrared light 304 illuminating the person, or the luminance value of the infrared camera 302. When the factor estimation unit 224 acquires information such as the position or density of the shadow detected by the shadow detection unit 222, the factor estimation unit 224 inputs the information, or the information and part or all of the acquired image, to the AI ​​model trained using the training data. The AI ​​model outputs the estimated cause of the shadow based on the input information.

[0070] The setting unit 214 can also set the photographing conditions for a person by using, for example, an arbitrary algorithm or a pre-trained AI model. The AI ​​model is trained by inputting training data including sample information indicating the causes of shadow generation and correct labels indicating the photographing conditions corresponding to the sample information into the AI ​​model. The setting unit 214 inputs the causes of shadow generation estimated by the factor estimation unit 224 to the AI ​​model trained using the training data. The AI ​​model outputs the photographing conditions to be set based on the input information.

[0071] Any technique such as logistic regression or neural network can be used as a learning method for the AI ​​model used by the factor estimation unit 224 or the setting unit 214.

[0072] After the setting unit 214 sets the photographing conditions for the person, the setting unit 214 may output an instruction to the image capturing device 30 to capture an image again. In response to the instruction, the infrared camera 302 captures an image of the person under the newly set photographing conditions. The captured image of the person is transmitted to the authentication system 20. The acquisition unit 204 acquires the transmitted image.

[0073] The iris authentication unit 210 detects the iris of the eye in the image acquired the second time. Then, the iris authentication unit 210 performs iris authentication on the detected iris of the eye. The details of this are as described above, so a detailed explanation will be omitted.

[0074] 7A and 7B are flowcharts showing an example of a typical process of the authentication system 20, and these flowcharts provide an overview of the process of the authentication system 20. Note that the details of each process are as described above, and therefore will not be described again as appropriate.

[0075] First, the acquisition unit 204 acquires an image (hereinafter also referred to as the first image) captured by the infrared camera 302 under first shooting conditions (step S22). The face detection unit 206 analyzes the first image to determine whether a person's face image is detected in the first image (step S24). If a person's face image is not detected (No in step S24), the face detection unit 206 instructs the image capture device 30 to capture an image again. In response to this instruction, the infrared camera 302 captures an image again under the first shooting conditions. Thereafter, the processing from step S22 onwards is performed on the captured image.

[0076] If a person's face image is detected in the first image (Yes in step S24), the face authentication unit 208 performs face authentication on the detected face image (step S26). The face authentication unit 208 determines whether face authentication is successful (step S28). If face authentication is unsuccessful (No in step S28), the face authentication unit 208 instructs the image capture device 30 to capture an image again. In response to this instruction, the infrared camera 302 captures an image again under the first shooting conditions. Thereafter, the processing from step S22 onwards is executed on the captured image.

[0077] If the face authentication is successful (Yes in step S28), the evaluation unit 212 evaluates the first image. First, the shadow detection unit 222 analyzes the first image to determine whether or not a shadow is cast on at least a part of the iris region in the first image (step S30).

[0078] If a shadow is cast on at least a part of the iris region (Yes in step S30), the factor estimation unit 224 estimates the cause of the shadow based on the position of the shadow detected by the shadow detection unit 222 (step S32). The setting unit 214 sets the photographing conditions for the person based on the cause of the shadow estimated by the factor estimation unit 224 so as to reduce the influence of the shadow (for example, so as to reduce the area of ​​the iris region that the shadow casts) (step S34). In other words, the photographing conditions for the person are updated from the first photographing conditions to the second photographing conditions.

[0079] The setting unit 214 outputs an instruction to the image capturing device 30 to capture an image again under the second capturing conditions set in step S34. In response to the instruction, the infrared camera 302 captures an image of the person under the second capturing conditions. The captured image (hereinafter also referred to as the second image) is transmitted to the authentication system 20. The acquisition unit 204 acquires the second image (step S36).

[0080] The iris authentication unit 210 detects the iris of the eye in the second image. Then, the iris authentication unit 210 performs iris authentication on the detected iris of the eye (step S38). The iris authentication unit 210 determines whether the performed iris authentication is successful (step S40).

[0081] If it is determined that the iris authentication has failed (No in step S40), the iris authentication unit 210 instructs the image capture device 30 to capture an image again. In response to this instruction, the infrared camera 302 captures an image again under the second shooting conditions. Thereafter, the processing from step S36 onwards is executed for the captured image. On the other hand, if the iris authentication has succeeded (Yes in step S40), the authentication system 20 ends the processing.

[0082] If it is determined in step S30 that the iris region is not shaded (No in step S30), the iris authentication unit 210 detects the iris of the eye in the first image. Then, the iris authentication unit 210 performs iris authentication on the detected iris of the eye (step S38). The iris authentication unit 210 determines whether the performed iris authentication was successful (step S40). If it is determined that the iris authentication failed (No in step S40), the iris authentication unit 210 instructs the image capture device 30 to capture an image again. In response to the instruction, the infrared camera 302 captures an image again under the first shooting conditions. Subsequently, the processing from step S36 onward is performed on the captured image. On the other hand, if the iris authentication was successful (Yes in step S40), the authentication system 20 ends the processing.

[0083] If face authentication fails in step S28, the face authentication unit 208 may perform face authentication again using the second image obtained in step S36 without instructing the imaging device 30 to re-capture the image.

[0084] Furthermore, the face authentication unit 208 may perform face authentication on the face image in the second image, rather than on the face image in the first image. In this case, the face authentication unit 208 does not perform the processes of steps S26 and S28, and performs face authentication using the second image after step S36. The face authentication unit 208 may perform face authentication first, or the iris authentication unit 210 may perform iris authentication first, or face authentication and iris authentication may be performed simultaneously. If both face authentication and iris authentication are successful, the authentication system 20 ends the processing.

[0085] [Explanation of Effects] As described above, even if a shadow is cast on part of the iris region in the captured first image, the authentication system 20 identifies the position of the shadow and updates the photographing conditions of the person based on the position of the shadow. This is thought to reduce the influence of the shadow on the iris region in the second image that is captured again. Therefore, the authentication system 20 makes it easier to perform iris authentication successfully (i.e., it can efficiently obtain high-quality iris images), thereby shortening the time required for authentication.

[0086] Furthermore, the iris authentication unit 210 may perform iris authentication of a person using a second image captured after the photographing conditions are set. This allows the authentication system 20 to use an image captured under photographing conditions that make iris authentication more likely to be successful, thereby enabling highly accurate iris authentication.

[0087] Furthermore, the face authentication unit 208 may perform face authentication of a person using the first image or the second image. This allows the authentication system 20 to perform multimodal authentication. By performing multimodal authentication, the authentication system 20 can perform authentication with higher accuracy than authentication using only the face or iris.

[0088] Furthermore, the shadow detection unit 222 may identify the position of the shadow by comparing images of each partial region in the first image. This allows the authentication system 20 to improve the accuracy of shadow detection compared to when the iris region is analyzed without dividing it. For example, when the iris region is divided into top, bottom, left, and right, the authentication system 20 can more accurately detect in which direction (top, bottom, left, or right) the shadow is occurring in the iris region compared to when the iris region is analyzed without dividing it. Therefore, the authentication system 20 can set shooting conditions that make iris authentication more likely to be successful.

[0089] Furthermore, the shadow detection unit 222 may set a partial region according to at least one of the number and the position of the infrared illuminators 304 that illuminate the person when capturing the first image. This allows the authentication system 20 to perform analysis taking into account the illumination when detecting the position of the shadow, thereby enabling more accurate detection of the position of the shadow.

[0090] The shadow detection unit 222 may also set a partial region according to the shape of the person's eyes, which allows the authentication system 20 to perform analysis taking into account the characteristics of the person when detecting the position of the shadow, thereby enabling more accurate detection of the position of the shadow.

[0091] Here, the shape of the person's eyes may be identified based on an image of the person's face. The first image or another image showing the person's face may be used to identify the position of the person's eyes. This allows the authentication system 20 to accurately identify the shape of the eyes, thereby enabling more accurate detection of the position of the shadow.

[0092] Furthermore, the setting unit 214 may identify the cause of the shadow according to the position of the shadow and set the photographing conditions for the person based on the identified cause of the shadow. This allows the authentication system 20 to set the photographing conditions based on the cause of the shadow, rather than simply setting the photographing conditions, making it easier to achieve successful iris authentication.

[0093] Here, the factor estimation unit 224 may identify the cause of the shadow by further using at least one of the density of the shadow, the position, orientation, or shape of the person's face, the shape of an accessory worn by the person, the position or intensity of the infrared light 304 that illuminates the person, or the luminance value of the infrared camera 302 that captured the acquired image. This enables the authentication system 20 to more accurately identify the cause of the shadow, thereby making it possible to set more appropriate shooting conditions for acquiring an image that will enable successful iris authentication.

[0094] Generally, infrared cameras capable of capturing both images for face authentication and images for iris authentication have a wide angle of view. Therefore, by using face images captured with a wide-angle infrared camera, the authentication system 20 can obtain information other than that surrounding the eye area, such as the position, shape, and orientation of the face, and the presence or shape of any accessories. Therefore, the authentication system 20 can more easily identify the causes of the shadows that reduce the quality of the iris, thereby setting more appropriate shooting conditions and increasing the possibility of improving the quality of the image for iris authentication.

[0095] Furthermore, the setting unit 214 may set, as the photographing conditions, at least one of the settings of the infrared illuminator 304 that illuminates the person, the settings of the image capture device 30, and guidance for the person. This allows the authentication system 20 to apply an effective method for acquiring an image that will result in successful iris authentication.

[0096] (2B) [Configuration] The following describes a variation of (2A). The points already explained in (2A) will be omitted as appropriate, and the description will focus on the configuration and processing unique to this variation.

[0097] 8 is a block diagram showing another example of an authentication system according to this disclosure. In FIG. 8, the authentication system 20 includes a multimodal authentication unit 216 instead of the face authentication unit 208 and the iris authentication unit 210.

[0098] The multimodal authentication unit 216 executes processing that integrates the functions of the face authentication unit 208 and the iris authentication unit 210. Here, the multimodal authentication unit 216 calculates a numerical value related to face authentication for the face of a person appearing in the first image (or the second image). Furthermore, when the multimodal authentication unit 216 detects the iris of an eye in the second image, it calculates a numerical value related to iris authentication for the iris of the eye appearing in the second image.

[0099] For example, the multimodal authentication unit 216 may calculate information about the facial features of a person based on the facial image in the first image, or may calculate information about the iris features based on the images of the right and left eyes in the second image.

[0100] The multimodal authentication unit 216 calculates the following values ​​for each person stored in the storage unit 202: (A) the similarity between the facial feature values ​​of the person stored in the storage unit 202 and the calculated facial feature values; (B) the similarity between the right eye feature values ​​of the person stored in the storage unit 202 and the calculated right eye feature values; and (C) the similarity between the left eye feature values ​​of the person stored in the storage unit 202 and the calculated left eye feature values. The multimodal authentication unit 216 sums the similarities of (A), (B), and (C) and compares the sum with a predetermined threshold for each person stored in the storage unit 202. Note that the multimodal authentication unit 216 may multiply at least one of the values ​​of (A), (B), or (C) by a predetermined coefficient. The multimodal authentication unit 216 calculates a total value by adding together the values ​​multiplied by the coefficients and other values ​​not multiplied by the coefficients.

[0101] If any of the total values ​​is equal to or greater than a predetermined threshold, the multimodal authentication unit 216 determines that the person associated with the total value equal to or greater than the predetermined threshold has been photographed, and determines that multimodal authentication has been successful. On the other hand, if no of the total values ​​is equal to or greater than the predetermined threshold, the multimodal authentication unit 216 determines that multimodal authentication has failed.

[0102] As another example, the multimodal authentication unit 216 may calculate information about a feature vector of a person's face based on the face image in the first image, and calculate information about a feature vector of the person's iris based on the images of the right and left eyes in the second image.

[0103] The multimodal authentication unit 216 calculates the following values ​​for each person stored in the storage unit 202: (A') The distance between the person's facial feature vector stored in the storage unit 202 and the calculated facial feature vector; (B') The distance between the person's right eye feature vector stored in the storage unit 202 and the calculated right eye feature vector; and (C') The distance between the person's left eye feature vector stored in the storage unit 202 and the calculated left eye feature vector. The multimodal authentication unit 216 sums the distances (A'), (B'), and (C') and compares the summed distance with a predetermined threshold for each person stored in the storage unit 202. Note that the multimodal authentication unit 216 may multiply at least one of the distances (A'), (B'), or (C') by a predetermined coefficient. The multimodal authentication unit 216 calculates a total distance by adding together the distances multiplied by the coefficient and other distances not multiplied by the coefficient.

[0104] If any of the total distances is less than the predetermined threshold, the multimodal authentication unit 216 determines that a person associated with the total distance less than the predetermined threshold has been photographed, and determines that multimodal authentication has been successful. On the other hand, if no of the total distances is less than the predetermined threshold, the multimodal authentication unit 216 determines that multimodal authentication has failed.

[0105] The multimodal authentication unit 216 may perform the calculation process as follows. For example, assume that the multimodal authentication unit 216 calculates one or two of the numerical values ​​(A), (B), or (C) for each person stored in the storage unit 202. In this case, the multimodal authentication unit 216 compares a first sum, which is the sum of the calculated values, with a predetermined first threshold value as the first comparison process. When the multimodal authentication unit 216 calculates and sums two of the numerical values ​​(A), (B), or (C), it may multiply one of the numerical values ​​by a predetermined coefficient. The multimodal authentication unit 216 calculates a first sum by summing the numerical value multiplied by the coefficient and the other numerical value that is not multiplied by the coefficient.

[0106] If any of the first sums is equal to or greater than the first threshold, the multimodal authentication unit 216 performs a second comparison process by comparing a second sum, which is the sum of the numerical values ​​of (A), (B), or (C) that were not used in the first comparison process, with the second threshold. Here, the multimodal authentication unit 216 may compare, with the second threshold, not each person stored in the storage unit 202, but only the person associated with the first sum that is equal to or greater than the first threshold. This reduces the amount of processing performed by the multimodal authentication unit 216.

[0107] If any of the second total values ​​is equal to or greater than the second threshold, the multimodal authentication unit 216 determines that the person associated with the second total value equal to or greater than the second threshold has been photographed, and determines that the multimodal authentication has been successful. On the other hand, if no of the second total values ​​is equal to or greater than the second threshold, the multimodal authentication unit 216 determines that the multimodal authentication has failed.

[0108] As another example, the multimodal authentication unit 216 may execute the following process. When the multimodal authentication unit 216 calculates one or two of distances (A'), (B'), or (C') for each person stored in the storage unit 202, the multimodal authentication unit 216 compares, as a first comparison process, a first total distance, which is the total distance of the calculated values, with a predetermined third threshold. Note that when the multimodal authentication unit 216 calculates and sums two of distances (A'), (B'), or (C'), it may multiply any of the distances by a predetermined coefficient. The multimodal authentication unit 216 calculates a first total distance by summing the distance multiplied by the coefficient and the other distances not multiplied by the coefficient.

[0109] If any of the first total distances is less than the predetermined third threshold, the multimodal authentication unit 216 performs a second comparison process by comparing the total value of the distances (A'), (B'), or (C') that were not used in the first comparison process with the predetermined fourth threshold. Here, the multimodal authentication unit 216 may compare, with the fourth threshold, not each person stored in the storage unit 202, but only the person associated with the first total distance that is less than the predetermined second threshold. This makes it possible to reduce the number of processes performed by the multimodal authentication unit 216.

[0110] If any of the second total distances is less than the fourth threshold, the multimodal authentication unit 216 determines that a person associated with the second total distance less than the fourth threshold has been photographed, and determines that multimodal authentication has been successful. On the other hand, if no of the second total distances is less than the fourth threshold, the multimodal authentication unit 216 determines that multimodal authentication has failed.

[0111] In the above calculation process, the multimodal authentication unit 216 may calculate either (B) or (C), or either (B') or (C'), as the iris feature amount. The multimodal authentication unit 216 may also use a method other than the above examples as a method for determining whether multimodal authentication is successful.

[0112] 9A and 9B are flowcharts showing an example of a typical process of the authentication system 20, and these flowcharts will be used to explain an overview of the process of the authentication system 20. Note that the explanation of the process shown in (2A) will be omitted as appropriate.

[0113] The processes described in steps S22 to S24 are the same as those shown in Fig. 7A. If a face image of a person is detected in the first image (Yes in step S24), the multimodal authentication unit 216 calculates a numerical value related to face authentication using the face image in the first image (step S31).

[0114] The processes described in steps S30 to S36 are the same as those shown in Figures 7A and 7B. After step S36, the multimodal authentication unit 216 detects the iris of the eye in the second image. Then, the multimodal authentication unit 216 calculates a numerical value related to iris authentication for the iris of the eye captured in the second image (step S33).

[0115] The multimodal authentication unit 216 determines whether the multimodal authentication was successful using the values ​​calculated in steps S31 and S33 (step S35). If the multimodal authentication failed (No in step S35), the multimodal authentication unit 216 calculates values ​​related to face authentication using the second image (step S31). Then, the processes from step S30 onward are executed again. However, the processes from step S30 to S36 do not have to be executed. In this case, the multimodal authentication unit 216 executes the determination process of step S35 again using the values ​​related to face authentication calculated using the second image and the values ​​related to iris face authentication calculated using the second image.

[0116] 9A, the multimodal authentication unit 216 calculates the numerical values ​​related to face authentication using the face image in the first image, but the multimodal authentication unit 216 may calculate the numerical values ​​related to face authentication using the face image in the second image. In this case, the multimodal authentication unit 216 does not execute the process of step S31, but after step S36, detects a face image of a person from the second image and calculates the numerical values ​​related to face authentication using the detected face image. The multimodal authentication unit 216 may calculate the numerical values ​​related to face authentication or the numerical values ​​related to iris authentication first, or both may be executed simultaneously.

[0117] [Explanation of Effect] As described above, the authentication system 20 can perform multimodal authentication by calculating numerical values ​​related to face authentication and numerical values ​​related to iris authentication without performing face authentication and iris authentication separately. Therefore, faster authentication is possible compared to (2A).

[0118] Embodiment 3 (3A) [Configuration] A variation of Embodiment 2 will be described below. In Embodiment 2, the shadow detection unit 222 detects the actual position of a shadow based on an image, and the factor estimation unit 224 estimates the cause of the shadow according to the detected position of the shadow. The setting unit 214 then sets the photographing conditions for the person based on the cause of the shadow estimated by the factor estimation unit 224. However, when setting the photographing conditions, it is not always necessary to detect the actual position of the shadow. In Embodiment 3, such an example will be described. Note that the description of the points already explained in the actual embodiment 2 will be omitted as appropriate, and the description will focus on the processing unique to Embodiment 3.

[0119] 10 is a block diagram showing an example of the evaluation unit 212. The evaluation unit 212 has a shadow prediction unit 242 instead of the shadow detection unit 222 and the factor estimation unit 224. The shadow prediction unit 242 predicts the position of a shadow cast on the iris in the image by analyzing the acquired image. Here, the shadow prediction unit 242 predicting the position of a shadow cast on the iris also includes determining whether or not at least a portion of the iris region is shadowed. If at least a portion of the iris region is shadowed, the shadow prediction unit 242 predicts the position of the shadow in the iris region.

[0120] Specifically, the shadow prediction unit 242 identifies at least one of the elements of the position, orientation, or shape of the person's face, or the presence or absence or shape of an item worn by the person, based on the acquired image. The shadow prediction unit 242 predicts the position of a shadow cast on the iris based on the identified elements.

[0121] For example, the shadow prediction unit 242 can estimate how the infrared illuminators 304 will illuminate the irises of the person based on information about the position, orientation, and shape of the identified person's face and information about the positions of the infrared illuminators 304. Furthermore, when multiple infrared illuminators 304 are provided, the shadow prediction unit 242 can estimate how the infrared illuminators 304 will illuminate the irises of the person by further using information about the positions and intensities (in other words, the lighting states) of the respective infrared illuminators 304. Based on the results of this estimation, the shadow prediction unit 242 may predict the position of a shadow.

[0122] Furthermore, the shadow prediction unit 242 may determine whether the person is wearing eyeglasses having frames with a predetermined width or more by further using the acquired image in addition to the above information. Then, the shadow prediction unit 242 may predict the position of a shadow cast on the iris by the eyeglass frames based on the determination result.

[0123] In addition to predicting the position of the shadow, the shadow prediction unit 242 also predicts at least one of the following factors that may cause the shadow: the position, orientation, or shape of the person's face, the shape of an accessory worn by the person, or the position or intensity of the infrared light 304 that illuminates the person. Specific examples of this are as described in the second embodiment.

[0124] The setting unit 214 reduces the influence of shadows on the iris area by setting the photographing conditions for the person based on the shadow generation factors predicted by the shadow prediction unit 242. The hardware control unit 232 controls the photographing conditions related to the hardware of the image capture device 30 (i.e., the photographing conditions for at least one of the infrared camera 302 and the infrared illuminator 304) based on the settings of the evaluation unit 212. The UI control unit 234 controls the display unit 306 to display a display screen that guides the person to perform a predetermined action. Details of this are as described in the second embodiment.

[0125] [Explanation of Processing Flow] FIGS. 11A and 11B are flowcharts showing an example of a typical process of the authentication system 20, and an overview of the process of the authentication system 20 will be explained using these flowcharts.

[0126] The processes from steps S22 to S28 in FIG. 11A are the same as those described in FIG. 7A, and therefore will not be described here.

[0127] If the face authentication is successful (Yes in step S28), the evaluation unit 212 evaluates the first image. Specifically, the shadow prediction unit 242 analyzes the first image to determine whether or not at least a part of the iris region in the first image is shaded (step S42).

[0128] If a shadow is cast on at least a portion of the iris region (Yes in step S42), the shadow prediction unit 242 predicts the cause of the shadow (step S44). The setting unit 214 sets the photographing conditions for the person based on the cause of the shadow predicted by the shadow prediction unit 242 so as to reduce the influence of the shadow on the iris region (step S46). That is, the photographing conditions for the person are updated from the first photographing conditions to the second photographing conditions.

[0129] The processing of steps S36 to S40 in Fig. 11B is the same as that described in Fig. 7B, and therefore will not be described here. The processing when it is determined in step S42 that the iris region is not shaded (No in step S42) is also the same as that described in Fig. 7B.

[0130] [Explanation of Effects] As described above, the authentication system 20 predicts, based on the image, that a shadow will be cast on part of the iris region in the captured first image, and updates the photographing conditions of the person based on the prediction result. As a result, it is believed that the influence of the shadow on the iris region will be reduced in the second image that is captured again. Therefore, the authentication system 20 achieves the same effects as those described in the second embodiment.

[0131] Furthermore, in the third embodiment, the process of detecting the actual position of the shadow is not required, and therefore the time required for authentication can be further reduced compared to the second embodiment.

[0132] (3B) The same variation as (2B) can be applied to (3A) described above. That is, in (3A), the authentication system 20 may be provided with a multimodal authentication unit 216 instead of the face authentication unit 208 and the iris authentication unit 210. This allows the authentication system 20 to perform multimodal authentication by calculating numerical values ​​related to face authentication and numerical values ​​related to iris authentication, without having to perform face authentication and iris authentication separately. Therefore, faster authentication is possible compared to (3A).

[0133] (3C) The configuration of the authentication system 20 shown in Fig. 3 (or Fig. 8) and Fig. 5 may be combined with the configuration of the authentication system 20 shown in Fig. 10. That is, the evaluation unit 212 may include a shadow detection unit 222, a factor estimation unit 224, and a shadow prediction unit 242. Furthermore, the processes shown in Figs. 7A and 7B or Figs. 9A and 9B may be combined in part or in whole with the processes shown in Figs. 11A and 11B as appropriate.

[0134] For example, assume that the authentication system 20 executes the processes described in FIGS. 11A and 11B and determines that iris authentication failed in step S40 of FIG. 11B (No in step S40). In this case, the authentication system 20 may execute the processes of steps S30 to S40 of FIGS. 7A and 7B. That is, when setting the photographing conditions for the second image for the first time, the authentication system 20 omits detection of the actual shadow position. If iris authentication is successful as a result of photographing under those photographing conditions, the authentication system 20 does not execute any special process. On the other hand, if iris authentication fails as a result of photographing under the photographing conditions set for the first time, the authentication system 20 detects the actual shadow position, estimates the cause of the shadow based on the detected shadow position, and sets the photographing conditions for the second image for the second time.

[0135] As described above, in the third embodiment, the time required for authentication can be shortened compared to the second embodiment. However, in the second embodiment, the photographing conditions are set by detecting the actual position of the shadow. Therefore, compared to the third embodiment, it is considered that the photographing conditions for successful iris authentication can be set more accurately. Therefore, by first performing the processing according to the third embodiment, the authentication system 20 can shorten the time until authentication. If the result is unsuccessful, the authentication system 20 can improve the possibility of successful iris authentication by performing the processing according to the second embodiment.

[0136] The authentication systems shown in the second and third embodiments may be configured as a single computer device or as a distributed system having multiple computer devices, similar to the information processing system 10 according to the first embodiment. Furthermore, some or all of the components of these systems may be provided on a cloud server built on a cloud, or on other types of virtualized servers generated using virtualization technology or the like.

[0137] In the above-described embodiments, this disclosure has been described as a hardware configuration, but this disclosure is not limited to this. This disclosure can also be realized by having a processor in a computer execute a computer program to perform the processing of the devices that make up the system described in the above-described embodiments.

[0138] 12 is a block diagram showing an example of the hardware configuration of an information processing device (i.e., a computer) on which the processing of the system or device shown in each embodiment is executed. Referring to FIG. 12, an information processing device 90 includes a signal processing circuit 91, a processor 92, and a memory 93.

[0139] The signal processing circuit 91 is a circuit for processing signals in accordance with the control of the processor 92. The signal processing circuit 91 may include a communication circuit for receiving signals from a transmitting device.

[0140] The processor 92 is connected to the memory 93, and performs the processing of the system described in the above embodiment by reading and executing a computer program from the memory 93. As an example of the processor 92, one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), and an ASIC (Application Specific Integrated Circuit) may be used, or a plurality of these may be used in parallel.

[0141] The memory 93 may be a volatile memory, a nonvolatile memory, or a combination thereof. The number of memories 93 is not limited to one, and multiple memories may be provided. The volatile memory may be, for example, a random access memory (RAM) such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The nonvolatile memory may be, for example, a read only memory (ROM) such as a programmable read only memory (PROM) or an erasable programmable read only memory (EPROM), a flash memory, or a solid state drive (SSD).

[0142] The memory 93 is used to store one or more instructions. Here, the one or more instructions are stored as programs in the memory 93. The processor 92 can perform the processes described in the above embodiments by reading and executing these programs from the memory 93.

[0143] The memory 93 may include a memory provided outside the processor 92, as well as a memory built into the processor 92. The memory 93 may also include a storage device located away from the processors constituting the processor 92. In this case, the processor 92 can access the memory 93 via an I / O (Input / Output) interface.

[0144] As described above, one or more processors included in each system or device in the above-described embodiments execute one or more programs including instructions for causing a computer to execute the algorithms described using the drawings. Execution of the programs enables the information processing described in each embodiment to be realized.

[0145] The program includes instructions or software code that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals. The transitory computer-readable medium or communication medium may provide the program to the computer via a wired communication path, such as an electric wire or optical fiber, or via a wireless communication path.

[0146] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) An information processing system comprising: an acquisition means for acquiring a first image including an iris of a subject; an identification means for identifying the position of a shadow cast on the iris based on the first image; and a setting means for setting photographing conditions for the subject based on the position of the shadow. (Supplementary Note 2) The information processing system according to Supplementary Note 1, further comprising: a first authentication means for performing iris authentication of the subject using a second image including the iris of the subject that is captured after the photographing conditions have been set. (Supplementary Note 3) The information processing system according to Supplementary Note 2, further comprising: a second authentication means for performing face authentication of the subject using an image including the face of the subject captured by the imaging means that captured the second image. (Supplementary Note 4) The information processing system according to any one of Supplements 1 to 3, wherein the identification means identifies the position of the shadow by comparing images of partial regions obtained by dividing the iris region into a plurality of regions in the first image. (Supplementary Note 5) The information processing system according to Supplementary Note 4, wherein the specifying means sets the partial region according to at least one of the number or position of lights that illuminate the object when capturing the first image. (Supplementary Note 6) The information processing system according to Supplementary Note 4 or 5, wherein the specifying means sets the partial region according to the shape of the object's eyes. (Supplementary Note 7) The information processing system according to Supplementary Note 6, wherein the specifying means specifies the shape of the object's eyes based on a facial image of the object. (Supplementary Note 8) The information processing system according to any one of Supplements 1 to 7, wherein the setting means specifies a cause of the shadow according to the position of the shadow, and sets a shooting condition for the object based on the specified cause of the shadow. (Supplementary Note 9) The information processing system according to Supplementary Note 8, wherein the setting means specifies the cause of the shadow by further using at least one of the density of the shadow, the position, orientation or shape of the object's face, the shape of an accessory worn by the object, the position or intensity of lights that illuminate the object, or the luminance value of an imaging means that captured the first image.(Supplementary Note 10) The information processing system according to any one of Supplements 1 to 7, wherein the identification means identifies at least one element of the position, orientation, or shape of the subject's face, or the presence or absence or shape of an item worn by the subject, based on the first image, and identifies the position of a shadow cast on the iris based on the identified element. (Supplementary Note 11) The information processing system according to any one of Supplements 1 to 10, wherein the setting means sets, as the shooting conditions, at least one of a setting of lighting for illuminating the subject, a setting of an imaging means that captured the first image, or guidance for the subject. (Supplementary Note 12) An information processing device comprising: an acquisition means that acquires an image including the iris of a subject; an identification means that identifies the position of a shadow cast on the iris based on the image; and a setting means that sets shooting conditions for the subject based on the position of the shadow. (Supplementary Note 13) An information processing method executed by a computer, comprising: acquiring an image including the iris of a subject; identifying the position of a shadow cast on the iris based on the image; and setting shooting conditions for the subject based on the position of the shadow. (Supplementary Note 14) A program that causes a computer to execute the following: acquiring an image including an iris of a subject; identifying a position of a shadow on the iris based on the image; and setting photographing conditions for the subject based on the position of the shadow.

[0147] Some or all of the elements (e.g., configurations and functions) described in Supplements 2 to 11 that are dependent on Supplement 1 may also be dependent on Supplement 12, Supplement 13, and Supplement 14 in the same dependency relationship as Supplements 2 to 11. Some or all of the elements described in any Supplement may be applied to various hardware, software, recording means for recording software, systems, and methods.

[0148] Although this disclosure has been described above with reference to the embodiments, this disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0149] 10 Information processing system 102 Acquisition unit 104 Identification unit 106 Setting unit 20 Authentication system 202 Storage unit 204 Acquisition unit 206 Face detection unit 208 Face authentication unit 210 Iris authentication unit 212 Evaluation unit 214 Setting unit 216 Multimodal authentication unit 222 Shadow detection unit 224 Factor estimation unit 232 Hardware control unit 234 UI control unit 242 Shadow prediction unit 30 Imaging device 302 Infrared camera 304 Infrared lighting 306 Display unit

Claims

1. An information processing system comprising: an acquisition means for acquiring a first image including the iris of a subject; an identification means for identifying the position of a shadow on the iris based on the first image; and a setting means for setting photographing conditions for the subject based on the position of the shadow.

2. The information processing system according to claim 1, further comprising a first authentication means for performing iris authentication of the subject using a second image including the iris of the subject that is captured after the photographing conditions are set.

3. The information processing system according to claim 2, further comprising a second authentication means for performing face authentication of the target using an image including the target's face captured by the imaging means that captured the second image.

4. An information processing system according to any one of claims 1 to 3, wherein the identification means identifies the position of the shadow by comparing images of each of the partial areas into which the iris area in the first image is divided.

5. An information processing system according to claim 4, wherein the identification means sets the partial area according to at least one of the number and position of lights that illuminate the object when the first image is captured.

6. The information processing system according to claim 4 or 5, wherein the specifying means sets the partial region according to the shape of the subject's eye.

7. An information processing system according to any one of claims 1 to 6, wherein the setting means identifies the cause of the shadow according to the position of the shadow, and sets the photographing conditions of the object based on the identified cause of the shadow.

8. The information processing system according to claim 7, wherein the setting means identifies the cause of the shadow by further using at least one of the density of the shadow, the position, orientation or shape of the subject's face, the shape of an accessory worn by the subject, the position or intensity of lighting illuminating the subject, or the luminance value of the imaging means that captured the first image.

9. An information processing system according to any one of claims 1 to 6, wherein the identification means identifies at least one element of the position, orientation or shape of the subject's face, or the presence or absence or shape of an item worn by the subject, based on the first image, and identifies the position of a shadow cast on the iris based on the identified element.

10. An information processing system according to any one of claims 1 to 9, wherein the setting means sets, as the photographing conditions, at least one of the lighting settings for illuminating the subject, the settings for the imaging means that captured the first image, or guidance for the subject.

11. An information processing device comprising: an acquisition means for acquiring an image including the iris of a subject; an identification means for identifying the position of a shadow cast on the iris based on the image; and a setting means for setting photographing conditions for the subject based on the position of the shadow.

12. An information processing method executed by a computer, comprising: acquiring an image including an iris of a subject; identifying the position of a shadow on the iris based on the image; and setting photographing conditions for the subject based on the position of the shadow.

13. A program that causes a computer to execute the following steps: acquire an image including the iris of a subject; identify the position of a shadow on the iris based on the image; and set shooting conditions for the subject based on the position of the shadow.

Images