Information processing device, information processing method, and program

The information processing device rapidly adjusts illumination to achieve a desired pupil size for iris authentication, addressing the challenge of obtaining accurate iris images efficiently and safely.

JP7578177B2Active Publication Date: 2024-11-06NEC CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023205147
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-12-05
Publication Date
2024-11-06
Estimated Expiration
2039-09-04

AI Technical Summary

Technical Problem

Existing iris authentication methods struggle to quickly obtain an iris image with a desired pupil size, leading to reduced authentication accuracy and potential discomfort or harm to the user due to prolonged exposure to varying light conditions.

Method used

An information processing device that acquires an eye region image, estimates illuminance, and adjusts illumination to achieve a desired pupil size by controlling the amount of light emitted, thereby ensuring rapid acquisition of an appropriate iris image.

Benefits of technology

This approach reduces the time required to obtain an iris image in a desired state, enhancing authentication accuracy and user comfort by maintaining optimal pupil size without causing discomfort or retina damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007578177000007
    Figure 0007578177000007
  • Figure 0007578177000008
    Figure 0007578177000008
  • Figure 0007578177000009
    Figure 0007578177000009
Patent Text Reader

Abstract

To provide a control device and the like that can shorten time required for acquiring an iris image in an objective state.SOLUTION: A control device 102 includes: an acquisition part 110 for acquiring an input image including an eye region that is a region of an eye part; an estimation part 120 for estimating illuminance of the eye part from the acquired input image; and a determination part 140 for determining a light volume of illumination of visible light with which the eye part is irradiated so that a size of a pupil of the eye part satisfies a size condition on the basis of an illuminance size relationship that is a relationship between the illuminance and the pupil.SELECTED DRAWING: Figure 10
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for controlling a device, and in particular to a technique for controlling lighting that irradiates light to a person. [Background technology]

[0002] Iris authentication is a method of biometric personal authentication that uses an image of the iris. The shape of the iris changes depending on the amount of light irradiated onto the pupil. For example, in a dark environment, the pupil dilates. In an image used for iris authentication, if the pupil is too dilated, the area of ​​the iris becomes narrow. As a result, the accuracy of authentication is likely to decrease. If strong light is irradiated onto the pupil, the pupil contracts and becomes smaller. If the pupil is too small, the detection of the pupil may fail, and the shape of the iris acquired as an image may differ significantly from the shape of the iris in the image used for authentication, resulting in a decrease in authentication accuracy. Irradiation of strong light that significantly reduces the size of the iris not only causes discomfort to the user, but may also damage the retina. In iris authentication, safe and highly accurate iris authentication can be achieved by using an iris image in which the size of the pupil is within a predetermined range.

[0003] Patent Document 1 describes an iris imaging device that captures an iris image suitable for personal authentication. The iris imaging device of Patent Document 1 estimates brightness based on a captured image. In a dark environment, the iris imaging device irradiates visible illumination light in the direction of the eyes. The iris imaging device appropriately controls the amount of visible light emission to capture a plurality of iris images with different pupil sizes.

[0004] Patent Document 2 describes a control system that irradiates light toward one of the left or right eye and controls an imaging unit to image the tail of the other eye while the light is being irradiated. Patent Document 4 describes a method for obtaining an iris image in which the pupil is a desired size by weakening or strengthening the illumination until the pupil size reaches the desired size.

[0005] Patent Document 3 describes a display device and a control method thereof that controls a light-emitting unit of the display device in consideration of the brightness of the environmental light at the time of shooting.

[0006] Patent Document 4 describes a biological information measuring device that measures biological information of a subject from the luminance of at least a part of the skin area around a detected pupil area. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2004-261515 A [Patent Document 2] International Publication No. 2016 / 080031 [Patent Document 3] JP 2014-155005 A [Patent Document 4] International Publication No. 2014 / 073645 Summary of the Invention [Problem to be solved by the invention]

[0008] For example, when performing iris authentication at a walk-through gate, it is necessary to quickly obtain an iris image suitable for iris authentication. In the technology of Patent Document 1, in order to obtain an appropriate iris image, it is necessary to select an appropriate iris image from iris images captured multiple times. In the technology of Patent Document 2, it is necessary to repeatedly capture images while changing the brightness of the lighting until an iris image with a pupil size of the desired size is obtained. With these technologies, there is a possibility that an iris image with a pupil size of the desired size cannot be obtained in a short time.

[0009] An object of the present disclosure is to provide a control device and the like that can shorten the time required to obtain an iris image in a desired state. [Means for solving the problem]

[0010] An information processing device according to one embodiment of the present disclosure comprises an acquisition means for acquiring a first input image including an eye region, which is an area of ​​the eye part of a subject, and a determination means for determining the amount of light of illumination to irradiate the eye region based on the pupil size of the subject included in the first input image, the acquisition means further comprising an authentication means for acquiring a second input image including the eye region irradiated with the amount of light determined by the determination means, and authenticating the subject using the second input image.

[0011] An information processing method according to one aspect of the present disclosure includes acquiring a first input image including an eye region that is an area of ​​the eye part of a subject, determining an amount of light of illumination to irradiate the eye region based on a pupil size of the subject included in the first input image, acquiring a second input image including the eye region illuminated with the determined amount of light, and authenticating the subject using the second input image.

[0012] A program according to one embodiment of the present disclosure causes a computer to execute an acquisition process for acquiring a first input image including an eye region, which is an area of ​​the eye part of a target, and a determination process for determining the amount of light of lighting to irradiate the eye region when irradiating a target of a first pupil size based on the pupil size of the target included in the first input image, the acquisition process further causing the computer to execute an authentication process for acquiring a second input image including the eye region irradiated with the amount of light determined by the determination process, and authenticating the target using the second input image. Effect of the Invention

[0013] The present disclosure has the advantage of being able to reduce the time required to obtain an iris image in a desired state. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of an imaging system according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram showing a schematic example of the arrangement of an imaging device, a visible light illumination device, and a near-infrared light illumination device relative to a person. [Diagram 3] FIG. 3 is a flowchart illustrating an example of the operation of the imaging system according to the first embodiment of the present disclosure. [Figure 4] FIG. 4 is a flowchart illustrating an example of the operation of the control device according to the first modified example of the first embodiment of the present disclosure. [Diagram 5] FIG. 5 is a flowchart illustrating an example of the operation of the control device according to the first modified example of the first embodiment of the present disclosure. [Figure 6] FIG. 6 is a block diagram illustrating an example of a configuration of an imaging system according to the second modified example of the first embodiment of the present disclosure. [Figure 7] FIG. 7 is a block diagram illustrating an example of a configuration of an imaging system according to the third modified example of the first embodiment of the present disclosure. [Figure 8] FIG. 8 is a block diagram illustrating a configuration of an imaging system according to a fourth modified example of the first embodiment of the present disclosure. [Figure 9] FIG. 9 is a block diagram illustrating an example of the configuration of an imaging system according to the second embodiment of the present disclosure. [Figure 10] FIG. 10 is a block diagram illustrating an example of the configuration of a control device according to the third embodiment of the present disclosure. [Figure 11] FIG. 11 is a flowchart illustrating an example of the operation of the control device according to the third embodiment of the present disclosure. [Figure 12] FIG. 12 is a diagram illustrating an example of a hardware configuration of a computer capable of realizing a control device according to an embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram illustrating an example of a lookup table according to the first embodiment of the present disclosure. [Figure 14] FIG. 14 is a diagram showing an all-around light environment model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Next, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0016] <First embodiment> First, a first embodiment of the present disclosure will be described in detail with reference to the drawings.

[0017] <<Configuration>> FIG. 1 is a block diagram illustrating an example of the configuration of an imaging system according to the present embodiment.

[0018] 1 includes a control device 100, an imaging device 200, a visible light illumination device 300, and a notification destination device 500. The imaging system 1 may further include a near-infrared light illumination device 400. In FIG. 1, the visible light illumination device 300 is illustrated below the control device 100, but the actual arrangement does not have to be as shown in this example.

[0019] FIG. 2 is a diagram showing an example of the arrangement of the imaging device 200, the visible light illumination device 300, and the near-infrared light illumination device 400 with respect to a person. As shown in FIG. 2, the imaging device 200 may be arranged to image the head of the person. The visible light illumination device 300 may be arranged to irradiate the head of the person being imaged with visible light. The near-infrared light illumination device 400 may be arranged to irradiate the head of the person being imaged with near-infrared light. The control device 100 is communicatively connected to the imaging device 200. The control device 100 is communicatively connected to the visible light illumination device 300. The control device 100 may be installed at a location away from the imaging device 200, the visible light illumination device 300, and the near-infrared light illumination device 400. The control device 100 may be installed adjacent to the imaging device 200, the visible light illumination device 300, and the near-infrared light illumination device 400.

[0020] In the following description, it is assumed that the person moves by walking. However, the person may move by a means other than walking. The person may not necessarily be moving.

[0021] <<Imaging device 200>> The imaging device 200 is a camera capable of capturing moving images and still images (hereinafter collectively referred to as images). The imaging device 200 transmits the captured images to the control device 100. The imaging device 200 may include an image sensor such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The image sensor of the imaging device 200 is a sensor sensitive to a wavelength band including at least a part of a near-infrared wavelength band (for example, 780 nm (nanometer) to 1000 nm). The imaging device 200 may be, for example, a visible light camera equipped with an image sensor to which an IR (Infrared) cut filter is not attached. In this case, the imaging device 200 may be a camera capable of capturing color images. The imaging device 200 may be a camera capable of capturing grayscale images. The imaging device 200 may be a near-infrared light camera.

[0022] The imaging device 200 may be installed, for example, at a walk-through gate or a passageway. The imaging device 200 may be installed so as to capture the faces of people passing through the walk-through gate or passageway (hereinafter also referred to as a walking path). The number of imaging devices 200 does not have to be one. The imaging device 200 may be realized by a plurality of cameras capturing images of different ranges. Specifically, the imaging device 200 may be realized by a plurality of cameras arranged so that, for example, when people of various heights walk along a walking path, the range through which the faces may pass can be captured by any of the cameras.

[0023] The imaging device 200 may be set to capture an image with a fixed focus position. The focus position is, for example, the distance from the camera center of the imaging device 200 to the focused position. In this case, the depth of field may be determined according to the focal length and aperture value of the imaging device 200 and the distance from the imaging device 200 to the focused plane.

[0024] In the description of this embodiment, the depth of field of the imaging device 200 represents a range of distance from the camera center of the imaging device 200 to an object that is considered to be in focus in an image captured by the imaging device 200. In other words, the lower limit of the depth of field of the imaging device 200 is the distance from the imaging device 200 to the closest object that is considered to be in focus in an image captured by the imaging device 200. The upper limit of the depth of field of the imaging device 200 is the distance from the imaging device 200 to the farthest object that is considered to be in focus in an image captured by the imaging device 200. The depth of field of the imaging device 200 represents a range from the above-mentioned lower limit to the above-mentioned upper limit. For example, when a moving person approaches the imaging device 200 from in front of the imaging device 200, whose focus position is fixed, when the imaging device 200 captures the person, it can be considered that the person is in focus if the distance to the person matches the upper limit of the range of distance represented by the depth of field. In such a situation, the focal position may be set to the upper limit of the depth of field. When the focal position of the image capturing device 200 is fixed, the imaging range of the image capturing device 200 is a range in which the distance to the image capturing device 200 is included in the depth of field.

[0025] The imaging device 200 may be set to automatically focus when a subject such as a person enters the imaging range. This imaging range may be a range in which a subject on which the imaging device 200 can focus may exist. In this case, the imaging device 200 may be set to detect eyes in an image obtained by imaging, and to focus on the eyes when the eyes are detected. The imaging device 200 may focus on the eyes by any of various existing methods for focusing on the eyes. In this case, the focus position may be the distance from the camera center of the imaging device 200 to the object that is considered to be most precisely in focus in the image captured by the imaging device 200. The imaging device 200 may be set to capture images even during the focusing operation. The imaging device 200 may be set to focus on the iris of a moving person in multiple consecutive images (frames). In other words, the image capturing apparatus 200 may be set so that the position of the iris of a person moving within the imaging range is included in the depth of field of the image capturing apparatus 200 while the image capturing apparatus 200 captures the person multiple times.

[0026] The imaging device 200 may be set to continuously capture images regardless of the presence or absence of a person (e.g., a pedestrian on a walking path) within the imaging range. The imaging device 200 may be set to start capturing images when a person such as a pedestrian enters the imaging range, and to continue capturing images until the moving object is no longer present within the imaging range. In this case, the imaging device 200 may be connected to a detection device that detects a person on the walking path using an object detection sensor realized by, for example, a near-infrared light sensor or an ultrasonic sensor, and transmits a signal indicating the detection to the imaging device 200 when a person is detected. In the following description of this embodiment, the imaging device 200 is set to continuously capture images after starting its operation.

[0027] The imaging device 200 transmits an image obtained by imaging (specifically, image data) to the control device 100. The image transmitted from the imaging device 200 to the control device 100 may be a moving image. The image transmitted from the imaging device 200 to the control device 100 may be a plurality of still images. The imaging device 200 further transmits camera parameters at the time of imaging and the like (hereinafter referred to as imaging information) to the control device 100. The imaging information is, for example, information such as shutter speed, aperture value, sensitivity, focal length, and focus position.

[0028] When the imaging device 200 is realized by a plurality of cameras, the imaging device 200 may send all images captured by the plurality of cameras to the control device 100. In this case, the imaging device 200 may send a set of an image and imaging information for each camera to the control device 100. When all the cameras included in the imaging device 200 are set to capture images under the same conditions and the imaging information of all the cameras is the same, the imaging device 200 may send the imaging information and an image for each camera to the control device 100.

[0029] <<Visible light illumination device 300>> The visible light illumination device 300 is an illumination device that irradiates visible light. The visible light illumination device 300 is configured so that the amount of light irradiated by the visible light illumination device 300 can be controlled by another device such as the control device 100. In other words, the visible light illumination device 300 is configured so that the intensity of the irradiated light can be changed under the control of the control device 100. The visible light illumination device 300 may be attached so as to irradiate the eye part of the face of a person (e.g., a person walking on a walking path) imaged by the imaging device 200. The eye part refers to a range including the eyes of the face of the person. Specifically, the visible light illumination device 300 is attached so that the light irradiated by the visible light illumination device 300 is incident on the pupil of the person imaged by the imaging device 200. The visible light illumination device 300 may be attached to the imaging device 200. The visible light illumination device 300 may be attached so as to face in the same direction as the direction of the optical axis of the imaging device 200. The visible light illumination device 300 may be attached near the image capture device 200 and oriented in the same direction as the optical axis of the image capture device 200 .

[0030] The relationship between the position and orientation of the visible light illumination device 300 and the position and orientation of the imaging device 200 is measured in advance and provided to the control device 100 .

[0031] <<Near infrared illumination device 400>> The near-infrared light illumination device 400 is a device that irradiates near-infrared light. The near-infrared light illumination device 400 may be installed so as to illuminate a position where the imaging device 200 is expected to perform imaging. The near-infrared light illumination device 400 may be configured so that the intensity of the irradiated near-infrared light falls within a predetermined range (for example, an intensity that does not damage the retina) in a range where the face of a person (particularly the eye part) imaged by the imaging device 200 may be present. For example, when the focus position of the imaging device 200 is fixed, the intensity of the irradiated near-infrared light may be configured so that the intensity of the irradiated near-infrared light falls within a predetermined range in a range of the plane of the focus position through which the face of the person may pass. The near-infrared light illumination device 400 may be realized by a plurality of near-infrared light illumination devices arranged in a row. The near-infrared light illumination device 400 may be configured so as to continuously irradiate near-infrared light after the imaging system 1 starts operating. The near-infrared light illumination device 400 may be configured to irradiate near-infrared light while the imaging device 200 is capturing an image. The near-infrared light illumination device 400 may be attached to the imaging device 200. The near-infrared light illumination device 400 may be attached near the imaging device 200. The near-infrared light illumination device 400 may be attached in the same or similar direction as the direction of the optical axis of the imaging device 200. The near-infrared light illumination device 400 may be connected to the above-mentioned object detection sensor and configured to irradiate near-infrared light when a moving object is detected on the walking path by the above-mentioned object detection sensor. In the following description of the present embodiment, the near-infrared light illumination device 400 is configured to continue irradiating near-infrared light regardless of whether the imaging device 200 is capturing an image.

[0032] <<Control device 100>> The control device 100 includes an acquisition unit 110, an estimation unit 120, a relationship storage unit 130, a determination unit 140, a control unit 150, and a notification unit 160. The control device 100 may further include an image storage unit 170.

[0033] <<Acquisition part 110>> The acquisition unit 110 acquires an image (specifically, data of the image) captured by the imaging device 200 from the imaging device 200. The acquisition unit 110 further acquires imaging information from the imaging device 200. When the acquisition unit 110 acquires a still image from the imaging device 200, it sends the still image and the imaging information to the estimation unit 120. When the acquisition unit 110 acquires a moving image from the imaging device 200, it may generate still image data for each frame, for example, from the acquired moving image data. Then, the acquisition unit 110 may send the generated still image data and imaging information to the estimation unit 120.

[0034] The acquisition unit 110 may detect a pupil from the acquired image. If a pupil is detected from the acquired image, the acquisition unit 110 may send the image to the estimation unit 120. If a pupil is not detected from the acquired image, the acquisition unit 110 does not need to send the image to the estimation unit 120.

[0035] The acquisition unit 110 may determine whether or not the eye region of the acquired image is in focus. The eye region represents a range of an area in the image that includes the eyes of a person's face. The method by which the acquisition unit 110 determines whether or not the eye region of the image is in focus may be any of the existing methods. The acquisition unit 110 may, for example, detect a high contrast area from the acquired image and detect a pupil area from the detected high contrast area. When the pupil area is detected from the high contrast area, the acquisition unit 110 may determine that the eye region is in focus. When the eye region of the acquired image is in focus, the acquisition unit 110 may send the image to the estimation unit 120. When the eye region of the acquired image is not in focus, the acquisition unit 110 does not need to send the image to the estimation unit 120.

[0036] When the imaging device 200 is realized by a plurality of cameras, the acquisition unit 110 receives a plurality of images captured by the plurality of cameras as described above. In this case, the acquisition unit 110 may detect the pupil area from the received plurality of images. The acquisition unit 110 may transmit, among the acquired plurality of images, an image in which a pupil is detected to the estimation unit 120, and may not transmit, to the estimation unit 120, an image in which a pupil is not detected.

[0037] The acquisition unit 110 may extract an image in which the eye region is in focus from the multiple received images. When an image in which the eye region is in focus is extracted from the multiple received images, the acquisition unit 110 may send the extracted image in which the eye region is in focus to the estimation unit 120. When an image in which the eye region is in focus is not extracted from the multiple received images, the acquisition unit 110 does not need to send the image to the estimation unit 120.

[0038] The acquisition section 110 may store the acquired images and imaging information in the image storage section 170 .

[0039] <<Estimation section 120>> The estimation unit 120 receives an image (specifically, image data) and imaging information from the acquisition unit 110. The estimation unit 120 estimates the illuminance in the eye portion from the received image based on the received imaging information. As described above, the eye portion refers to a face area including the eye. In this embodiment, the eye portion includes at least the iris including the pupil, and the sclera. In this embodiment, the illuminance of the pupil is considered to be the same as the illuminance of the eye portion. In other words, in this embodiment, the illuminance of the pupil is considered to be the same as the illuminance of the sclera.

[0040] The estimation unit 120 first detects the region of the sclera of the eye from the received image. Specifically, the estimation unit 120 may extract the region of the pupil in the received image. The estimation unit 120 may extract the region of the sclera based on the position of the extracted pupil. Specifically, the estimation unit 120 may detect, for example, the outline of the iris surrounding the pupil and the outline of the eye including the iris. The estimation unit 120 may extract, as the region of the sclera, an area brighter than the region of the iris between the outline of the iris and the outline of the eye. The method of extracting the region of the sclera by the estimation unit 120 is not limited to the above example.

[0041] The estimation unit 120 further estimates the illuminance of the sclera based on the imaging information and the pixel values ​​of the sclera region. Specifically, the estimation unit 120 estimates the illuminance of the sclera based on the imaging information such as the sensitivity, the aperture value, and the shutter speed, and the brightness of the sclera region represented by the pixel values ​​of the sclera region. The estimation unit 120 may estimate the illuminance of the sclera when the pixel value of the sclera extracted from the image becomes the pixel value of the sclera having a given reflection characteristic when the sclera is imaged under the conditions represented by the received imaging information. The reflection characteristic of the sclera may be appropriately determined approximately. For example, the estimation unit 120 may consider the part of the sclera other than the part where the reflection close to the specular reflection occurs as a uniform diffuse reflection surface having a certain reflectance. The estimation unit 120 may detect pixels brighter than a predetermined standard in the detected sclera region as pixels of the part where the reflection close to the specular reflection occurs. The estimation unit 120 may exclude pixels detected as pixels in a portion where reflection close to specular reflection occurs from the sclera region whose pixel values ​​are used to calculate the illuminance. The estimation unit 120 may then calculate the illuminance in the sclera region based on the pixel values ​​of the sclera region in the image and the reflectance of the sclera. For example, an administrator of the control device 100 may experimentally obtain the reflectance in advance and provide it to the estimation unit 120. The estimation unit 120 sets the estimated illuminance of the sclera as the illuminance of the eye portion including the pupil.

[0042] When the image obtained by imaging by the imaging device 200 is a color image, the estimation unit 120 may calculate the luminance of the sclera region in the color image according to a calculation method described below.

[0043] A method for calculating the luminance of the sclera in a color image captured by a camera such as the imaging device 200 will be described. The pixel value of each pixel included in the color image is represented by the value of an R (Red) component, a value of a G (Green) component, and a value of a B (Blue) component. In other words, the color image is composed of RGB. The chromaticity of this RGB and the chromaticity of white can be specified in advance as the color characteristics of the camera. The chromaticity of RGB and the chromaticity of white may be given in advance to the estimation unit 120. The RGB data (three values ​​of RGB) in the RGB color system are set so that they can be uniquely converted into tristimulus values ​​XYZ in the XYZ color system. An example of a conversion method for converting RGB data into tristimulus value XYZ data is shown below. The relationship between the RGB of the input image and the tristimulus values ​​XYZ is determined, for example, as shown in the following formula (1).

number

[0044] In formula (1), M RX is a 3×3 transformation matrix. Here, the captured image output from the camera is generally RGB with gamma correction, but RGB in formula (1) is assumed to have linear characteristics (γ=1.0) without gamma correction. The transformation formula for converting RGB to XYZ is not limited to the example of formula (1). The transformation formula may be any formula that can uniquely convert RGB to XYZ, and may be defined as follows, for example, with a quadratic term added:

number

[0045] Here, M' RX is a 3x9 transformation matrix.

[0046] In addition, M RX Or M' RXmay be calculated in advance by performing color calibration of the camera using known color patches, and provided to the estimation unit 120.

[0047] If information on the aperture value F, shutter speed, and gain of the camera lens at the time of shooting is available, the tristimulus value Y can be calculated by dividing the absolute luminance L (cd / m 2 ) can be converted to

[0048] The imaging device 200 may send information on the aperture value F, shutter speed, and gain at the time of shooting to the acquisition unit 110 of the control device 100. The acquisition unit 110 may receive the information on the aperture value F, shutter speed, and gain at the time of shooting from the control device 100, and send the received information on the aperture value F, shutter speed, and gain at the time of shooting to the estimation unit 120. The estimation unit 120 may receive the information on the aperture value F, shutter speed, and gain at the time of shooting from the acquisition unit 110. The estimation unit 120 derives tristimulus values ​​Y in terms of absolute luminance L (cd / m2) from the information on the aperture value F, shutter speed, and gain at the time of shooting, for example, according to the method shown below. 2 ) may be converted to

[0049] The tristimulus value Y calculated by formula (2) is expressed as absolute luminance L (cd / m 2 )(Candera / meter 2) is shown. Tristimulus value Y represents a stimulus value Y among the tristimulus values ​​XYZ. The estimation unit 120 may convert the tristimulus value Y to absolute luminance L according to the following method. In general, when taking a picture with a camera, for example, the camera sets the lens aperture value F, shutter speed, and gain to obtain a proper exposure. In the following description, it is assumed that the gain when the input image is taken is set to 1.0. That is, it is assumed that the estimation unit 120 acquires data indicating 1.0 as the gain value set when the input image was taken from the imaging device 200 via the acquisition unit 110. In this case, the tristimulus value Y corresponding to the pixel value (R, G, B) on the captured image for the absolute luminance L, which is the amount of light incident on the camera, is determined by two variables, the lens aperture value F and the shutter speed.

[0050] First, the camera to be used is calibrated in advance. Specifically, the calibration is carried out by, for example, calibrating the imaging conditions (i.e., the amount of incident light (absolute luminance L (cd / m 2 A plurality of captured images are captured while changing the color temperature (R), the lens aperture value F, and the shutter speed. Tristimulus values ​​Y are calculated from pixel values ​​(R, G, B) of the captured images using formula (1) or formula (2). Then, the relationship between the imaging conditions (i.e., absolute luminance L, which is the amount of incident light, the lens aperture value F, and the shutter speed) when the captured images were captured and the tristimulus values ​​Y obtained from the captured images is derived.

[0051] Specifically, a look-up table (LUT) may be generated as such a relationship. In this LUT, a plurality of values ​​corresponding to the imaging conditions (i.e., absolute luminance L, lens aperture value F, and shutter speed S) may be associated with tristimulus values ​​Y calculated from pixel values ​​of pixels corresponding to the location of the above-mentioned absolute luminance L in the captured image. The LUT may include a plurality of such combinations of absolute luminance L, lens aperture value F, shutter speed S, and tristimulus values ​​Y. The LUT may be generated such that the absolute luminance L is uniquely determined by the combination of lens aperture value F, shutter speed S, and tristimulus value Y.

[0052] If the lens aperture value F, shutter speed, and tristimulus value F calculated from the pixel values ​​of the captured image are obtained, the absolute luminance L can be obtained by this LUT. Therefore, this calibration generates an LUT for obtaining the absolute luminance L, which is the amount of incident light corresponding to the lens aperture value F, shutter speed, and tristimulus value Y. By using this LUT, the absolute luminance L (cd / m), which is the amount of incident light at the time of shooting, can be calculated from the conditions at the time of shooting with the camera (i.e., the lens aperture value F and shutter speed) and the tristimulus value Y corresponding to the pixel values ​​(R, G, B) of the image captured under those conditions. 2 ) can be determined.

[0053] Such an LUT may be provided to the estimation unit 120 in advance. The estimation unit 120 receives an input image and a lens aperture value F and a shutter speed when the input image was captured. The estimation unit 120 extracts a sclera area from the input image. The estimation unit 120 calculates a tristimulus value Y from pixel values ​​(R, G, B) of pixels included in the extracted sclera area. The estimation unit 120 determines an absolute luminance L from the received lens aperture value and shutter speed and the calculated tristimulus value Y using the LUT. The LUT may include a plurality of different combinations of absolute luminance L and tristimulus value Y for the same combination of aperture value and shutter speed.

[0054] If the LUT contains a tristimulus value that is associated with the received aperture value and shutter speed and matches the tristimulus value calculated from the pixel value, the estimation unit 120 identifies the absolute luminance associated with the tristimulus value in the LUT. The estimation unit 120 may treat the identified tristimulus value as the absolute luminance corresponding to the tristimulus value. Note that the absolute luminance corresponding to the tristimulus value represents the absolute luminance of an object whose brightness is represented as the pixel value from which the tristimulus value is calculated.

[0055] FIG. 13 is a diagram showing an example of an LUT. An example of an LUT will be described with reference to FIG. 13. In the calibration, a completely white board is photographed multiple times while changing the aperture value and the shutter speed. For example, when photographing is performed under eight conditions in which the amount of light incident on the camera from the completely white board is L1 or L2, the aperture value is F1 or F2, and the shutter speed is S1 or S2, eight photographed images are obtained. Eight tristimulus values ​​Y are obtained by calculating the tristimulus values ​​for the pixel values ​​of each of the obtained photographed images using formula (1) or formula (2). An LUT as shown in FIG. 13 is generated from the eight photographing conditions and the eight tristimulus values ​​Y obtained under the eight photographing conditions. The estimation unit 120 holds, for example, an LUT as shown in FIG. 13 in advance.

[0056] Next, a method for calculating the amount of incident light using the LUT will be described. The estimation unit 120 calculates tristimulus values ​​Y from pixel values ​​of a received input image.

[0057] As a specific example, a case will be described in which the amount of incident light L (i.e., absolute luminance L) is calculated from an input image captured with an aperture value of F1 and a shutter speed of S2. The estimation unit 120 calculates the tristimulus values ​​Y XThen, the estimation unit 120 extracts from the LUT tristimulus values ​​calculated from an image taken under the same shooting conditions as those of the input image (i.e., the state where the aperture value is F1 and the shutter speed is S2). In the example shown in FIG. 13, the tristimulus values ​​calculated from an image taken under the state where the aperture value is F1 and the shutter speed is S2 are Y2 and Y6. The estimation unit 120 compares the tristimulus values ​​Y2 and Y6 extracted from the LUT with the tristimulus value YX calculated from the input image. The estimation unit 120 reads out from the LUT the amount of incident light when the tristimulus value is Y2 and the amount of incident light when the tristimulus value is Y6. In the example shown in FIG. 13, the amount of incident light when the tristimulus value is Y2 is L1, and the amount of incident light when the tristimulus value is Y6 is L2. The estimation unit 120 calculates the amount of incident light LX by using an existing method such as an interpolation calculation, based on the difference between the tristimulus values ​​Y2 and Y6 extracted from the LUT and the tristimulus value YX calculated from the input image, and the amount of incident light L1 and L2 read from the LUT. In the example shown in FIG. 13, there are eight shooting conditions, but the number of shooting conditions is not limited to eight. When creating an LUT, shooting may be performed under more conditions to expand the amount of information in the LUT. By using such an LUT, a more accurate amount of incident light (i.e., absolute luminance L) can be obtained.

[0058] Note that the relationship between the amount of incident light, lens aperture value F, shutter speed, and image output value may vary depending on the model and individual differences of the camera (or the camera and lens, if the lens can be removed from the camera). Therefore, by performing calibration for each camera (or the combination of camera and lens, if the lens can be removed from the camera), it is possible to convert the tristimulus value Y into a highly accurate absolute luminance L (cd / m 2 ) can be converted to a LUT.

[0059] Next, a method for calculating the illuminance in the sclera region will be described. The estimation unit 120 calculates the apparent irradiance at each pixel position of the reflector in the image. The method described in Reference 1 or Reference 2 can be used to calculate the irradiance. (References 1) Imari Sato, Yoichi Sato, Katsushi Ikeuchi, “Estimation of Light Source Environment Based on Object Shadows”, Transactions of the Information Processing Society of Japan: Computer Vision and Image Media, Vol. 41, No. SIG 10 (CVIM 1), December 2000. (Reference 2) Takeshi Oishi, Sonoko Okura, Rei Kawakami, Takahiko Sakano, Katsushi Ikeuchi, “Superimposition of a human model onto an MR system based on a method for simultaneously capturing the light source environment and the target object using an omnidirectional camera”, Image Recognition and Understanding Symposium (MIRU2009), July 2009.

[0060] FIG. 14 is a schematic diagram of an all-around lighting environment model considering a surface light source. The calculation of irradiance will be described using the all-around lighting environment model considering a surface light source shown in FIG. 14. Point A is the center of a sphere. Any point on the reflector can be selected as the center A of the sphere. In this model, the radiance distribution of the light source from all periphery is modeled, observed at the center A of the sphere assuming that there is no obstruction between the light source and the center A. This radiance distribution is defined as L(θ,φ). Here, φ represents the azimuth angle, and θ represents the zenith angle.

[0061] Illuminance E at center A A is the small zenith angle dθ i and the small azimuth angle dφ i The small solid angle dω expressed by i Illuminance E is calculated by integrating the incident light energy received from A is expressed, for example, by the following equation:

number

[0062] Here, dθ i represents the small zenith angle, and dφ irepresents a small azimuth angle (i indicates incident). The condition for the above formula (3) to hold is that the light reflection characteristic of the reflector is Lambertian reflection. The reflection at the sclera is not necessarily Lambertian reflection. However, the situation in which the imaging device 200 captures the face of a person passing by corresponds to the situation in which a minute object, namely an eye, is observed from a distance. Therefore, when a point on the sclera is selected as the center A, the light reflected at the center A is in the form of an integration of the ambient light incident on the center A from all directions, and is constant regardless of the viewing direction. Therefore, it can be said that the above formula (3) is valid for the illuminance of the sclera. In other words, the illuminance of the sclera is approximately expressed by formula (3).

[0063] The luminance value I of a reflector recorded as an image A is the illuminance E as shown by equation (4). A and the surface reflectance of the reflector S A The surface reflectance of the reflector, S, is expressed as A can be obtained in advance, for example, by measurement. Here, the luminance value I A The absolute luminance L, which can be determined from the tristimulus value Y calculated by the formula (1) or (2), the aperture value F of the camera lens at the time of shooting, the shutter speed, the gain, etc., can be used as I A Let us take the previously obtained surface reflectance S A By dividing by, the illuminance E A may be calculated.

number

[0064] If the luminance value is the image luminance recorded by a camera, then the image luminance reflects the spectral sensitivity of the camera, expressed as a function of wavelength λ. If the spectral sensitivity of the camera is approximated by a delta function, then the wavelength λ can be considered as a constant. Therefore, the image luminance I at point A is A k (k is r, g, b) is expressed as follows:

number

[0065] Here, τ k is the camera gain. The estimation unit 120 may acquire the camera gain from the camera. That is, from the formula (5), the illuminance E A k is the image intensity I at point A A and the camera gain. The surface reflectance of the reflector S A k (k is r, g, b) is assumed to be obtained in advance, for example, by measurement. A k τ k and S. A k By dividing by E A k You can calculate the illuminance of visible light E A As shown in equation (6), E A r , E A g , E A b It is obtained by adding

number

[0066] From the above calculations, the illuminance around the sclera E A is obtained.

[0067] The estimation unit 120 sends the illuminance of the eye portion to the determination unit 140. The estimation unit 120 may further send distance information to the determination unit 140. The distance information may be information representing the distance from the visible light illumination device 300 to the eye portion in three-dimensional space.

[0068] For example, when the distance from the visible light illumination device 300 to the eye portion is considered to be the same as the distance from the imaging device 200 to the eye portion, the estimation unit 120 may send information indicating the distance from the imaging device 200 to the eye portion as the above-mentioned distance information to the determination unit 140. For example, when the imaging device 200 is configured to have a fixed focus distance and the distance from the imaging device 200 to the eye portion is given, the estimation unit 120 may set the given distance as the distance from the visible light illumination device 300 to the eye portion. When the acquisition unit 110 acquires the focus distance sent from the imaging device 200, the estimation unit 120 may set the focus distance acquired by the acquisition unit 110 as the distance from the visible light illumination device 300 to the eye portion.

[0069] When the distance from the visible light illumination device 300 to the eye portion is considered to be the same as the distance from the imaging device 200 to the eye portion, the estimation unit 120 may estimate the distance from the visible light illumination device 300 to the eye portion. Specifically, the estimation unit 120 may estimate the distance from the visible light illumination device 300 to the eye portion based on the positional relationship between the imaging device 200 and the visible light illumination device 300 and the positional relationship between the imaging device 200 and the eye portion. In this case, the positional relationship between the imaging device 200 and the visible light illumination device 300 may be given in advance. Furthermore, the estimation unit 120 may calculate the positional relationship between the imaging device 200 and the eye portion based on an image including the region of the eye portion captured by the imaging device 200 and imaging information when the image including the region of the eye portion is captured by the imaging device 200.

[0070] (Modification) Next, a modified example of the estimation unit 120 will be described. When the estimation unit 120 of this modified example calculates the absolute luminance using the LUT, the estimation unit 120 may calculate the absolute luminance as described below. The estimation unit 120 of this modified example may operate in the same manner as the estimation unit 120 described above, except when it operates as described below. When the LUT does not contain a tristimulus value that is associated with the received aperture value and shutter speed and matches the tristimulus value calculated from the pixel value, the estimation unit 120 extracts, for example, two tristimulus values ​​that are closest to the calculated tristimulus value from the tristimulus values ​​included in the LUT. Then, the estimation unit 120 may calculate an absolute luminance for the calculated tristimulus value from the absolute luminance associated with the extracted tristimulus value by interpolation based on the difference between the extracted tristimulus value and the calculated tristimulus value. For example, in the LUT, it is assumed that the two tristimulus values ​​that are closest to the calculated tristimulus value Ya that are associated with the received aperture value and shutter speed are Yb and Yc, the absolute luminance associated with Yb is Lb, and the absolute luminance associated with Yc is Lc. In this case, the estimation unit 120 may calculate the absolute luminance La of the object corresponding to the pixel having the pixel value from which the tristimulus value Ya is derived, for example, by the formula La = Lb + (Lc - Lb) x (Ya - Yb) / (Yc - Yb).

[0071] If the LUT does not contain any tristimulus values ​​associated with the received aperture value and shutter speed and matching the tristimulus values ​​calculated from the pixel values, the estimation unit 120 may select three or more tristimulus values ​​from the LUT, each of which is associated with an absolute luminance. The method of selecting three or more tristimulus values ​​may be any of various existing methods, such as a method of selecting by excluding outliers. In this case, the estimation unit 120 calculates the absolute luminance L corresponding to the calculated tristimulus value Y based on the selected tristimulus values. Specifically, the estimation unit 120 may derive a function representing the relationship between the tristimulus values ​​and the absolute luminance, for example, by the least squares method. Such a function may be a predetermined type of curve (for example, a straight line). The estimation unit 120 may calculate the absolute luminance L from the calculated tristimulus value Y by the derived function.

[0072] If the LUT does not include the combination of the received aperture value and shutter speed, the estimation unit 120 may use the LUT to calculate a combination of absolute luminance L and tristimulus values ​​Y when the shooting conditions are the received aperture value and shutter speed.

[0073] Specifically, the estimation section 120 may first extract a combination including an aperture value and a shutter speed that is closest to the received aperture value and shutter speed from the combinations included in the LUT.

[0074] When the aperture value is expressed by the F-number, the area of ​​the aperture opening is halved when the aperture value is multiplied by the square root of 2. The F-number is the focal length of the lens divided by the effective aperture. The F-number may also be expressed by a string consisting of the letter "F" followed by a decimal or integer with up to two significant digits, but here the F-number is expressed as a real number. When the shutter speed is expressed by the length of time the shutter is open, when the shutter speed value is halved the length of time the shutter is open is also halved.

[0075] When the optical system of the camera and the subject to be photographed are the same, if the sum of the logarithm of the squared aperture value (i.e., twice the logarithm of the aperture value) and the value obtained by inverting the sign of the logarithm of the shutter speed is the same, the brightness of the subject in the captured image should be the same. In these cases, the base of the logarithm may be, for example, 2. The sum of the logarithm of the squared aperture value, which is the F-number, and the value obtained by inverting the sign of the logarithm of the shutter speed, which is expressed by the length of time, is hereinafter referred to as the brightness value. The smaller this brightness value is, the brighter the image captured with the aperture value and shutter speed at which the brightness value was calculated will be. The sign of the brightness value may be reversed.

[0076] The aperture value may be expressed, for example, as a step number based on a specific aperture state. This step number is a numerical value that expresses the difference between two aperture states by the magnification of the aperture opening area. Hereinafter, the reference aperture state will be referred to as the reference state. In general, the area of ​​the aperture opening when the aperture is in one state is twice as large as the area of ​​the aperture opening when the aperture is in the other state. n double (or (1 / 2) n In this case, if the area of ​​the aperture of the iris in a certain state (state A) is 1 / 2 the area of ​​the aperture of the iris in the reference state, the number of steps representing the difference between the two states is n. n If the area of ​​the aperture in a certain state (state B) is twice the area of ​​the aperture in the reference state, the number of steps in state A is +n. n If the number of stops is n, then the number of stops for state B is -n, where the number of stops is the difference in the base 2 logarithms of the two F-stops representing the two aperture states.

[0077] Shutter speeds may also be expressed, for example, as a number of steps based on a certain shutter speed. This number of steps is a numerical value that expresses the difference between two shutter speeds as a magnification of the length of time represented by the shutter speed. Hereinafter, the reference shutter speed will be referred to as the reference shutter speed. In general, when the length of time represented by one shutter speed is twice as long as the length of time represented by the other shutter speed, n double (or (1 / 2) n In this case, if the length of time represented by a certain shutter speed (shutter speed A) is 1 / 2 the length of time represented by the reference shutter speed, the number of steps representing the difference between the two shutter speeds is n. n If the shutter speed is twice as long as the reference shutter speed, the number of stops for shutter speed A is -n. n If the shutter speed B is twice the normal shutter speed, then the number of stops for shutter speed B is -n, where the number of stops is the logarithm base 2 difference between the two shutter speeds.

[0078] In this case, the brightness value may be a value obtained by subtracting the shutter speed, which is represented by the number of stops from a predetermined shutter speed, from the aperture value, which is represented by the number of stops from a predetermined aperture.

[0079] The estimation unit 120 may calculate a brightness value from the received aperture value and shutter speed. The brightness value calculated from the received aperture value and shutter speed is hereinafter referred to as a target brightness value. The estimation unit 120 may calculate the above-mentioned brightness value for each combination of aperture value and shutter speed included in the LUT. The brightness value calculated from the combination of aperture value and shutter speed included in the LUT is hereinafter referred to as a LUT brightness value.

[0080] The estimation unit 120 may extract, from the LUT brightness values, a LUT brightness value that matches a target brightness value calculated from the received aperture value and shutter speed.

[0081] The estimation unit 120 may extract, from the LUT, a combination of the aperture value and the shutter speed for which the extracted LUT brightness value is calculated. If the extracted combination includes a combination associated with a tristimulus value Y that matches the tristimulus value Y calculated from the input image, the estimation unit 120 identifies an absolute luminance associated with the combination. The estimation unit 120 determines the identified absolute luminance as the absolute luminance of the target.

[0082] If there is no combination associated with a tristimulus value Y that matches the tristimulus value Y calculated from the input image among the extracted combinations, the estimation unit 120 selects a plurality of combinations from the extracted combinations. The estimation unit 120 may select two combinations associated with two tristimulus values ​​closest to the tristimulus value calculated from the input image. The estimation unit 120 may calculate an absolute luminance corresponding to the tristimulus value calculated from the input image from two absolute luminances associated with the two selected tristimulus values ​​Y by the above-mentioned interpolation. The estimation unit 120 may select three or more combinations associated with three or more tristimulus values ​​Y. The estimation unit 120 may derive a function, for example, as described above, from the tristimulus values ​​and absolute luminances associated with the selected three or more combinations. Then, the estimation unit 120 may calculate an absolute luminance corresponding to the tristimulus value Y calculated from the input image by the derived function.

[0083] If there is no LUT brightness value that matches the target brightness value, the estimation unit 120 calculates a value p by subtracting the target brightness value from the LUT brightness value. The estimation unit 120 further multiplies the LUT brightness value by 2 to the tristimulus value Y associated with the calculated aperture value and shutter speed. pThen, when there is a corrected tristimulus value that matches the tristimulus value calculated from the input image, the estimation unit 120 may determine the absolute luminance corresponding to the tristimulus value calculated from the input image as the absolute luminance corresponding to the tristimulus value calculated from the input image. When there is no corrected tristimulus value that matches the tristimulus value calculated from the input image, the estimation unit 120 may select two corrected tristimulus values ​​close to the tristimulus value calculated from the input image. The estimation unit 120 may calculate an absolute luminance corresponding to the tristimulus value Y calculated from the input image by the above-mentioned interpolation based on the two selected corrected tristimulus values ​​and two absolute luminances corresponding to the tristimulus values ​​calculated from the corrected tristimulus values. The estimation unit 120 may select three or more corrected tristimulus values ​​in a manner similar to the above-mentioned method of calculating three or more tristimulus values. Then, the estimation unit 120 may derive an absolute luminance corresponding to the tristimulus value Y calculated from the input image from the selected three or more corrected tristimulus values. In this case, the method of deriving the absolute luminance may be the same as the above-mentioned method of deriving the absolute luminance corresponding to the tristimulus value Y calculated from the input image based on three or more tristimulus values.

[0084] <<Relationship Storage Unit 130>> The relationship storage unit 130 stores the relationship between pupil size and illuminance. The size of a human pupil (i.e., pupil size) varies between approximately 2 mm (millimeter) and approximately 8 mm depending on the illuminance of the eye area. The relationship storage unit 130 stores data representing the relationship between pupil size and illuminance, which is determined, for example, experimentally. The data representing the relationship between illuminance and pupil stored in the relationship storage unit 130 is, for example, data that allows the pupil size to be obtained from the illuminance when the illuminance of the eye area is that illuminance, and data that allows the illuminance to be obtained from the pupil size when the pupil size is that pupil size. The data representing the relationship between illuminance and pupil is hereinafter also referred to as illuminance-size relationship.

[0085] <<Determination Unit 140>> The determination unit 140 receives the illuminance of the eye portion from the estimation unit 120. Based on the illuminance of the eye portion and the illuminance-size relationship, the determination unit 140 determines the amount of light of illumination to be irradiated onto the eye portion so that the size of the pupil satisfies a predetermined condition (hereinafter, referred to as the size condition). The size condition may be represented by, for example, a target value of the pupil size. The size condition may be represented by a range of the pupil size. In this case, the determination unit 140 may determine the target value of the pupil size by a predetermined method based on the size condition. For example, the determination unit 140 may determine the center value of the size range represented by the size condition as the target value. For example, the determination unit 140 may determine the maximum value of the range represented by the size condition as the target value. For example, the determination unit 140 may determine the minimum value of the range represented by the size condition as the target value. The determination unit 140 may determine another value as the target value.

[0086] The determination unit 140 determines an illuminance (hereinafter referred to as a target illuminance) at which the pupil size becomes a target value, for example, based on the illuminance-size relationship. Then, the determination unit 140 calculates a difference between the target illuminance and the received illuminance of the eye portion (hereinafter also referred to as an additional illuminance). In order to set the illuminance of the eye portion to the target illuminance, the determination unit 140 determines an amount of light (hereinafter also referred to as an amount of illumination) to be irradiated to the eye portion by the visible light illumination device 300, for example, based on the distance between the visible light illumination device 300 and the eye portion. In other words, the determination unit 140 determines an amount of light to be irradiated by the visible light illumination device 300, for example, based on the distance between the visible light illumination device 300 and the eye portion, so that the illuminance by the visible light illumination device 300 at the eye portion becomes the additional illuminance. The determination unit 140 may determine a setting value of the brightness of the visible light illumination device 300 required to irradiate the determined amount of light to the eye portion.

[0087] In order to calculate the amount of light irradiated by the visible light illumination device 300, the determination unit 140 may first calculate the distance from the visible light illumination device 300 to the eye area. The determination unit 140 may calculate the distance from the visible light illumination device 300 to the eye area, for example, based on the imaging information of the imaging device 200 and the positional relationship between the imaging device 200 and the visible light illumination device 300. Specifically, the determination unit 140 may find the positional relationship between the imaging device 200 and the eye area based on the camera parameters of the imaging device 200, the distance from the imaging device 200 to the focused position, and the position of the eye area in the captured image.

[0088] For example, the determination unit 140 may first calculate the direction of the eye site with respect to the imaging device 200, based on the position of the area of ​​the eye site in the captured screen and camera parameters (e.g., angle of view). When the eye site is in focus in the captured image, the eye site exists on the in-focus surface. The shape of the in-focus surface and its position relative to the imaging device 200 may be given to the determination unit 140 in advance. The determination unit 140 may calculate the intersection of a straight line toward the eye site and the in-focus surface as the position of the eye site. The determination unit 140 may calculate the relative position of the eye site with respect to the imaging device 200 as the position of the eye site. The determination unit 140 may calculate the relative position of the eye site with respect to the visible light illumination device 300, based on the relative position of the eye site with respect to the imaging device 200 and the relative position between the imaging device 200 and the visible light illumination device 300.

[0089] The determination unit 140 further calculates the amount of light from the visible light illuminator 300 such that the illuminance at the position of the eye area becomes the target illuminance, based on the positional relationship between the eye area and the visible light illuminator 300 (i.e., the above-mentioned relative position) and the illumination characteristics (e.g., light distribution characteristics) of the visible light illuminator 300. The illumination characteristics of the visible light illuminator 300 may be, for example, light distribution characteristics that represent the amount of light according to the angle from the direction in which the visible light illuminator 300 faces. The light distribution characteristics may be represented, for example, by a ratio to the amount of light in the direction in which the visible light illuminator faces. The illumination characteristics of the visible light illuminator 300 may be given in advance. In addition, the shape of the eye area may be, for example, assumed to be flat. In other words, the shape of the eye area may be approximated by, for example, a flat surface. The direction of the eye area may be, for example, assumed to be perpendicular to the direction of the walking path. For example, the estimation unit 120 may detect a face in the acquired image and estimate the direction of the detected face. In this case, estimation section 120 may estimate the face direction based on the orientation of imaging device 200. Determination section 140 may consider the determined face direction to be a perpendicular line to a plane that approximates the eye region. In this case, the positional relationship between the eye region and visible light illumination device 300 may further include the relationship between the direction of visible light illumination device 300 and the direction of the face (for example, the angle between the direction of visible light illumination device 300 and the direction of the face). Then, determination section 140 may calculate the light amount of visible light illumination device 300 further based on the relationship between the direction of visible light illumination device 300 and the direction of the face.

[0090] When the angle of view and the focal position of the imaging device 200 are fixed, the relationship between the position on the plane where the focus is achieved and the coordinates in the image captured by the imaging device 200 is fixed. In other words, the position of the eye part relative to the imaging device 200 can be calculated based on the position of the eye part extracted from the image captured by the acquisition unit 110. The positional relationship between the imaging device 200 and the visible light illumination device 300 is also fixed. In this case, a conversion formula and parameters for calculating a relative position with respect to the visible light illumination device 300 from a position in the image captured by the imaging device 200 may be derived in advance. Then, the determination unit 140 may calculate a relative position with respect to the visible light illumination device 300 from a position in the captured image using the conversion formula and parameters derived in advance and provided to the determination unit 140.

[0091] Determination unit 140 may calculate a setting value for visible light illuminator 300 to irradiate the calculated amount of light, based on the calculated amount of light, the light distribution characteristic, and the relationship between the setting value of the brightness of visible light illuminator 300 and the amount of light irradiated by visible light illuminator 300. The relationship between the setting value of the brightness of visible light illuminator 300 and the amount of light irradiated by visible light illuminator 300 may be given in advance.

[0092] The determination unit 140 sends the calculated setting value to the control unit 150 .

[0093] Note that determination section 140 may send the calculated amount of light (specifically, a value representing the amount of light) to control section 150. In this case, control section 150 receives the amount of light from determination section 140. Control section 150 may calculate a setting value for visible light illumination device 300 to irradiate the received amount of light, based on the received amount of light, the light distribution characteristics of visible light illumination device 300, and the relationship between the brightness setting value of visible light illumination device 300 and the light irradiated by visible light illumination device 300.

[0094] <<<About glare level>>> Determination unit 140 may change the amount of light emitted by visible light illumination device 300 so that a glare condition is satisfied. The glare condition is, for example, that the glare felt by a person illuminated with light by visible light illumination device 300 does not exceed a predetermined standard.

[0095] Specifically, the determination unit 140 may determine the amount of light emitted by the visible light illumination device 300 so that the glare felt by the person illuminated by the visible light illumination device 300 does not exceed a predetermined standard. For example, a value representing the degree of glare according to the illuminance of the eye area, in other words, an index of glare (hereinafter referred to as glare level) may be determined. The glare level may be, for example, a numerical value determined experimentally. In the following description, the larger the value of the glare level, the more dazzling the person feels the glare level. In this case, the above-mentioned predetermined standard of the glare level is represented by a glare level value (hereinafter referred to as glare upper limit value) representing the limit of allowable glare. Information representing the relationship between the illuminance of the eye area and the glare level may be stored in the relationship storage unit 130, for example. The relationship between the illuminance of the eye area and the glare level may be represented in the form of a lookup table, for example. The determination unit 140 may read out the relationship between the illuminance of the eye area and the glare level from the relationship storage unit 130. The determination unit 140 can specify the illuminance of the eye area corresponding to the upper glare limit (hereinafter, referred to as the upper limit illuminance) based on the relationship between the illuminance of the eye area and the glare level. The determination unit 140 may determine the additional illuminance based on the information representing the relationship between the illuminance of the eye area and the glare level so that the glare level does not exceed a predetermined value (i.e., the upper glare limit) representing the above-mentioned predetermined standard.

[0096] Specifically, the determination unit 140 may calculate the maximum value of the illuminance of the eye part (upper limit illuminance) at which the glare level does not exceed the above-mentioned predetermined standard (i.e., the upper limit glare level) based on, for example, the relationship between the illuminance of the eye part and the glare level. The determination unit 140 may further calculate the difference between the upper limit illuminance and the illuminance of the eye part estimated by the estimation unit 120 as the maximum value of the additional illuminance. If the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum value of the additional illuminance calculated based on the upper limit glare level, the determination unit 140 may change the value of the additional illuminance to the maximum value of the additional illuminance. If the additional illuminance calculated based on the illuminance-size relationship is not greater than the maximum value of the additional illuminance calculated based on the upper limit glare level, the determination unit 140 may not change the value of the additional illuminance.

[0097] The relationship between the illuminance and the glare level may be determined for each type of illumination. The relationship storage unit 130 may store the relationship between the illuminance and the glare level of the eye area for each type of illumination. The relationship storage unit 130 may store, for example, the relationship between the illuminance and the glare level of the eye area by natural light (for example, light when the visible light illumination device 300 is not irradiating light) and the relationship between the illuminance and the glare level of the eye area by the visible light illumination device 300 separately. In this case, the glare level may be determined so that the glare level when the eye area is illuminated by two light sources can be calculated based on the glare level when the eye area is illuminated by one of the two light sources and the glare level when the eye area is illuminated by the other of the two light sources. In other words, the relationship between the glare level when the eye area is illuminated by two light sources, the glare level when the eye area is illuminated by one of the two light sources, and the glare level when the eye area is illuminated by the other of the two light sources may be obtained in advance. Hereinafter, the relationship between the glare level when the eye area is illuminated by two light sources, the glare level when the eye area is illuminated by one of the two light sources, and the glare level when the eye area is illuminated by the other of the two light sources will be referred to as the relationship between the three glare levels. In this case, the two light sources are natural light and visible light illuminator 300. Based on the estimated illuminance of the eye area and the relationship between the three glare levels, determining unit 140 may calculate the maximum illuminance of visible light illuminator 300 when visible light illuminator 300 irradiates light such that the illuminance of the eye area satisfies the glare condition.

[0098] For example, the glare level of the light irradiated by the visible light illuminating device 300 may be determined according to the illuminance of natural light. For example, the magnitude of the illuminance of natural light may be divided into a plurality of ranges. Then, the relationship between the illuminance of the visible light illuminating device 300 and the glare level may be determined in advance for each range of the illuminance of natural light. In this case, the determining unit 140 identifies a range including the illuminance of the eye area estimated by the estimating unit 120, and can calculate the glare level from the illuminance of the visible light illuminating device 300 based on the relationship between the illuminance of the visible light illuminating device 300 and the glare level determined for the identified range. Furthermore, the determining unit 140 can calculate the illuminance of the visible light illuminating device 300 when the glare level is a value from the value of the glare level (for example, the maximum value of the glare level that satisfies a predetermined standard) based on the relationship between the illuminance of the visible light illuminating device 300 and the glare level determined for the identified range.

[0099] As described above, for example, the determination unit 140 calculates the maximum value of the illuminance of light by the visible light illumination device 300 when the glare level meets a predetermined standard and the illuminance of the eye area in a state in which the visible light illumination device 300 is not irradiating light is the illuminance of the eye area estimated by the estimation unit 120. The determination unit 140 may calculate this maximum value based on the relationship between the illuminance and glare level of natural light, the relationship between the illuminance and glare level of the visible light illumination device 300, and the relationships between the three glare levels described above. As described above, when the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum value of the calculated illuminance, the determination unit 140 may change the value of the additional illuminance to the maximum value of the calculated illuminance. When the additional illuminance calculated based on the illuminance-size relationship is not greater than the maximum value of the calculated illuminance, the determination unit 140 does not need to change the value of the additional illuminance.

[0100] <<<About controllability>>> The determination unit 140 determines whether the amount of light from the visible light illumination device 300 can be controlled so that the size of the pupil satisfies the size condition. In other words, the determination unit 140 may determine whether the illuminance of the eye area can be controlled by the irradiation of light from the visible light illumination device 300 to an illuminance that satisfies the size condition for the size of the pupil.

[0101] Specifically, determination unit 140 may determine whether the amount of light that visible light illumination device 300 needs to irradiate in order to set the illuminance of the eye area to an illuminance where the pupil size satisfies the size condition is within the range of the amount of light that visible light illumination device 300 can irradiate. Hereinafter, the amount of light that visible light illumination device 300 needs to irradiate in order to set the illuminance of the eye area to an illuminance where the pupil size satisfies the size condition is referred to as the required light amount. The range of the amount of light that visible light illumination device 300 can irradiate is referred to as the possible range.

[0102] If the requested light amount is not within the possible range, the determination unit 140 may determine that the light amount of the visible light illumination device 300 cannot be controlled so that the illuminance of the eye part becomes an illuminance at which the size of the pupil satisfies the size condition. In other words, if the requested light amount is not within the possible range, the determination unit 140 may determine that the light amount of the visible light illumination device 300 cannot be controlled so that the size of the pupil satisfies the size condition. If the requested light amount is within the possible range, the determination unit 140 may determine that the light amount of the visible light illumination device 300 can be controlled so that the illuminance of the eye part becomes an illuminance at which the size of the pupil satisfies the size condition. In other words, if the requested light amount is within the possible range, the determination unit 140 may determine that the light amount of the visible light illumination device 300 can be controlled so that the illuminance of the eye part becomes an illuminance at which the size of the pupil satisfies the size condition.

[0103] Specifically, for example, when the estimated illuminance of the eye portion is greater than the illuminance at which the pupil size satisfies the size condition, the determination unit 140 may determine that the illuminance of the eye portion cannot be controlled to an illuminance at which the pupil size satisfies the size condition by irradiating light from the visible light illumination device 300. For example, when the illuminance of the eye portion irradiated with the maximum amount of light is smaller than the illuminance at which the pupil size satisfies the size condition, the determination unit 140 may determine that the illuminance of the eye portion cannot be controlled to an illuminance at which the pupil size satisfies the size condition by irradiating light from the visible light illumination device 300. Note that the above-mentioned maximum amount of light is the strongest amount of light that the visible light illumination device 300 can irradiate.

[0104] When it is determined that the illuminance of the eye portion cannot be controlled to an illuminance where the size of the pupil satisfies the size condition, the determining unit 140 may issue a notification via the notifying unit 160 indicating that the illuminance of the eye portion cannot be controlled by the visible light illumination device 300 to an illuminance where the size of the pupil satisfies the size condition. Hereinafter, a notification indicating that the illuminance of the eye portion cannot be controlled by the visible light illumination device 300 to an illuminance where the size of the pupil satisfies the size condition will be referred to as an uncontrollable notification. For example, the determining unit 140 may send an instruction to the notifying unit 160 to transmit an uncontrollable notification. As described below, when an instruction to transmit an uncontrollable notification is received, the notifying unit 160 transmits the uncontrollable notification to a predetermined notification destination (for example, the notification destination device 500).

[0105] <<<Glare level and controllability>>> The determination unit 140 may be configured to determine that the illuminance of the eye portion cannot be controlled to an illuminance that satisfies the size condition of the pupil when the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum illuminance so that the glare condition is satisfied. In this case, the determination unit 140 may send an instruction to the notification unit 160 to send an uncontrollable notification.

[0106] The decision unit 140 may determine whether the illuminance of the eye area can be controlled to an illuminance at which the size of the pupil satisfies the size condition by irradiating light from the visible light illumination device 300, regardless of whether the glare condition is satisfied. Then, when the decision unit 140 determines that the amount of light from the visible light illumination device 300 cannot be controlled so that the size of the pupil satisfies the size condition, the decision unit 140 may send an instruction to the notification unit 160 to transmit an uncontrollable notification. Furthermore, the decision unit 140 may determine whether the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum value of the illuminance so that the glare condition is satisfied. When the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum value of the calculated illuminance, the decision unit 140 may change the value of the additional illuminance to the maximum value of the calculated illuminance. Furthermore, the determination unit 140 may send an instruction to the notification unit 160 to send a notification indicating that the glare condition cannot be satisfied if the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum illuminance such that the glare condition is satisfied.

[0107] If the additional illuminance calculated based on the illuminance-size relationship is smaller than the maximum illuminance such that the glare condition is satisfied, the decision unit 140 may determine that the illuminance of the eye area can be controlled to an illuminance where the size of the pupil satisfies the size condition.

[0108] In the following description of the operation, regardless of whether the glare condition is satisfied, decision unit 140 determines whether the illuminance of the eye area can be controlled to an illuminance where the size of the pupil satisfies the size condition by irradiating light from visible light illumination device 300. Furthermore, when the additional illuminance calculated based on the illuminance-size relationship is greater than the maximum illuminance so that the glare condition is satisfied, decision unit 140 does not issue an instruction to transmit a notification indicating that the glare condition cannot be satisfied.

[0109] <<Control unit 150>> The control unit 150 controls the visible light illumination device 300 so that the visible light illumination device 300 emits light with the amount of light determined by the determination unit 140 .

[0110] Specifically, control unit 150 receives from control unit 150 setting values ​​for visible light illumination device 300 to emit the amount of light determined by determination unit 140. Control unit 150 sets the received setting values ​​in visible light illumination device 300.

[0111] The control unit 150 may set the received setting value in the visible light illumination device 300, and then transmit an instruction to the imaging device 200 to perform imaging. If the imaging device 200 is set to continue imaging, the control unit 150 does not need to transmit an instruction to the imaging device 200 to perform imaging.

[0112] As described above, determination section 140 may send the calculated amount of light (specifically, a value representing the amount of light) to control section 150. In this case, control section 150 receives the amount of light from determination section 140. Control section 150 may calculate a setting value for visible light illumination device 300 to irradiate the received amount of light, based on the received amount of light, the light distribution characteristics of visible light illumination device 300, and the relationship between the brightness setting value of visible light illumination device 300 and the light irradiated by visible light illumination device 300.

[0113] <<Notification section 160>> When the determination unit 140 sends out a setting impossible notification, the notification unit 160 receives an instruction to transmit a setting impossible notification from the determination unit 140. As described above, when a setting impossible notification is received, the notification unit 160 transmits a setting impossible notification to a predetermined notification destination (e.g., notification destination device 500). The setting impossible notification may be a predetermined signal or the like. The setting impossible notification may indicate the reason why the illuminance of the eye part cannot be controlled to the target illuminance. The reason why the illuminance of the eye part cannot be controlled to the target illuminance is, for example, that the illuminance of the eye part due to natural light is too bright, or that the illuminance of the eye part due to natural light is too dark.

[0114] The notification unit 160 may receive an instruction to transmit a notification indicating that the glare condition cannot be satisfied (hereinafter, referred to as a glare notification) from the determination unit 140. When receiving an instruction to transmit a glare notification, the notification unit 160 transmits the glare notification to a predetermined notification destination. The glare notification may be a predetermined signal or the like different from the setting impossible notification.

[0115] <<Image storage unit 170>> Image storage unit 170 stores the image and imaging information acquired by acquisition unit 110. For example, when control unit 150 controls the amount of light emitted by visible light illumination device 300 and the size of the pupil of the eye portion changes to satisfy a size condition, the size of the pupil in the image newly acquired by acquisition unit 110 and stored in image storage unit 170 satisfies the size condition.

[0116] <<Notification destination device 500>> The notification destination device 500 receives the setting impossible notification transmitted by the notification unit 160. The notification destination device 500 performs output in response to the received setting impossible notification. The notification destination device 500 may be, for example, a terminal device held by an administrator of the imaging system 1. The notification destination device 500 may be, for example, a display device that displays information output from the control device 100. The notification destination device 500 may display, as an output in response to the setting impossible notification, a display (for example, a message) indicating that the illuminance of the eye part cannot be set by the visible light illumination device 300 to an illuminance that satisfies the size condition of the pupil. The notification destination device 500 may output, for example, a predetermined sound as an output in response to the setting impossible notification. The notification destination device 500 may output, for example, a predetermined image or animation as an output in response to the setting impossible notification. The notification destination device 500 may vibrate according to a predetermined pattern as an output in response to the setting impossible notification. The notification destination device 500 may output a combination of two or more of the above examples as an output in response to the setting impossible notification.

[0117] The notification destination device 500 may further receive the glare notification transmitted by the notification unit 160. The notification destination device 500 performs output in response to the received glare notification.

[0118] <<Operation>> Next, the operation of the control device 100 of this embodiment will be described in detail with reference to the drawings.

[0119] Fig. 3 is a flowchart showing an example of the operation of the control device 100 of this embodiment. In the operation shown in Fig. 3, the acquisition unit 110 acquires an input image including an eye area (step S101). Next, the estimation unit 120 estimates the illuminance of the eye area from the acquired input image. In this embodiment, the estimation unit 120 estimates the illuminance of the eye area based on pixel values ​​of the sclera area in the eye area of ​​the acquired input image.

[0120] Next, the determination unit 140 determines the amount of light to be lit so that the size of the pupil of the eye portion satisfies the size condition. The determination unit 140 determines whether the amount of light to be lit can be controlled so that the size of the pupil satisfies the size condition (step S104). If the amount of light to be lit cannot be controlled so that the size of the pupil satisfies the size condition (NO in step S104), the control device 100 notifies, for example, the notification destination device 500. Specifically, the determination unit 140 sends an instruction to the notification unit 160 to send a setting impossible notification. When the instruction to send a setting impossible notification is received, the notification unit 160 sends a setting impossible notification to the notification destination device 500. The control device 100 then performs the operation of step S106.

[0121] If the amount of illumination light can be controlled so that the pupil size satisfies the size condition (YES in step S104), control device 100 then performs the operation of step S106.

[0122] In step S106, decision unit 140 determines whether the glare condition is satisfied when visible light illumination device 300 irradiates the amount of light determined in step S103. If the glare condition is satisfied (YES in step S106), control device 100 then performs the operation of step S108.

[0123] If the glare condition is not satisfied (NO in step S106), the decision unit 140 changes the amount of light emitted by the visible light illumination device 300 so that the glare condition is satisfied (step S107). The control device 100 then performs the operation of step S108.

[0124] In step S108, control unit 150 controls the amount of light of visible light illumination device 300 so that the amount of light of visible light illumination device 300 becomes the determined amount of light. Specifically, determination unit 140 may determine a setting value of visible light illumination device 300 for making the amount of light of visible light illumination device 300 the determined amount of light, based on a relationship between the setting value and the amount of light, for example, represented by a look-up table. Determination unit 140 transmits the determined setting value to control unit 150. Control unit 150 receives the setting value from the determination unit, and sets the setting value of visible light illumination device 300 to the received setting value.

[0125] <<Effects>> This embodiment has an effect of shortening the time required to obtain an iris image in a desired state. The iris image in a desired state is, for example, an image of an iris in which the size of the pupil satisfies the size condition. This is because the estimation unit 120 estimates the illuminance of the eye part from the acquired input image, and the determination unit 140 determines the amount of light irradiated by the visible light illumination device 300 so that the size of the pupil of the eye part satisfies the size condition. The control unit 150 controls the visible light illumination device 300 so that the amount of light irradiated by the visible light illumination device 300 becomes the determined amount of light, and thereby the size of the pupil changes so that the size of the pupil of the eye part satisfies the size condition. The iris image in a desired state is obtained by capturing an image of the eye part after the size of the pupil has changed.

[0126] <First Modification> Next, a first modified example will be described. The first to fifth modified examples described below represent examples of modifications of the first embodiment.

[0127] <<Configuration>> The imaging system 1 of this modified example is the same as the imaging system of the first embodiment, except for the difference in the control device 100 described below. The control device 100 of this modified example is the same as the control device 100 of the first embodiment, except for the following differences.

[0128] In a first modified example, the estimation unit 120 detects an iris region in the input image, and further estimates the color of the iris based on the pixel values ​​of pixels included in the detected iris region. The estimation unit 120 may estimate which of a plurality of predefined colors the iris color is based on the pixel values ​​of pixels in the detected iris region. The estimation unit 120 sends information representing the estimated iris color to the determination unit 140.

[0129] The relationship storage unit 130 stores the illuminance-size relationship for each iris color. The relationship storage unit 130 may store, for example, a lookup table representing the relationship between the illuminance of the eye part and the size of the pupil as the illuminance-size relationship for each of a plurality of predefined colors.

[0130] The determination unit 140 receives information representing the estimated iris color from the estimation unit 120. The determination unit 140 reads out the illuminance-size relationship stored for the estimated iris color, and determines the amount of light to be irradiated by the visible light illumination device 300 based on the read-out illuminance-size relationship.

[0131] In this modification, furthermore, a glare condition is obtained for each iris color. For example, the glare condition set for each iris color may be stored in the relationship storage unit 130.

[0132] The determination unit 140 may read out the glare condition for the estimated iris color from, for example, the relation storage unit 130. The determination unit 140 may store the glare condition for each iris color and select the glare condition for the estimated iris color. The determination unit 140 may change the amount of light irradiated by the visible light illumination device 300 according to the above-mentioned glare level or instruct the transmission of a glare notification based on the glare condition for the estimated iris color.

[0133] The same glare condition may be defined for each of the multiple iris colors. A relationship between illuminance and glare intensity may be defined for each iris color. In this case, the relationship storage unit 130 may store the relationship between illuminance and glare intensity for each iris color. The determination unit 140 may determine the glare intensity from the illuminance using the relationship between illuminance and glare intensity for each iris color. The determination unit 140 may also determine the illuminance at which the glare intensity becomes a given value from the value of the glare intensity using the relationship between illuminance and glare intensity for each iris color.

[0134] <<Operation>> Next, the operation of the control device 100 of this modified example will be described in detail with reference to the drawings.

[0135] 4 and 5 are flowcharts showing an example of the operation of the control device 100 of this modified example.

[0136] In steps S101 and S102 shown in FIG. 4, the control device 100 performs the same operations as those in steps S101 and S102 of the control device 100 of the first embodiment shown in FIG.

[0137] In step S203, the estimation unit 120 estimates the iris color based on the input image (step S203). The estimation unit 120 may, for example, detect an iris area from the input image. The estimation unit 120 may estimate the iris color from pixel values ​​of pixels included in the iris area. The estimation unit 120 may, for example, estimate a color corresponding to the iris color of the detected area from among a plurality of predetermined iris colors. The estimation unit 120 sends information representing the estimated iris color to the determination unit 140. The determination unit 140 receives the information representing the iris color from the estimation unit 120.

[0138] Next, the determination unit 140 determines a glare condition based on the estimated iris color (step S204). The determination unit 140 may select a glare condition determined for the estimated iris color from among a plurality of glare conditions.

[0139] Next, the determining unit 140 selects an illuminance-size relationship based on the estimated iris color (step S205). The determining unit 140 may select an illuminance-size relationship determined for the estimated iris color from a plurality of illuminance-size relationships.

[0140] Based on the selected illuminance-size relationship, determination unit 140 determines the amount of illumination (i.e., the amount of light irradiated by visible light illumination device 300) such that the size of the pupil of the eye portion satisfies the predetermined condition (i.e., the above-mentioned size condition) (step S206). Next, control device 100 performs the operation of step S104 in FIG. 5.

[0141] The operations from step S104 to step S108 of the control device 100 of this modified example shown in FIG. 5 are the same as the operations from step S104 to step S108 of the control device 100 of the first embodiment shown in FIG.

[0142] As described above, the determination unit 140 may use the same glare condition regardless of the iris color, and control the visible light illumination device 300 so that the illuminance of the eye area becomes an illuminance whose glare level satisfies the glare condition, based on the relationship between the illuminance and the glare level according to the iris color. In this case, in step S204, the determination unit 140 selects the relationship between the illuminance and the glare level determined for the estimated iris color. The other operations are the same as those in the example of the operation described above.

[0143] <<Effects>> This modification has the same effects as the first embodiment, for the same reasons that the effects of the first embodiment are obtained.

[0144] This modification has a second advantage in that it is possible to obtain an iris image in a desired state regardless of the iris color, because the estimation unit 120 estimates the iris color from the input image, and the determination unit 140 determines the amount of light to be irradiated by the visible light illumination device 300 based on the illuminance-size relationship according to the estimated iris color.

[0145] This modification has a third advantage in that it can appropriately reduce glare regardless of the color of the iris, because determination unit 140 changes the amount of light emitted by visible light illumination device 300 based on the glare condition according to the estimated iris color.

[0146] In this modification, the determination unit 140 may change the amount of light emitted by the visible light illumination device 300 based on the same glare condition regardless of the color of the iris. In that case, this modification provides the same effect as the first embodiment, as well as the second effect described above.

[0147] In this modification, the determination unit 140 may determine the amount of light to be emitted by the visible light illumination device 300 based on the same illuminance-size relationship regardless of the color of the iris. In that case, this modification provides the same effects as the first embodiment, as well as the third effect described above.

[0148] <Second Modification> Next, a second modified example will be described. An imaging system 1A of this modified example is the same as the imaging system 1 of the first embodiment, except for the differences described below.

[0149] <<Configuration>> 6 is a block diagram showing an example of the configuration of an imaging system 1A according to this modification. The imaging system 1A according to this modification includes a control device 100, an imaging device 200, a visible light illumination device 300, a near-infrared light illumination device 400, a notification destination device 500, and in addition, a detection device 600.

[0150] The detection device 600 detects a person, for example, on the walking path described above. When the detection device 600 starts to detect a person, it transmits a notification indicating that the person has started to be detected to the imaging device 200 and the near-infrared light illumination device 400. When the detection device 600 stops detecting a person, it transmits a notification indicating that the person has stopped being detected to the imaging device 200 and the near-infrared light illumination device 400. The notification indicating that the person has started to be detected (hereinafter, referred to as a detection notification) may be realized by transmitting a predetermined signal (hereinafter, also referred to as a detection signal). The notification indicating that the person has stopped being detected (hereinafter, a non-detection notification) may be realized by transmitting another predetermined signal (a non-detection signal). The detection notification may be realized by starting to transmit a predetermined signal (hereinafter, referred to as a detection continuation signal). The non-detection notification may be realized by ending the transmission of the detection continuation signal. The detection device 600 may be realized by a sensor that detects a person, etc., by, for example, near-infrared light, ultrasonic waves, or the like.

[0151] The imaging device 200 receives a detection notification and a non-detection notification from the detection device 600. The imaging device 200 continuously captures images during the period from when the imaging device 200 receives the detection notification to when the imaging device 200 receives the non-detection notification from the detection device 600, and transmits the images obtained by the capture to the control device 100. When the imaging device 200 receives a non-detection notification, the imaging device 200 may suspend imaging and image transmission until the next detection notification is received.

[0152] The near-infrared light illumination device 400 receives a detection notification and a non-detection notification from the detection device 600. The near-infrared light illumination device 400 continuously irradiates near-infrared light during the period from when it receives the detection notification to when it receives the non-detection notification from the detection device 600. When the near-infrared light illumination device 400 receives a non-detection notification, it may stop irradiating near-infrared light until it next receives a detection notification.

[0153] <<Operation>> The control device 100 of this modified example performs the same operations as in the first embodiment.

[0154] <Third Modification> Next, a third modified example will be described. An imaging system 1B of this modified example is the same as the imaging system 1A of the second modified example, except for the differences described below.

[0155] <<Configuration>> FIG. 7 is a block diagram showing an example of the configuration of the imaging system 1B of this modification. The imaging system 1B of this modification includes a detection device 600 in addition to the control device 100, the imaging device 200, the visible light illumination device 300, the near-infrared light illumination device 400, and the notification destination device 500, as in the imaging system 1A of the first modification. In FIG. 7, two imaging devices 200 are depicted. However, the imaging device 200 depicted above the control device 100 is the same imaging device 200 as the imaging device 200 depicted below the control device 100. In other words, the same imaging device 200 is depicted above the control device 100 and below the control device 100. In this modification, the imaging device 200 is connected to each of the acquisition unit 110 and the control unit 150 of the control device 100. The near-infrared light illumination device 400 is connected to the control unit 150 of the control device 100. The detection device 600 is connected to the acquisition unit 110 of the control device 100.

[0156] The detection device 600 detects a person, for example, on the walking path described above. In this modification, the detection device 600 transmits a detection notification and a non-detection notification to the acquisition unit 110 of the control device 100.

[0157] The acquiring unit 110 receives a detection notification and a non-detection notification from the detection device 600. When the acquiring unit 110 receives a detection notification from the detection device 600, it sends an instruction to the control unit 150 to transmit a detection notification to the image capture device 200 and the near-infrared light illumination device 400. When the acquiring unit 110 receives a non-detection notification from the detection device 600, it sends an instruction to the control unit 150 to transmit a non-detection notification to the image capture device 200 and the near-infrared light illumination device 400.

[0158] The control unit 150 receives an instruction to transmit a detection notification and an instruction to transmit a non-detection notification from the acquisition unit 110. When the control unit 150 receives an instruction to transmit a detection notification from the acquisition unit 110, the control unit 150 transmits the detection notification to the image capture device 200 and the near-infrared light illumination device 400. When the control unit 150 receives an instruction to transmit a non-detection notification from the acquisition unit 110, the control unit 150 transmits the non-detection notification to the image capture device 200 and the near-infrared light illumination device 400.

[0159] The imaging device 200 receives a detection notification and a non-detection notification from the control unit 150. The imaging device 200 continuously captures images during the period from when it receives the detection notification to when it receives the non-detection notification from the control unit 150, and transmits the images obtained by the capture to the control device 100. When the imaging device 200 receives a non-detection notification, the imaging device 200 may suspend imaging and image transmission until it receives the next detection notification.

[0160] The near-infrared light illumination device 400 receives a detection notification and a non-detection notification from the control unit 150. The near-infrared light illumination device 400 continuously irradiates near-infrared light during the period from when it receives the detection notification to when it receives the non-detection notification from the control unit 150. When the near-infrared light illumination device 400 receives a non-detection notification, it may interrupt the irradiation of near-infrared light until it next receives a detection notification.

[0161] <<Operation>> The control device 100 of this modified example performs the same operations as in the first embodiment.

[0162] <Fourth modified example> Next, a fourth modified example will be described. An imaging system 1C of this modified example is the same as the imaging system 1 of the first embodiment, except for the differences described below.

[0163] <<Configuration>> Fig. 8 is a block diagram showing a configuration of an imaging system 1C of this modified example. The imaging system 1C in Fig. 8 includes imaging devices 200A and 200B instead of the imaging device 200. In the example shown in Fig. 8, a near-infrared light illumination device 400 is connected to the imaging device 200B.

[0164] The imaging devices 200A and 200B are attached so as to capture an image of the head of a person walking along the walking path described above, for example. The focus position of the imaging device 200A and the focus position of the imaging device 200B are fixed. The focus position of the imaging device 200A and the focus position of the imaging device 200B are set so that the head of a person walking along the walking path passes through the focus position of the imaging device 200A and then passes through the focus position of the imaging device 200B. The imaging devices 200A and 200B may continuously capture images. The near-infrared light illumination device 400 may continuously irradiate near-infrared light.

[0165] The control unit 150 may control the visible light illumination device 300 so that the amount of light irradiated by the visible light illumination device 300 becomes the determined amount of light, and then transmit an instruction to the imaging device 200B to perform imaging. In this case, the imaging device 200B may start imaging according to the instruction from the control unit 150. When starting imaging, the imaging device 200B may transmit a notification to start imaging to the near-infrared light illumination device 400. When ending imaging, the imaging device 200B may transmit a notification to end imaging to the near-infrared light illumination device 400. The near-infrared light illumination device 400 may receive a notification to start imaging and a notification to end imaging from the imaging device 200B. The near-infrared light illumination device 400 may continue to irradiate near-infrared light while the imaging device 200B is performing imaging, for example, from the time when it receives a notification to start imaging to the time when it receives a notification to end imaging.

[0166] The acquisition section 110 of this modified example receives an image captured by the imaging device 200A.

[0167] <<Operation>> The control device 100 of this modified example performs the same operations as the control device 100 of the first embodiment shown in FIG.

[0168] Note that control unit 150 in this embodiment may transmit an instruction to image capture device 200B to capture an image after controlling the illumination in step S108, i.e., after setting the brightness setting value of visible light illumination device 300. In this case, image capture device 200B may start capturing an image upon receiving the instruction to capture an image. Image capture device 200B may capture an image upon receiving the instruction to capture an image.

[0169] <Fifth modified example> Next, a fifth modification of this modified example will be described.

[0170] At least two of the first modification, the second modification or the third modification, and the fourth modification may be combined. The fifth modification is a combination of at least two of the first modification, the second modification or the third modification, and the fourth modification.

[0171] When the first modified example is combined with the second modified example or the third modified example, the imaging device 200 does not need to be connected to the near-infrared light illumination device 400. When the first modified example is combined with the fourth modified example, the imaging device 200 does not need to be connected to the near-infrared light illumination device 400. When only the first modified example is combined with the fourth modified example, the configuration of the imaging system of the fifth modified example may be the same as the configuration of the imaging system 1C of the fourth modified example shown in FIG.

[0172] When the first and second modified examples are combined, the detection device 600 is connected to the imaging device 200 and the near-infrared light illumination device 400, as in the example shown in FIG. 6. Then, a detection notification and a non-detection notification are transmitted from the detection device 600 to the imaging device 200 and the near-infrared light illumination device 400. When the first and third modified examples are combined, the control unit 150 is connected to the imaging device 200 and the near-infrared light illumination device 400, as in the example shown in FIG. 7. Then, a detection notification and a non-detection notification are transmitted from the control unit 150 to the imaging device 200 and the near-infrared light illumination device 400.

[0173] When the second or third modified example is combined with the fourth modified example, the imaging device 200B does not need to be connected to the near-infrared light illumination device 400. The same applies to the case where the first modified example is combined with the second or third modified example and the fourth modified example.

[0174] When the second modified example and the fourth modified example are combined, the detection device 600 is connected to the imaging device 200A, the imaging device 200B, and the near-infrared light illumination device 400. Then, a detection notification and a non-detection notification are transmitted from the detection device 600 to the imaging device 200A, the imaging device 200B, and the near-infrared light illumination device 400. The imaging devices 200A and 200B may continuously capture images from the time when they receive the detection notification until they receive the non-detection notification. The near-infrared light illumination device 400 may continuously irradiate near-infrared light from the time when they receive the detection notification until they receive the non-detection notification. The same applies to the case where the first modified example, the second modified example, and the fourth modified example are combined.

[0175] When the third modified example and the fourth modified example are combined, the control unit 150 may be connected to the imaging device 200A, the imaging device 200B, and the near-infrared light illumination device 400. Then, the control unit 150 may transmit a detection notification and a non-detection notification to the imaging device 200A, the imaging device 200B, and the near-infrared light illumination device 400. Then, the detection notification and the non-detection notification are transmitted from the detection device 600 to the imaging device 200A, the imaging device 200B, and the near-infrared light illumination device 400. The imaging devices 200A and 200B may continuously capture images from the time when they receive the detection notification until they receive the non-detection notification. The near-infrared light illumination device 400 may continuously irradiate near-infrared light from the time when they receive the detection notification until they receive the non-detection notification. The same applies to the case where the first modified example, the third modified example, and the fourth modified example are combined.

[0176] When the second modified example and the fourth modified example are combined, the detection device 600 may be connected to the imaging device 200A. Then, a detection notification and a non-detection notification may be transmitted from the detection device 600 to the imaging device 200A. The imaging device 200A may continuously capture images from the time when it receives the detection notification until it receives the non-detection notification. In this case, the imaging device 200B may continuously capture images, and the near-infrared light illumination device 400 may continuously irradiate near-infrared light. In this case, the control unit 150 may transmit an instruction to the imaging device 200B to start capturing images. The imaging device 200B may start capturing images according to an instruction from the control unit 150. Furthermore, when the imaging device 200B starts capturing images, it may transmit a notification to the near-infrared light illumination device 400. When the notification is received, the near-infrared light illumination device 400 may start irradiating near-infrared light. The same applies to the case where the first modified example, the second modified example, and the fourth modified example are combined.

[0177] <Second embodiment> Next, a second embodiment of the present disclosure will be described in detail with reference to the drawings. In this embodiment, the size of the pupil of the imaged eye region is estimated from an image captured by the imaging device 200, and the illuminance of the eye region is estimated based on the size of the pupil.

[0178] <<Configuration>> 9 is a block diagram showing an example of the configuration of an imaging system 2 of this embodiment. The imaging system 2 is the same as the imaging system 1 of the first embodiment, except for the differences described below.

[0179] The imaging system 2 includes a control device 101 instead of the control device 100. The control device 101 is the same as the control device 100 of the first embodiment, except for the differences described below.

[0180] In the control device 101, the estimation unit 120 is connected to the relationship storage unit 130. The estimation unit 120 of this embodiment is the same as the estimation unit 120 of the first embodiment, except for the differences described below.

[0181] The estimation unit 120 extracts the pupil area from the image acquired by the acquisition unit 110 (i.e., the input image). The estimation unit 120 may extract the pupil area according to any of various existing methods for extracting the pupil area. The estimation unit 120 estimates the size of the pupil in the eye area based on the imaging information (particularly, the camera parameters) of the imaging device 200. Specifically, the estimation unit 120 may estimate the pupil size based on the focus position of the imaging device 200, the angle of view of the imaging device 200, the size of the image captured by the imaging device 200 (i.e., the number of vertical pixels and the number of horizontal pixels), and the number of pixels of the diameter of the pupil area in the input image. The estimation unit 120 may read the illuminance-size relationship from the relationship storage unit 130. The estimation unit 120 estimates the illuminance of the eye area from the estimated pupil size based on the illuminance-size relationship.

[0182] <<Operation>> The control device 101 of this embodiment performs the same operations as the control device 100 of the first embodiment shown in FIG. 3, except for the following differences.

[0183] In this embodiment, in step S102, the estimation unit 120 extracts a pupil region from the acquired input image. The estimation unit 120 estimates the size of the extracted pupil based on the imaging information.

[0184] The operations of the control device 101 of this embodiment in steps other than step S102 are the same as the operations of the control device 100 of the first embodiment in steps to which the same reference numerals are assigned.

[0185] <<Effects>> This embodiment has the same effects as the first embodiment, for the same reasons that the effects of the first embodiment are obtained.

[0186] <Modification> The above-mentioned first to fifth modified examples can be applied to this embodiment.

[0187] <Third embodiment> Next, a third embodiment of the present disclosure will be described in detail with reference to the drawings.

[0188] <<Configuration>> First, the configuration of the control device 102 according to this embodiment will be described in detail with reference to the drawings.

[0189] Fig. 10 is a block diagram showing an example of the configuration of the control device 102 according to this embodiment. In the example shown in Fig. 10, the control device 102 includes an acquisition unit 110, an estimation unit 120, and a determination unit 140. The acquisition unit 110 acquires an input image including an eye region, which is a region of the eye portion. The estimation unit 120 estimates the illuminance of the eye portion from the acquired input image. The determination unit 140 determines the amount of light of the visible light illumination to irradiate the eye portion based on an illuminance-size relationship, which is a relationship between illuminance and pupil size, so that the size of the pupil of the eye portion satisfies a size condition.

[0190] <<Operation>> Next, the operation of the control device 102 of this embodiment will be described with reference to the drawings.

[0191] FIG. 11 is a flowchart showing an example of the operation of the control device 102 of this embodiment. In the example of the operation shown in FIG. 11, first, the acquisition unit 110 acquires an input image including an area of ​​the eye part (step S201). The acquisition unit 110 may acquire the input image from, for example, the above-mentioned imaging device 200 or an imaging device equivalent to the imaging device 200A. Next, the estimation unit 120 estimates the illuminance of the eye part from the acquired input image (step S202). The method by which the estimation unit 120 estimates the illuminance of the eye part may be the same as the method by which the estimation unit 120 of the first embodiment estimates the illuminance of the eye part. The method by which the estimation unit 120 estimates the illuminance of the eye part may be the same as the method by which the estimation unit 120 of the second embodiment estimates the illuminance of the eye part. Next, the determination unit 140 determines the amount of light of the visible light illumination such that the size of the pupil of the eye part satisfies the size condition (step S203). The visible light illumination may be, for example, the same device as the visible light illumination device 300 in the first and second embodiments. The method by which the determination unit 140 determines the amount of visible light illumination may be the same as the method by which the determination unit 140 in the first and second embodiments determines the amount of visible light illumination. In this case, the determination unit 140 may hold the above-mentioned illuminance-size relationship.

[0192] <<Effects>> This embodiment has the same effects as the first embodiment, for the same reasons that the effects of the first embodiment are obtained.

[0193] <Other embodiments> The control devices 100, 101, and 102 can be realized by a computer including a memory into which a program read from a storage medium is loaded and a processor that executes the program. The control devices 100, 101, and 102 can also be realized by dedicated hardware. The control devices 100, 101, and 102 can also be realized by a combination of the above-mentioned computer and dedicated hardware.

[0194] FIG. 12 is a diagram showing an example of a hardware configuration of a computer 1000 that can realize the control device (i.e., the control device 100, the control device 101, and the control device 102) according to the above-mentioned embodiment. The computer 1000 shown in FIG. 12 includes a processor 1001, a memory 1002, a storage device 1003, and an I / O (Input / Output) interface 1004. The computer 1000 can also access a storage medium 1005. The memory 1002 and the storage device 1003 are, for example, storage devices such as a RAM (Random Access Memory) and a hard disk. The storage medium 1005 is, for example, a storage device such as a RAM or a hard disk, a ROM (Read Only Memory), or a portable storage medium. The storage device 1003 may be the storage medium 1005. The processor 1001 can read and write data and programs from and to the memory 1002 and the storage device 1003. The processor 1001 can access, for example, the image capturing device 200, the image capturing device 200A, the image capturing device 200B, the visible light illumination device 300, the near-infrared light illumination device 400, the notification destination device 500, and the detection device 600 via the I / O interface 1004. The processor 1001 can access a storage medium 1005. The storage medium 1005 stores a program that causes the computer 1000 to operate as the control device according to the above-described embodiment.

[0195] The processor 1001 loads a program stored in the storage medium 1005, which causes the computer 1000 to operate as the control device according to the embodiment described above, into the memory 1002. The processor 1001 then executes the program loaded into the memory 1002, causing the computer 1000 to operate as the control device according to the embodiment described above.

[0196] The acquisition unit 110, the estimation unit 120, the determination unit 140, the control unit 150, and the notification unit 160 can be realized, for example, by a processor 1001 that executes a program loaded into a memory 1002. Furthermore, the relationship storage unit 130 and the image storage unit 170 can be realized by the memory 1002 or a storage device 1003 such as a hard disk device included in the computer 1000. Alternatively, some or all of the acquisition unit 110, the estimation unit 120, the relationship storage unit 130, the determination unit 140, the control unit 150, the notification unit 160, and the image storage unit 170 can be realized by a dedicated circuit that realizes the function of each unit.

[0197] <Additional Notes> Furthermore, some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0198] (Appendix 1) An acquisition means for acquiring an input image including an eye region, which is a region of an eye part; an estimation means for estimating an illuminance of the eye portion from the acquired input image; a determining means for determining an amount of illumination of visible light to be irradiated to the eye portion based on an illuminance-size relationship that is a relationship between illuminance and a size of a pupil, so that the size of the pupil of the eye portion satisfies a size condition; A control device comprising:

[0199] (Appendix 2) The estimation means detects a sclera area of ​​the eye area from the input image, and estimates the illuminance of the eye part based on information of pixels of the detected sclera area. 2. The control device according to claim 1.

[0200] (Appendix 3) The estimation means detects a pupil area of ​​the eye area from the input image, estimates a pupil size of the eye area based on the detected pupil area, and estimates illuminance of the eye area based on the estimated pupil size and the illuminance-size relationship. 2. The control device according to claim 1.

[0201] (Appendix 4) The determining means determines the amount of light of the illumination so that the illuminance of the eye area satisfies a glare condition based on a glare index determined based on the illuminance. 4. The control device according to claim 1 ,

[0202] (Appendix 5) The estimation means estimates an iris color based on the acquired input image, The determining means selects the glare condition according to the estimated iris color. 5. The control device according to claim 4.

[0203] (Appendix 6) The estimation means estimates an iris color based on the acquired input image, The determining means selects an illuminance size relationship based on the estimated iris color from a plurality of illuminance size relationships according to iris colors, and determines the amount of light of the illumination based on the selected illuminance size relationship. 5. The control device according to claim 1 .

[0204] (Appendix 7) the determining means determines whether it is possible to control the amount of light of the illumination so that the size of the pupil satisfies the size condition, based on the estimated illuminance and the illuminance-size relationship; a notification means for notifying a user when it is determined that the amount of light emitted by the illumination cannot be controlled so that the size of the pupil satisfies the size condition; 7. The control device according to any one of claims 1 to 6, further comprising:

[0205] (Appendix 8) A control means for controlling the lighting so that the lighting irradiates visible light with the determined amount of light of the lighting. 8. The control device according to any one of claims 1 to 7, further comprising:

[0206] (Appendix 9) A control device according to any one of appendix 1 to 8; The illumination and an imaging device that captures the input image; 2. An imaging system comprising:

[0207] (Appendix 10) Obtain an input image including an eye region, the eye region being a region of an eye portion; Estimating an illuminance of the eye portion from the acquired input image; determining an amount of illumination of visible light to irradiate the eye portion based on an illuminance-size relationship that is a relationship between illuminance and a pupil size, so that the size of the pupil of the eye portion satisfies a size condition; Control methods.

[0208] (Appendix 11) A sclera area of ​​the eye area is detected from the input image, and the illuminance of the eye portion is estimated based on pixel information of the detected sclera area. 11. The control method of claim 10.

[0209] (Appendix 12) Detecting a pupil area of ​​the eye area from the input image, estimating a pupil size of the eye area based on the detected pupil area, and estimating an illuminance of the eye area based on the estimated pupil size and the illuminance-size relationship. 11. The control method of claim 10.

[0210] (Appendix 13) The amount of light of the illumination is determined so that the illuminance of the eye area satisfies a glare condition based on the illuminance of the eye area. 13. A control method according to any one of appendixes 10 to 12.

[0211] (Appendix 14) Estimating an iris color based on the acquired input image; Selecting the glare condition according to the estimated iris color 14. The control method of claim 13.

[0212] (Appendix 15) Estimating an iris color based on the acquired input image; From a plurality of illuminance size relationships according to iris colors, the illuminance size relationship is selected based on the estimated iris color, and the amount of light of the illumination is determined based on the selected illuminance size relationship. 14. A control method according to any one of claims 10 to 13.

[0213] (Appendix 16) determining whether it is possible to control the amount of light of the illumination so that the size of the pupil satisfies the size condition based on the estimated illuminance and the illuminance-size relationship; When it is determined that the amount of light of the illumination cannot be controlled so that the size of the pupil satisfies the size condition, a notification is given. 16. A control method according to any one of appendix 10 to 15.

[0214] (Appendix 17) The lighting is controlled so that the lighting emits visible light with the determined amount of light of the lighting. 17. A control method according to any one of appendix 10 to 16.

[0215] (Appendix 18) An acquisition process for acquiring an input image including an eye region, the eye region being a region of an eye part; an estimation process of estimating an illuminance of the eye portion from the acquired input image; a determination process for determining an amount of visible light illumination to irradiate the eye portion based on an illuminance-size relationship that is a relationship between illuminance and a pupil size, so that the size of the pupil of the eye portion satisfies a size condition; A storage medium that stores a program that causes a computer to execute the above.

[0216] (Appendix 19) The estimation process detects a sclera area of ​​the eye area from the input image, and estimates the illuminance of the eye part based on information of pixels of the detected sclera area. 19. The storage medium of claim 18.

[0217] (Appendix 20) The estimation process detects a pupil area of ​​the eye area from the input image, estimates a pupil size of the eye area based on the detected pupil area, and estimates illuminance of the eye area based on the estimated pupil size and the illuminance-size relationship. 19. The storage medium of claim 18.

[0218] (Appendix 21) The determination process determines the amount of light of the illumination so that the illuminance of the eye area satisfies a glare condition based on the illuminance. 21. A storage medium according to any one of appendices 18 to 20.

[0219] (Appendix 22) The estimation process estimates an iris color based on the acquired input image, The determination process selects the glare condition according to the estimated iris color. 22. A storage medium as described in claim 21.

[0220] (Appendix 23) The estimation process estimates an iris color based on the acquired input image, The determination process includes selecting an illuminance size relationship based on the estimated iris color from a plurality of illuminance size relationships according to iris colors, and determining the amount of light of the illumination based on the selected illuminance size relationship. 22. A storage medium according to any one of claims 18 to 21.

[0221] (Appendix 24) the determination process includes determining whether it is possible to control the amount of light of the illumination so that the size of the pupil satisfies the size condition, based on the estimated illuminance and the illuminance-size relationship; The program is A notification process for notifying a user when it is determined that the amount of light emitted by the illumination cannot be controlled so that the pupil size satisfies the size condition. 24. The storage medium according to any one of appendices 18 to 23, further causing a computer to execute the steps.

[0222] (Appendix 25) The program is A control process for controlling the lighting so that the lighting emits visible light with the determined amount of light of the lighting. 25. The storage medium according to any one of claims 18 to 24, further causing the computer to execute the steps.

[0223] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0224] 1. Imaging system 1A Imaging System 1B Imaging system 1C Imaging System 2. Imaging system 100 Control device 101 Control device 102 Control device 110 Acquisition Department 120 Estimation part 130 Relationship Memory Unit 140 Decision Section 150 Control section 160 Notification Department 170 Image storage unit 200 Imaging device 200A Imaging Device 200B Imaging device 300 Visible light illumination device 400 Near-infrared illumination device 500 Notification destination device 600 Detection Device 1000 Computers 1001 Processor 1002 Memory 1003 Storage device 1004 I / O Interface 1005 Storage medium

Claims

1. an acquisition means for acquiring a first input image including an eye region, the eye region being an area of ​​an eye portion of a subject; a determining means for determining an amount of illumination for illuminating the eye region based on a pupil size of a subject included in the first input image; Equipped with The acquisition means acquires a second input image including the eye region illuminated with the amount of light determined by the determination means; means for authenticating an object using said second input image; The information processing device further comprises:

2. an estimation means for detecting a pupil area of ​​the eye area from the first input image, estimating a pupil size of the eye area based on the detected pupil area, and estimating an illuminance of the eye area based on the estimated pupil size and an illuminance-size relationship which is a relationship between illuminance and pupil size. The information processing device according to claim 1 .

3. The estimation means estimates an iris color based on the acquired first input image, The determining means selects a glare condition according to the estimated color of the iris, and determines the amount of light of the illumination such that the illuminance of the eye region satisfies the glare condition, the glare index being determined based on the illuminance. The information processing device according to claim 2 .

4. The estimation means estimates an iris color based on the acquired first input image, The determining means selects an illuminance size relationship based on the estimated iris color from a plurality of illuminance size relationships according to iris colors, and determines the amount of light of the illumination based on the selected illuminance size relationship. The information processing device according to claim 2 .

5. a determining means for determining whether it is possible to control the amount of light of the illumination so that the size of the pupil satisfies a size condition, based on the estimated illuminance and the illuminance-size relationship; a notification means for notifying the user when it is determined that the amount of light of the illumination cannot be controlled so that the size of the pupil satisfies the size condition; The information processing device according to claim 2 , further comprising:

6. A control means for controlling the lighting so that the lighting irradiates visible light with the determined amount of light of the lighting. The information processing device according to claim 1 , further comprising:

7. an estimation means for detecting a sclera area of ​​the eye area from the first input image, and estimating an illuminance of the eye part based on information of the detected sclera area; the determining means determines the amount of illumination of the visible light to irradiate the eye portion such that a size of a pupil of the eye portion satisfies a predetermined condition. The information processing device according to claim 1 .

8. An estimation means for estimating an illuminance of the eye portion from the acquired first input image, The determining means determines the amount of light of the illumination illuminating the eye portion such that an illuminance of the eye area satisfies a glare condition based on a glare index determined based on the illuminance. The information processing device according to claim 1 .

9. obtaining a first input image including an eye region that is a region of an eye portion of a subject; determining an amount of illumination to illuminate the eye region based on a pupil size of a subject included in the first input image; acquiring a second input image including the eye region illuminated with the determined amount of light; authenticating the subject using the second input image; Information processing methods.

10. A program, comprising: an acquisition process for acquiring a first input image including an eye region, the eye region being an area of ​​an eye portion of a subject; a determination process for determining an amount of light to be irradiated onto the eye region when irradiating an object having a first pupil size based on a pupil size of the object included in the first input image; Run the following on your computer: the acquiring process acquiring a second input image including the eye region illuminated with the amount of light determined in the determining process; The program is an authentication process for authenticating an object using the second input image; A program that causes a computer to further execute the above.

Citation Information

Patent Citations

  • Individual authenticating device

    JP2003016434A

  • Iris image pickup device

    JP2004261515A

  • Display apparatus and control method of the same

    JP2014155005A

  • Iris data registration device, iris data registration method and iris data registration program

    JP2018073369A

  • Biological information measurement device and input device using same

    WO2014073645A1