Information processing device and information processing method

Abstract

This information processing device comprises an acquisition unit and a control unit. The acquisition unit acquires: the positional relationship between a subject of biometric authentication and an imaging device that captures an image of the subject; ambient light information that includes at least one of illumination light radiated at the subject by an illumination device and ambient light in the environment of the subject; and brightness information for the subject in the image captured by the imaging device. The control unit controls at least one of the illumination device and the imaging device on the basis of the positional relationship, the ambient light information, and the brightness information.

Description

[Technical Field]

[0001] This disclosure relates to an information processing device and Information Processing By law Regarding. [Background technology]

[0002] Patent Document 1 describes an authentication system that authenticates a target person using a face image generated by capturing an image of the target person's face. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2015-136938 Summary of the Invention [Problem to be solved by the invention]

[0004] This disclosure aims to improve upon the techniques described in the prior art documents cited above. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided an acquisition unit that acquires a positional relationship between a subject of biometric authentication and an image capture device that captures an image of the subject, ambient light information including at least one of illumination light irradiated onto the subject by an illumination device and ambient light around the subject, and luminance information of the subject in an image captured by the image capture device; and a control unit that controls at least one of the illumination device and the image capture device based on the positional relationship, the ambient light information, and the luminance information. a learning unit that learns the relationship between a change in the imaging angle of the imaging device, a change in the luminance information, the presence or absence of an article worn by the subject, the presence or absence of a shadow of the article on a body part of the subject, and a reflection state of the illumination light on the article; Equipped with the control unit controls at least one of the lighting device and the imaging device based on the learning result of the learning unit. An information processing device is provided.

[0006] According to another aspect of the present disclosure, there are provided a method for obtaining a positional relationship between a subject of biometric authentication and an image capturing device capturing an image of the subject, ambient light information including at least one of illumination light irradiated onto the subject by an illumination device and ambient light around the subject, and luminance information of the subject; and a method for controlling at least one of the illumination device and the image capturing device based on the positional relationship, the ambient light information, and the luminance information. learning a relationship between a change in the imaging angle of the imaging device, a change in the luminance information, the presence or absence of an article worn by the subject, the presence or absence of a shadow of the article on a body part of the subject, and a reflection state of the illumination light on the article; Equipped with In the controlling step, at least one of the illumination device and the imaging device is controlled based on a learning result in the learning step. A method for processing information is provided.

[0007] According to another aspect of the present disclosure, a program for causing a computer to execute the steps of acquiring a positional relationship between a subject of biometric authentication and an imaging device that images the subject, ambient light information including at least one of illumination light irradiated by an illumination device on the subject and ambient light around the subject, and luminance information of the subject in an image captured by the imaging device, and controlling at least one of the illumination device and the imaging device based on the positional relationship, the ambient light information, and the luminance information. Mu Provided.

[0008] According to another aspect of the present disclosure, there is provided an information processing device including: an acquisition unit that acquires ambient light information representing factors that may affect the illuminance of a subject of biometric authentication; and a control unit that controls, based on the ambient light information, at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject.

[0009] According to another aspect of the present disclosure, there is provided an information processing method comprising the steps of: acquiring ambient light information representing factors that may affect the illuminance of a subject of biometric authentication; and controlling, based on the ambient light information, at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject.

[0010] According to another aspect of the present disclosure, there is provided a recording medium having recorded thereon a program for causing a computer to execute the steps of acquiring ambient light information representing factors that may affect the illuminance of a subject of biometric authentication, and controlling at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject based on the ambient light information. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the overall configuration of an authentication system according to a first embodiment. [Figure 2] 2 is a block diagram showing an example of a hardware configuration of the authentication device according to the first embodiment. FIG. [Figure 3] FIG. 1 is a schematic diagram illustrating an authentication system according to a first embodiment. [Figure 4] 1 is a front view showing the appearance of an authentication device according to a first embodiment. [Figure 5] 1 is a perspective view illustrating the internal structure of an authentication device according to a first embodiment. [Figure 6] FIG. 3 is a diagram illustrating an example of information stored in an ambient light information database according to the first embodiment. [Figure 7] FIG. 3 is a diagram showing an example of information stored in a registrant information database according to the first embodiment. [Figure 8] 1 is a functional block diagram showing the overall configuration of an authentication device according to a first embodiment. [Figure 9] 3 is a flowchart showing an outline of processing executed by the authentication device according to the first embodiment. [Figure 10] 5A to 5C are schematic diagrams illustrating a method for changing the imaging angle and imaging range of the second camera in the authentication device according to the first embodiment. [Figure 11] 5A to 5C are schematic diagrams illustrating a method for changing the imaging angle and imaging range of the second camera in the authentication device according to the first embodiment. [Figure 12] FIG. 10 is a schematic diagram illustrating an authentication system according to a second embodiment. [Figure 13]FIG. 10 is a block diagram showing an example of the overall configuration of an authentication system according to a second embodiment. [Figure 14] FIG. 11 is a diagram illustrating an example of information stored in an ambient light information database according to the second embodiment. [Figure 15] FIG. 10 is a diagram showing an example of information stored in a layout information database according to the second embodiment. [Figure 16] 10 is a flowchart showing an outline of processing executed by an authentication device according to a second embodiment. [Figure 17] 10 is a flowchart showing an outline of processing executed by an authentication device according to a third embodiment. [Figure 18] FIG. 10 is a block diagram showing an example of a hardware configuration of an authentication device according to a fourth embodiment. [Figure 19] 10 is a flowchart showing an outline of processing executed by an authentication device according to a fourth embodiment. [Figure 20] 10 is a flowchart showing an outline of processing executed by an authentication device according to a fifth embodiment. [Figure 21] 13 is a flowchart showing an outline of processing executed by an authentication device according to a sixth embodiment. [Figure 22] FIG. 13 is a functional block diagram showing the overall configuration of an authentication device according to a seventh embodiment. [Figure 23] 13 is a flowchart showing an outline of processing executed by an authentication device according to the seventh embodiment. [Figure 24] 13 is a flowchart showing an outline of processing executed by an authentication device according to the seventh embodiment. [Figure 25] 13 is a flowchart showing an outline of processing executed by an authentication device according to the seventh embodiment. [Figure 26] FIG. 13 is a functional block diagram showing the overall configuration of an authentication device according to an eighth embodiment. [Figure 27] FIG. 13 is a schematic diagram illustrating a neural network used in a learning process according to the eighth embodiment. [Figure 28] 13 is a flowchart showing an outline of processing executed by an authentication device according to the eighth embodiment. [Figure 29] 13 is a flowchart showing an outline of processing executed by an authentication device according to the eighth embodiment. [Figure 30] 13 is a flowchart showing an outline of processing executed by an authentication device according to a ninth embodiment. [Figure 31] FIG. 20 is a functional block diagram showing the overall configuration of an information processing device according to a tenth embodiment. [Figure 32] FIG. 22 is a functional block diagram showing the overall configuration of an information processing device according to an eleventh embodiment. [Figure 33] 10A and 10B are schematic diagrams illustrating a method for changing the imaging angle and imaging range of the second camera in the authentication device according to a modified embodiment. [Figure 34] 10A and 10B are schematic diagrams illustrating a method for changing the imaging angle and imaging range of the second camera in the authentication device according to a modified embodiment. [Figure 35] FIG. 10 is a diagram showing an example of the positional relationship between an imaging device, a lighting device, and a person to be authenticated in a modified embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. In the drawings, similar or corresponding elements are designated by the same reference numerals, and descriptions thereof may be omitted or simplified. [First embodiment]

[0013] 1 is a block diagram showing an example of the overall configuration of an authentication system 1 according to the first embodiment. The authentication system 1 is composed of an authentication device 10, an authentication server 20, and a gate device 30. Each device is connected to networks NW1 and NW2 such as a LAN (Local Area Network) or the Internet.

[0014] The authentication system 1 acquires biometric information of a person to be authenticated (hereinafter referred to as "person to be authenticated") and performs biometric authentication by comparing the acquired biometric information with pre-registered registered biometric information.

[0015] The term "biometric information" in the first embodiment refers to a face image and features extracted from the face image, but the biometric information is not limited to a face image and facial features. That is, the authentication system 1 may perform biometric authentication using biometric images other than a face image (such as an iris image, a fingerprint image, a palm print image, a vein image, a gait image, and an auricle image) and features as the biometric information of the person to be authenticated.

[0016] The authentication system 1 can be applied to, for example, identity verification for immigration at airports, identity verification at government agencies, identity verification for entry and exit at factories and offices, identity verification for entry and exit at event venues, etc. The authentication system 1 can also be applied to payments based on biometric authentication. The authentication system 1 can be applied to authentication in various technical fields.

[0017] The authentication device 10 is a computer that captures an image of a person to be authenticated who is present in an authentication area and outputs the biometric image to the authentication server 20. In the first embodiment, the "authentication area" refers to a three-dimensional space of a predetermined range located in front of the authentication device 10. Furthermore, the "biometric image" in the first embodiment is a facial image.

[0018] Furthermore, the authentication device 10 controls at least one of the second camera 107B and the lighting device 108 (described later) to capture an image of the person to be authenticated, based on information (hereinafter referred to as "ambient light information") that predefines ambient light irradiated onto the person to be authenticated from various light sources such as the sun. Specific examples of ambient light information include the illuminance and irradiation direction of ambient light. In the first embodiment, ambient light includes illumination light unless otherwise specified.

[0019] The authentication server 20 is a computer that executes biometric authentication. The authentication server 20 includes an authentication engine 21, an ambient light information database 22, and a registrant information database 23. The authentication engine 21 executes a process of matching a biometric image (or feature amount) of the person to be authenticated captured by the authentication device 10 with a registered biometric image (or feature amount) of the registrant pre-registered in the registrant information database 23, and performs biometric authentication of the person to be authenticated based on the matching result.

[0020] The ambient light information database 22 stores ambient light information for each point where biometric authentication is performed. The registrant information database 23 stores information on registrants who are permitted to pass through the gate device 30.

[0021] The gate device 30 is a passage control device that opens and closes a gate (not shown) based on control information from the authentication device 10, and controls the passage of people. When the authentication device 10 successfully authenticates a person, the gate device 30 transitions from a closed standby state that blocks the passage of people to an open state that allows the passage of people. The type of gate is not particularly limited, and examples include a flapper gate in which flappers installed on one or both sides of the passage open and close, and a turnstile gate in which three bars rotate.

[0022] 2 is a block diagram showing an example of the hardware configuration of the authentication device 10 according to the first embodiment. The authentication device 10 is a computer that performs calculations, control, and storage, and includes a processor 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, a storage 104, a communication I / F (Interface) 105, a display 106, a first camera 107A, a second camera 107B, a lighting device 108, a distance sensor 109, and a motor 110. The devices are connected to each other via a bus, wiring, a driving device, etc. (not shown).

[0023] The processor 101 has a function of performing predetermined calculations in accordance with programs stored in the ROM 103, storage 104, etc., and controlling each part of the authentication device 10. Furthermore, as the processor 101, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), etc. may be used. Furthermore, one of the above examples may be used, or multiple may be used in parallel.

[0024] The RAM 102 is made up of a volatile storage medium and provides a temporary memory area necessary for the operation of the processor 101. The RAM 102 may be, for example, a D-RAM (Dynamic RAM). The ROM 103 is made up of a non-volatile storage medium and stores necessary information such as programs used in the operation of the authentication device 10. The ROM 103 may be, for example, a P-ROM (Programmable ROM).

[0025] The storage 104 is configured from a nonvolatile storage medium, and stores data and programs for operating the authentication device 10. The storage 104 is configured from, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0026] The communication I / F 105 is a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), 4G, or 5G, and is a module for communicating with other devices.

[0027] The processor 101 loads a program stored in the ROM 103, the storage 104, etc. into the RAM 102 and executes it.

[0028] The display 106 is a display device that displays moving images, still images, characters, etc. The display 106 displays, for example, guide information related to authentication and authentication results. The display 106 may be a liquid crystal display, an OLED (Organic Light Emitting Diode) display, or the like.

[0029] The first camera 107A is an imaging device that captures an image of the entire authentication area. The first camera 107A includes a light receiving element configured to be sensitive to infrared light. As the first camera 107A, a digital camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, or the like is used so as to be suitable for image processing in the authentication device 10.

[0030] Second camera 107B is an imaging device that captures an image of a predetermined body part of the person to be authenticated. Second camera 107B includes a light receiving element configured to be sensitive to infrared light. As second camera 107B, a digital camera using a CMOS image sensor, a CCD image sensor, or the like is used. First camera 107A and second camera 107B differ from each other in imaging target, resolution, etc. The imaging range of first camera 107A may include the imaging range of second camera 107B.

[0031] Furthermore, in this embodiment, first camera 107A and second camera 107B are infrared cameras, but the combination pattern of first camera 107A and second camera 107B is not limited to this. For example, first camera 107A may be a visible light camera and second camera 107B may be an infrared camera. Also, first camera 107A may be an infrared camera and second camera 107B may be a visible light camera. Similarly, both first camera 107A and second camera 107B may be visible light cameras.

[0032] Illumination device 108 includes a light-emitting element that emits infrared light, such as an infrared LED. The wavelength of the infrared light emitted from illumination device 108 may be, for example, in the near-infrared region of about 800 nm. The timing of illumination light irradiation by illumination device 108 is synchronized with the timing of image capture by first camera 107A and second camera 107B. However, the timing of illumination light irradiation by illumination device 108 may not be synchronized with the timing of image capture by each camera.

[0033] In the present embodiment, the illumination device 108 is an infrared light emitting device provided in correspondence with the second camera 107B, which is an infrared camera, but is not limited to this. For example, the illumination device 108 may be a visible light emitting device. Furthermore, a plurality of illumination devices 108 may be provided to emit infrared light and visible light, respectively.

[0034] The distance sensor 109 is a measuring device that measures the distance from the authentication device 10 to the person to be authenticated. The distance sensor 109 may be capable of optically measuring the distance from the authentication device 10 to the person to be authenticated. Examples of the distance sensor 109 that can optically measure the distance include a TOF (Time Of Flight) sensor, a triangulation sensor, a LiDAR (Light Detection and Ranging) sensor, etc. Furthermore, the distance sensor 109 may also be a proximity sensor that detects the approach of an object without contact.

[0035] Motor 110 is a driving device that drives an object connected to a rotation shaft (not shown). In the first embodiment, second camera 107B is connected to the rotation shaft of motor 110. Note that second camera 107B does not have to be directly connected to the rotation shaft of motor 110. For example, second camera 107B may be indirectly connected to the rotation shaft via a gear, a belt, or the like, and driven by motor 110.

[0036] The hardware configuration shown in Fig. 2 is an example, and other devices may be added, or some devices may not be provided. Also, some devices may be replaced with other devices having similar functions. Also, some functions of the first embodiment may be provided by other devices via a network, or the functions of the first embodiment may be distributed and realized among multiple devices. The illustrated hardware configuration can be modified as appropriate.

[0037] FIG. 3 is a schematic diagram illustrating an authentication system 1 according to a first embodiment. As shown in FIG. 3, the authentication system 1 is a walk-through biometric authentication system that uses a first camera 107A and a second camera 107B to capture images of a person to be authenticated who moves within an authentication area toward the authentication device 10. Note that FIG. 3 illustrates an example in which biometric authentication is performed three times on the person to be authenticated, in the order of points P3, P2, and P1, but the points at which biometric authentication is performed are not limited to these. Distance D is the distance from position P0 of the authentication device 10 to point P1 where person T to be authenticated is standing.

[0038] Fig. 4 is a front view showing the appearance of the authentication device 10 according to the first embodiment. In Fig. 4, the positional relationship of the components that make up the authentication device 10 is explained using a three-dimensional coordinate system consisting of an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other. The X-axis and Y-axis are axes on a horizontal plane. The X-axis and Y-axis are orthogonal to each other. The Z-axis is an axis that is orthogonal to the horizontal plane.

[0039] 4, a display 106, a first camera 107A, a distance sensor 109, and a second camera 107B are arranged from top to bottom along the center line of the authentication device 10. In addition, a pair of lighting devices 108 are arranged on both sides of the second camera 107B.

[0040] FIG. 5 is a perspective view illustrating the internal structure of the authentication device 10 according to the first embodiment. As shown in FIG. 5, the second camera 107B and the lighting device 108 are disposed in a case 112 together with the motor 110. The first camera 107A has a wider angle of view than the second camera 107B. The first camera 107A is not provided in the case 112 to maintain a constant imaging angle. The second camera 107B and the lighting device 108 are connected to a rotation axis A of the motor 110. Therefore, the second camera 107B and the lighting device 108 rotate together with the rotation axis A. The optical axis Ax of the second camera 107B changes up and down due to the rotation about the rotation axis A. The imaging angle of the second camera 107B in the first embodiment refers to the angle of the optical axis Ax with respect to the horizontal plane.

[0041] As motor 110 is driven, second camera 107B rotates in the direction of arrow UP or arrow DW around rotation axis A. When second camera 107B rotates in the direction of arrow UP, the imaging range of second camera 107B moves upward. Conversely, when second camera 107B rotates in the direction of arrow DW, the imaging range of second camera 107B moves downward.

[0042] FIG. 6 is a diagram illustrating an example of information stored in the ambient light information database 22 according to the first embodiment. The ambient light information database 22 includes data items such as the distance from the authentication device 10 to the receiving point of the ambient light, the light source of the ambient light, the illuminance of the ambient light, and the direction of illumination of the ambient light. The light source is the sun, the lighting device 108, and a lighting device other than the lighting device 108 of the authentication device 10. By referencing the ambient light information based on the distance, it is possible to determine, for example, the illuminance at which sunlight and illumination light from the lighting device 108 will irradiate the authentication subject at a point two meters ahead of the authentication device 10. It is also possible to determine how the illuminance of the illumination light changes depending on the distance. The ambient light information is generated in advance based on, for example, measurements actually taken using an illuminance sensor in the authentication area or predicted values ​​calculated by computer simulation.

[0043] 7 is a diagram showing an example of information stored in the registrant information database 23 according to the first embodiment. The registrant information database 23 includes, as data items, a registrant ID for identifying a registrant, name, age, sex, registered biometric image, feature amount, etc.

[0044] 8 is a functional block diagram showing the overall configuration of an authentication device 10 according to the first embodiment. The authentication device 10 includes a face detection unit 11, a distance acquisition unit 12, an ambient light information acquisition unit 13, a luminance information acquisition unit 14, and a control unit 15.

[0045] The face detection unit 11 analyzes the captured image of the authentication area captured by the first camera 107A and detects the face of the person to be authenticated from the captured image. In the first embodiment, the face detection unit 11 detects the position and size of the face of the person to be authenticated from the captured image.

[0046] Distance acquisition unit 12 acquires the distance from authentication device 10 to the person to be authenticated based on the detection signal of distance sensor 109. In the first embodiment, second camera 107B and lighting device 108 are provided inside the housing of authentication device 10, and therefore the positional relationship between second camera 107B, lighting device 108, and the person to be authenticated can be specified by the distance from authentication device 10 to the person to be authenticated. In other words, the positional relationship between second camera 107B, lighting device 108, and the person to be authenticated in the first embodiment corresponds to the positional relationship between authentication device 10 and the person to be authenticated.

[0047] Furthermore, in the first embodiment, information such as the illuminance and irradiation direction of ambient light is pre-associated with distance, i.e., the positional relationship between the authentication device 10 (camera) and the person to be authenticated, in the ambient light information database 22. Therefore, the ambient light information acquisition unit 13 refers to the ambient light information database 22 based on the distance from the authentication device 10 to the person to be authenticated, and acquires the ambient light information corresponding to that distance. For example, the ambient light information may differ between a first point that is 3 meters away from the authentication device 10 and a second point that is 2 meters away from the authentication device 10.

[0048] The luminance information acquisition unit 14 analyzes the captured image of the authentication area captured by the first camera 107A or the biometric image of the person to be authenticated captured by the second camera 107B, and acquires luminance information about a predetermined body part of the person to be authenticated in the image. Examples of the body part from which luminance information is acquired include the face and eyes of the person to be authenticated. The luminance of a body part may be, for example, the average luminance value of a pixel group in the skin area of ​​the face.

[0049] The control unit 15 controls at least one of the second camera 107B (authentication device 10) and the person to be authenticated, ambient light information including at least one of the illumination light irradiated by the illumination device 108 on the person to be authenticated and the ambient light surrounding the person to be authenticated, and luminance information of the person to be authenticated in the captured image captured by the second camera 107B. The control unit 15 can, for example, determine the skin color of the person to be authenticated based on the luminance information and adjust the illuminance and wavelength of the illumination light irradiated from the second camera 107B or the imaging conditions of the second camera 107B. Similarly, the control unit 15 may adjust the illuminance and wavelength of the illumination light based on the eye color of the person to be authenticated. This ensures that the luminance of the biometric image captured by the second camera 107B falls within a predetermined range suitable for biometric authentication. Performing biometric authentication using a biometric image captured at appropriate luminance can improve the authentication accuracy of biometric authentication.

[0050] The processor 101 loads a program stored in the ROM 103, storage 104, etc. into the RAM and executes it. As a result, the processor 101 realizes the functions of the face detection unit 11, distance acquisition unit 12, ambient light information acquisition unit 13, luminance information acquisition unit 14, control unit 15, etc.

[0051] FIG. 9 is a flowchart showing an outline of the processing executed by the authentication device 10 according to the first embodiment.

[0052] In step S101, the authentication device 10 captures an image of the entire authentication area by using the first camera 107A to capture an image of the authentication area in front of the device.

[0053] In step S102, the authentication device 10 performs face detection on the captured image. Specifically, the authentication device 10 detects an area that matches the shape pattern of the person's face from the captured image. When the authentication device 10 detects the face of the person to be authenticated, the process proceeds to step S103.

[0054] In step S103, the authentication device 10 analyzes the captured image to obtain luminance information of the face area of ​​the person to be authenticated, thereby enabling the authentication device 10 to obtain the skin color of the face area of ​​the person to be authenticated.

[0055] In step S104, the authentication device 10 measures the horizontal distance from the authentication device 10 to the person to be authenticated based on the detection signal from the distance sensor 109.

[0056] In step S105, the authentication device 10 identifies the position of the face in three-dimensional space based on the coordinates of the face in the captured image and the distance measured in step S104. By identifying the actual position of the face, it becomes possible to align the orientation of the second camera 107B and the lighting device 108 with the person to be authenticated. By aligning the orientation of the second camera 107B and the lighting device 108 with the person to be authenticated, it is possible to acquire a biometric image more suitable for authentication.

[0057] In step S106, the authentication device 10 refers to the ambient light information database 22 based on the distance measured in step S104, and acquires ambient light information at that distance. For example, if the person to be authenticated is two meters away from the authentication device 10, ambient light information corresponding to two meters is acquired.

[0058] In step S107, the authentication device 10 controls the second camera 107B and the lighting device 108 based on the ambient light information and the brightness information. Specifically, the authentication device 10 controls the exposure conditions, gain, imaging timing, imaging angle, etc. of the second camera 107B. The authentication device 10 also controls the illuminance, wavelength, irradiation direction, irradiation timing, etc. of the illumination light emitted from the lighting device 108.

[0059] 10 and 11 are schematic diagrams illustrating the imaging angle and imaging range of second camera 107B according to the first embodiment. In FIG. 10, the position of the face of person T1 to be authenticated is higher in the vertical direction than the center position of second camera 107B. Therefore, the imaging angle of second camera 107B is set so that the optical axis Ax of second camera 107B is directed upward from the horizontal. The imaging range R1 of second camera 107B in FIG. 10 includes the face of person T1 to be authenticated.

[0060] In Fig. 11, the height of the person to be authenticated T2 is shorter than the height of the person to be authenticated T1. Furthermore, the position of the face of the person to be authenticated T2 is lower in the vertical direction than the center position of the second camera 107B. Therefore, the imaging angle is set so that the optical axis Ax of the second camera 107B is directed downward from the horizontal direction. Therefore, the imaging range R2 of the second camera 107B in Fig. 11 is moved vertically downward from the imaging range R1 shown in Fig. 11. The face of the person to be authenticated T2 is included in the imaging range R2.

[0061] In step S108, the authentication device 10 captures an image of the face of the person to be authenticated using the second camera 107B, and generates a face image.

[0062] In step S109, the authentication device 10 transmits the facial image generated in step S108 to the authentication engine 21, and causes the authentication engine 21 to perform facial authentication. The authentication engine 21 performs facial authentication by comparing the received facial image with the registered facial image of the registrant registered in the registrant information database 23, and then transmits the authentication result to the authentication device 10.

[0063] In step S110, the authentication device 10 records the distance when the face image was captured and the authentication result received from the authentication engine 21 in association with the image capture time in a storage device such as the RAM 102 or the storage 104. This provides time-series data on the distance and authentication result of the person to be authenticated moving within the authentication area.

[0064] In step S111, the authentication device 10 determines whether to complete authentication of the person to be authenticated. For example, the authentication device 10 may determine whether to complete authentication based on whether the number of authentication attempts has reached a predetermined number or whether authentication has been successful one or more times.

[0065] Here, if the authentication device 10 determines that the authentication of the person to be authenticated is to be completed (step S111: YES), the process proceeds to step S112. On the other hand, if the authentication device 10 determines that the authentication of the person to be authenticated is not to be completed (step S111: NO), the process returns to step S101. As a result, the authentication process of the person to be authenticated who moves in the authentication area toward the authentication device 10 is repeatedly executed.

[0066] In step S112, the authentication device 10 determines whether or not to permit the person to be authenticated to pass through the gate device 30 based on the authentication result for the person to be authenticated that is recorded in the storage device.

[0067] Here, if the authentication device 10 determines that the authentication subject is permitted to pass through the gate device 30 (step S112: YES), it opens the gate (step S113) and ends the process.

[0068] On the other hand, if the authentication device 10 determines that the authentication target is not permitted to pass through the gate device 30 (step S112: NO), it displays information about an authentication error on the display 106 (step S114) and ends the process.

[0069] Conventional authentication systems such as those described in Patent Document 1 do not take into consideration the influence of ambient light at the point where a biometric image is captured. As a result, the captured biometric image may not be suitable for biometric authentication. In contrast, the authentication system 1 according to the first embodiment can control the second camera 107B and the lighting device 108 based on ambient light information predefined for each distance and brightness information acquired from the captured image. This allows the second camera 107B to appropriately adjust the conditions for capturing an image of the face of the person to be authenticated, thereby obtaining a face image suitable for face authentication performed by the authentication engine 21. [Second embodiment]

[0070] The authentication system 1 according to the second embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and explanations of common parts will be omitted or simplified.

[0071] Fig. 12 is a schematic diagram illustrating an authentication system 1 according to a second embodiment. Fig. 12 shows that face authentication is performed on an authentication target T standing at point P1. An exterior wall 40 located behind the authentication device 10 is provided with a window 41 at a position higher than the authentication device 10, through which ambient light can enter.

[0072] 13 is a block diagram showing an example of the overall configuration of an authentication system 1 according to the second embodiment. Unlike the first embodiment, the authentication device 10 includes a face detection unit 11, an acquisition unit 16, and a control unit 15. The authentication server 20 further includes a layout information database 24.

[0073] The acquisition unit 16 acquires ambient light information that indicates factors that may affect the illuminance of a person to be biometrically authenticated. In the second embodiment, the acquisition unit 16 acquires ambient light information including the date and time when an image of the person to be authenticated is captured and layout information of a facility where an authentication area is located. The layout information includes optical characteristics, shapes, sizes, and arrangements of structures located in the facility. The layout information is defined based on coordinates in a three-dimensional space.

[0074] FIG. 14 is a diagram showing an example of information stored in the ambient light information database 22 according to the second embodiment. The ambient light information database 22 includes data items such as a location ID, date and time, weather, light source, illuminance, and irradiation direction. For example, if the date and time is 2:00 p.m. on August 1st, ambient light information including the illuminance and irradiation direction of sunlight irradiating the authentication area at location P1 can be obtained from the ambient light information database 22 using the date and time as a key. The illuminance and irradiation direction of ambient light differ depending on the type of light source, the date and time of image capture, and the weather. In the second embodiment, it is assumed that information such as the illuminance and irradiation direction of ambient light in the ambient light information database 22 is not associated with distance (positional relationship).

[0075] 15 is a diagram showing an example of information stored in the layout information database 24 according to the second embodiment. The layout information database 24 includes data items such as a location ID, the position of a daylighting opening, the size of the daylighting opening, the position of an obstruction, and the size of the obstruction. The ambient light information of the ambient light irradiated onto the authentication area varies depending on the position and size of the daylighting opening and the obstruction in the facility where the authentication area is located. Therefore, by combining the ambient light information with the layout information, highly accurate ambient light information regarding the illuminance of the ambient light in the authentication area can be obtained.

[0076] FIG. 16 is a flowchart showing an outline of the processing executed by the authentication device 10 according to the second embodiment.

[0077] In step S201, the authentication device 10 captures an image of the entire authentication area by using the first camera 107A to capture an image of the authentication area in front of the device.

[0078] In step S202, the authentication device 10 executes a face detection process for the person to be authenticated on the captured image. When the authentication device 10 detects the face of the person to be authenticated, the process proceeds to step S203.

[0079] In step S203, the authentication device 10 refers to the ambient light information database 22 based on the image capture date and time, and acquires ambient light information. Note that since sunlight conditions differ between sunny and rainy weather even at the same date and time, weather information may be acquired from an external system based on the location information of the image capture location and the image capture date and time (current time), and then the ambient light information database 22 may be referenced based on the image capture date and time and weather. By taking into account the location of the image capture location, the date and time, the weather, etc., a more appropriate biometric image can be acquired.

[0080] In step S204, the authentication device 10 acquires layout information from the layout information database 24 using the point ID corresponding to the authentication area in the authentication device 10 as a key.

[0081] In step S205, the authentication device 10 acquires final ambient light information in the authentication area based on the ambient light information and layout information.

[0082] In step S206, the authentication device 10 controls the second camera 107B and the lighting device 108 based on the acquired ambient light information. Specifically, the authentication device 10 controls the exposure conditions, gain, imaging timing, imaging angle, etc. of the second camera 107B. The authentication device 10 also controls the illuminance, wavelength, irradiation direction, irradiation timing, etc. of the illumination light emitted from the lighting device 108.

[0083] In step S207, the authentication device 10 captures an image of the face of the person to be authenticated using the second camera 107B, and generates a face image.

[0084] In step S208, the authentication device 10 transmits the facial image generated in step S207 to the authentication engine 21, and causes the authentication engine 21 to perform facial authentication. The authentication engine 21 performs facial authentication by comparing the received facial image with the registered facial image of the registrant registered in the registrant information database 23, and then transmits the authentication result to the authentication device 10.

[0085] In step S209, the authentication device 10 determines whether or not to permit the authentication target to pass through the gate device 30 based on the authentication result received from the authentication engine 21.

[0086] Here, if the authentication device 10 determines that the authentication subject is permitted to pass through the gate device 30 (step S209: YES), it opens the gate (step S210) and ends the process.

[0087] On the other hand, if the authentication device 10 determines that the authentication target is not permitted to pass through the gate device 30 (step S209: NO), it displays information about an authentication error on the display 106 (step S211) and ends the process.

[0088] As described above, the authentication system 1 according to the second embodiment can acquire ambient light information based on the capture date and time of a biometric image and layout information of the facility where biometric authentication is performed, and can control the second camera 107B and the lighting device 108, respectively. This makes it possible to appropriately adjust the conditions under which the second camera 107B captures an image of the face of the person to be authenticated, and obtain a face image that is suitable for face authentication performed by the authentication engine 21. For example, if the capture date and time is 3:00 p.m. and sunlight is irradiating the authentication point from a window 41 of the facility at an irradiation angle θ, the second camera 107B and the lighting device 108 can be controlled to suppress the influence of sunlight based on the positional relationship between the window 41 and the authentication point. [Third embodiment]

[0089] The authentication system 1 according to the third embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and explanations of common parts will be omitted or simplified.

[0090] Fig. 17 is a flowchart showing an outline of the processing executed by the authentication device according to the third embodiment. In Fig. 17, the processing in steps S101 to S110 and steps S112 to S114 is the same as in the first embodiment, but the processing in steps S301 to S303 is different from the first embodiment.

[0091] When the process of step S110 is completed, the process proceeds to step S301. In step S301, the authentication device 10 determines whether or not to complete authentication of the authentication target.

[0092] Here, if the authentication device 10 determines that the authentication of the authentication subject is to be completed (step S301: YES), the process proceeds to step S112. On the other hand, if the authentication device 10 determines that the authentication of the authentication subject is not to be completed (step S301: NO), the process proceeds to step S302.

[0093] In step S302, the authentication device 10 acquires the imaging conditions of the second camera 107B when capturing an image of the face of the person to be authenticated.

[0094] In step S303, the authentication device 10 updates the ambient light information in the ambient light information database 22 based on the image capturing conditions, the positional relationship, and the authentication result. For example, if the illuminance of the ambient light when the face was captured differs from the expected illuminance and authentication failed, the authentication device 10 estimates the actual illuminance of the ambient light by analyzing the captured image. The authentication device 10 then updates the ambient light information database 22 with the estimated ambient light information. By taking into account the combination of the image capturing conditions, the positional relationship, and the authentication result, the authentication device 10 can control the second camera 107B and the lighting device 108 in detail and with high accuracy. When the processing of step S303 ends, the processing returns to step S101.

[0095] As described above, according to the authentication system 1 of the first embodiment, the ambient light information in the ambient light information database 22 can be flexibly changed based on the imaging conditions and the authentication results, thereby improving the accuracy of the biometric authentication performed thereafter. [Fourth embodiment]

[0096] The authentication system 1 according to the fourth embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and the description of the common parts will be omitted or simplified.

[0097] Fig. 18 is a block diagram showing an example of the hardware configuration of an authentication device 10 according to the fourth embodiment. Unlike the case of Fig. 2, Fig. 18 shows that the authentication device 10 includes a first illumination device 108A and a second illumination device 108B. The first illumination device 108A is an illumination device that irradiates infrared light toward the face of the person to be authenticated when capturing an image of the face of the person to be authenticated. The second illumination device 108B is an illumination device that irradiates infrared light toward the eyes of the person to be authenticated when capturing an image of the iris of the person to be authenticated.

[0098] Fig. 19 is a flowchart showing an outline of the processing executed by the authentication device 10 according to the fourth embodiment. In Fig. 19, the processing in steps S101 to S105 and steps S110 to S114 is the same as in the first embodiment, but the processing in steps S401 to S407 is different from the first embodiment.

[0099] When the process of step S105 is completed, the process proceeds to step S401. In step S401, the authentication device 10 determines whether or not to perform face authentication on the person to be authenticated.

[0100] Here, if the authentication device 10 determines that face authentication is to be performed on the person to be authenticated (step S401: YES), the process proceeds to step S402. On the other hand, if the authentication device 10 determines that iris authentication is to be performed on the person to be authenticated without performing face authentication on the person to be authenticated (step S401: NO), the process proceeds to step S405.

[0101] In step S402, the authentication device 10 controls the first camera 107A and the first lighting device 108A based on the ambient light information and the brightness information.

[0102] In step S403, the authentication device 10 captures an image of the face of the person to be authenticated using the first camera 107A, and generates a face image.

[0103] In step S404, the authentication device 10 transmits the facial image generated in step S403 to the authentication engine 21, and causes the authentication engine 21 to perform facial authentication. The authentication engine 21 performs facial authentication by comparing the received facial image with the registered facial image of the registrant registered in the registrant information database 23, and then transmits the authentication result to the authentication device 10. Thereafter, the process proceeds to step S110.

[0104] In step S405, the authentication device 10 controls the second camera 107B and the second lighting device 108B based on the ambient light information and the brightness information.

[0105] In step S406, the authentication device 10 captures an image of the iris of the person to be authenticated using the second camera 107B, and generates an iris image.

[0106] In step S407, the authentication device 10 transmits the iris image generated in step S406 to the authentication engine 21, and causes the authentication engine 21 to perform iris authentication. The authentication engine 21 performs iris authentication by comparing the received iris image with the registered iris image of the registrant registered in the registrant information database 23, and then transmits the authentication result to the authentication device 10. Thereafter, the process proceeds to step S110.

[0107] As described above, the authentication system 1 according to the fourth embodiment can realize two-factor authentication consisting of face authentication and iris authentication, unlike the first embodiment. Furthermore, the first illumination device 108A is controlled when capturing an image of the face of the person to be authenticated, and the second illumination device 108B is controlled when capturing an image of the iris. Since the illumination light can be appropriately switched depending on the type of biometric authentication to be performed, biometric images suitable for both face authentication and iris authentication can be acquired. [Fifth embodiment]

[0108] The authentication system 1 according to the fifth embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and explanations of common parts will be omitted or simplified.

[0109] Fig. 20 is a flowchart showing an outline of the processing executed by the authentication device according to the fifth embodiment. In Fig. 20, the processing in steps S101 to S105 and steps S108 to S114 is the same as in the first embodiment, but the processing in steps S501 to S505 is different from the first embodiment.

[0110] When the process of step S105 is completed, the process proceeds to step S501. In step S501, the authentication device 10 determines whether or not the acquisition of time-series data relating to distance and face position is completed.

[0111] If the authentication device 10 determines that the acquisition of the time-series data has been completed (step S501: YES), the process proceeds to step S502. On the other hand, if the authentication device 10 determines that the acquisition of the time-series data has not been completed (step S501: NO), the process returns to step S101.

[0112] In step S502, the authentication device 10 calculates the moving speed of the person to be authenticated based on the time-series data.

[0113] In step S503, based on the movement speed calculated in step S502, the authentication device 10 estimates the position of the face after movement and the distance from the authentication device 10. The position of the face means position coordinates in three-dimensional space.

[0114] In step S504, the authentication device 10 acquires ambient light information from the ambient light information database 22 based on the estimated post-movement distance.

[0115] In step S505, the authentication device 10 controls the second camera 107B and the lighting device 108 based on the ambient light information and the brightness information. After that, the process proceeds to step S108.

[0116] As described above, the authentication system 1 according to the fifth embodiment can acquire ambient light information at a location after the target person moves based on the estimated moving speed of the target person. This allows the second camera 107B and the lighting device 108 to be appropriately adjusted, thereby further improving the quality of the acquired biometric image. [Sixth embodiment]

[0117] The authentication system 1 according to the sixth embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and explanations of common parts will be omitted or simplified.

[0118] Fig. 21 is a flowchart showing an outline of the processing executed by the authentication device according to the sixth embodiment. In Fig. 21, the processing in steps S101 to S105 and steps S108 to S114 is the same as in the first embodiment, but the processing in steps S601 to S605 is different from the first embodiment.

[0119] When the process of step S105 is completed, the process proceeds to step S601. In step S601, the authentication device 10 determines whether or not the acquisition of time-series data relating to distance and face position is completed.

[0120] If the authentication device 10 determines that the acquisition of the time-series data has been completed (step S601: YES), the process proceeds to step S602. On the other hand, if the authentication device 10 determines that the acquisition of the time-series data has not been completed (step S601: NO), the process returns to step S101.

[0121] In step S602, the authentication device 10 identifies the gait of the person to be authenticated based on the time-series data. Gait is a person's walking pattern. Gait differs from person to person in terms of arm swing, stride length, posture, left-right asymmetry of movement, etc.

[0122] In step S603, the authentication device 10 estimates the position of the face after movement and the distance from the authentication device 10 based on the gait of the person to be authenticated identified in step S602. That is, the authentication device 10 estimates the point where the person to be authenticated, who is moving within the authentication area, will be located at the timing of capturing the next biometric image.

[0123] In step S604, the authentication device 10 acquires ambient light information from the ambient light information database 22 based on the estimated post-movement distance.

[0124] In step S605, the authentication device 10 controls the second camera 107B and the lighting device 108 based on the ambient light information and the brightness information. After that, the process proceeds to step S108.

[0125] As described above, the authentication system 1 according to the sixth embodiment can acquire ambient light information based on the gait of the person to be authenticated. This allows the second camera 107B and the lighting device 108 to be appropriately adjusted, thereby further improving the quality of the acquired biometric image. [Seventh embodiment]

[0126] The authentication system 1 according to the seventh embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and the description of the common parts will be omitted or simplified.

[0127] Fig. 22 is a functional block diagram showing the overall configuration of an authentication device 10 according to the seventh embodiment. In Fig. 22, the authentication device 10 includes a face detection unit 11, a distance acquisition unit 12, an ambient light information acquisition unit 13, a luminance information acquisition unit 14, and a control unit 15, as well as a determination unit 17.

[0128] The determination unit 17 analyzes the captured image captured by the first camera 107A and determines whether or not an article worn by the person to be authenticated (hereinafter referred to as an attachment) affects the biometric image. Specifically, the determination unit 17 determines whether or not there is an attachment in the captured image, the type of attachment, the size of the attachment, the transmission state of illumination light (infrared light) or environmental light at the attachment, the reflection state of illumination light or environmental light, the presence or absence of a shadow caused by the attachment, etc.

[0129] 23, 24 and 25 are flowcharts showing an outline of the processing executed by the authentication device according to the seventh embodiment.

[0130] Fig. 23 shows a process for controlling second camera 107B and lighting device 108 depending on the transmission state of illumination light through an attachment worn by the person to be authenticated. In Fig. 23, the processes of steps S101 to S106 and steps S108 to S114 are the same as those in the first embodiment, but the processes of steps S701 to S704 are different from those in the first embodiment.

[0131] When the process of step S106 is completed, the process proceeds to step S701. In step S701, the authentication device 10 analyzes the captured image and determines whether or not an article is attached to the head of the person to be authenticated.

[0132] If the authentication device 10 determines that the subject is wearing an attachment on their head (step S701: YES), the process proceeds to step S702. On the other hand, if the authentication device 10 determines that the subject is not wearing an attachment on their head (step S701: NO), the process proceeds to step S704.

[0133] In step S702, the authentication device 10 captures an image of the face of the person to be authenticated together with the attached article of clothing, such as a hat, glasses, sunglasses, or a helmet.

[0134] In step S703, the authentication device 10 analyzes the face image captured in step S702 and determines the transmission state of illumination light (infrared light) through the attachment. For example, if the attachment is glasses or sunglasses, the transmission state of infrared light can be determined from the pixel values ​​of the lenses.

[0135] In step S704, the authentication device 10 controls the second camera 107B and the illumination device 108 based on the determination result of the transmission state of the illumination light (infrared light) in step S703. For example, if it is determined that the worn item is sunglasses with black lenses that do not transmit infrared light of a predetermined wavelength, the authentication device 10 preferably changes the wavelength range of the infrared light emitted from the illumination device 108. Alternatively, the light receiving wavelength range of the second camera 107B may be changed. By taking the transmission state of the illumination light into consideration, a high-quality biometric image suitable for biometric authentication can be acquired.

[0136] Fig. 24 shows a process for controlling second camera 107B and lighting device 108 based on the state of reflection of illumination light from an attachment. The process in Fig. 24 differs from that in Fig. 23 in steps S801 and S802.

[0137] When the process of step S702 is completed, the process proceeds to step S801. In step S801, the authentication device 10 analyzes the face image captured in step S702 and determines the state of reflection of illumination light on the surface of the attachment.

[0138] In step S802, the authentication device 10 controls the second camera 107B and the lighting device 108 based on the determination result of the reflection state of the illumination light in step S801. For example, if the object being worn is glasses that reflect the illumination light, the authentication device 10 preferably controls the second camera 107B to change the imaging angle. Furthermore, the authentication device 10 may change the illuminance and irradiation angle of the illumination light emitted from the lighting device 108. By taking the reflection state of infrared light into consideration, a biometric image can be more appropriately acquired.

[0139] Fig. 25 shows a process for controlling second camera 107B and lighting device 108 depending on whether or not there is a shadow cast by an attached item. The process in Fig. 25 differs from that in Fig. 23 in steps S901 and S902.

[0140] When the process of step S702 is completed, the process proceeds to step S901. In step S901, the authentication device 10 analyzes the captured image and determines whether or not there is a shadow of an attachment on a body part of the person to be authenticated, and the position and size of the shadow.

[0141] In step S902, the authentication device 10 controls the second camera 107B and the lighting device 108 based on information about the shadow of the attached article. For example, if the attached article is a hat and the shadow of the hat overlaps the face area of ​​the person to be authenticated in the image, the authentication device 10 preferably controls to change the imaging angle of the second camera 107B. The authentication device 10 may also change the illuminance and irradiation angle of the infrared light emitted from the lighting device 108. By taking into account information about the shadow of the attached article, a biometric image can be more appropriately acquired. [Eighth embodiment]

[0142] The authentication system 1 according to the eighth embodiment will be described below. The differences from the first embodiment will be mainly described below, and descriptions of common parts will be omitted or simplified.

[0143] 26 is a functional block diagram showing the overall configuration of an authentication device 10 according to the eighth embodiment. The authentication device 10 includes a face detection unit 11, a distance acquisition unit 12, an ambient light information acquisition unit 13, a luminance information acquisition unit 14, a control unit 15, a determination unit 17, and a learning unit 18.

[0144] The learning unit 18 receives inputs of the presence or absence of an article worn by the person to be authenticated in the captured image, the presence or absence of a shadow on the person's body part, and the relationship between the size of the shadow, and generates a learning model that outputs control information for the second camera 107B and the lighting device 108. Generating a learning model using this learning data makes it possible to acquire high-quality biometric images suitable for biometric authentication. Note that the data used in the learning process is not limited to these. The input data may be related to factors that change the luminance information of the captured image. For example, the input data may include data on the displacement of the imaging angle of the second camera 107B when capturing an image of the face, and data on the displacement of the positional relationship between the authentication device 10 (the second camera 107B and the lighting device 108) and the person to be authenticated at the time of capturing the image. Learning the displacement data of the imaging angle and the luminance information can prevent degradation of the quality of the biometric image due to a sudden change in the imaging angle. Similarly, learning the displacement data of the positional relationship and the luminance information can prevent degradation of the quality of the biometric image due to a displacement of the positional relationship.

[0145] FIG. 27 is a schematic diagram illustrating a neural network used in the learning process according to the eighth embodiment. The neural network shown in FIG. 27 includes an input layer having a plurality of nodes, an intermediate layer having a plurality of nodes, and an output layer having one node. Multiple types of biological information are input as input values ​​to each node in the input layer. Each node in the intermediate layer is connected to each node in the input layer. Each element of the input value input to the node in the intermediate layer is used for calculation in each node in the intermediate layer. Each node in the intermediate layer calculates a calculated value using, for example, the input value input from each node in the input layer, a predetermined weighting coefficient, and a predetermined bias value. Each node in the intermediate layer is connected to the output layer and outputs the calculated calculated value to a node in the output layer. The node in the output layer receives the calculated value from each node in the intermediate layer.

[0146] The nodes in the output layer output a value indicating optimal control information y using the calculated values, weighting coefficients, and bias values ​​input from the nodes in the intermediate layer. Examples of the control information y for the second camera 107B include exposure conditions, gain, imaging timing, imaging angle, etc. Furthermore, examples of the control information y for the lighting device 108 include the illuminance, wavelength, irradiation direction, irradiation timing, etc. of the illumination light.

[0147] When training a neural network, for example, the backpropagation method is used. Specifically, the output value when data is input to the input layer is compared with the output value obtained from the training data, and the error between the two compared output values ​​is fed back to the middle layer. This process is repeated until the error falls below a predetermined threshold. This type of training process generates a training model that can output optimal control information y when data about the subject's clothing is input.

[0148] 28 and 29 are flowcharts showing an outline of the processing executed by the authentication device 10 according to the eighth embodiment.

[0149] Fig. 28 shows an example of learning processing in the learning unit 18. This processing can be executed independently of the processing in Fig. 29, which will be described later.

[0150] In step S1001, the authentication device 10 inputs a biometric image (face image) of the person to be authenticated.

[0151] In step S1002, the authentication device 10 acquires the distance from the authentication device 10 to the person to be authenticated when the biometric image is captured.

[0152] In step S1003, the authentication device 10 analyzes the biometric image and acquires luminance information of the face of the person to be authenticated.

[0153] In step S1004, the authentication device 10 analyzes the biometric image and determines whether or not there is a shadow of an article attached to the face area of ​​the person to be authenticated, as well as the position and size of the shadow.

[0154] In step S1005, the authentication device 10 analyzes the biometric image and determines the state of reflection of the illumination light on the attachment.

[0155] In step S1006, the authentication device 10 acquires the ambient light information from the ambient light information database 22 based on the distance.

[0156] In step S1007, the authentication device 10 acquires the imaging conditions of the second camera 107B and the irradiation conditions of the illumination light in the illumination device 108 that were determined before capturing an image of the face.

[0157] In step S1008, the authentication device 10 acquires the authentication result of the face authentication based on the face image captured under the conditions in step S1007.

[0158] In step S1009, the authentication device 10 generates a learning model based on the luminance information, ambient light information, distance, information related to the attachment, the reflection state of the illumination light, control information, and the authentication result.

[0159] Fig. 29 shows an example of a process for controlling second camera 107B and lighting device 108 based on the learning model generated by the learning process shown in Fig. 28. In Fig. 29, the processes of steps S101 to S106 and steps S108 to S114 are the same as those in the first embodiment, but the processes of steps S1101 to S1102 are different from those in the first embodiment.

[0160] When the process of step S106 is completed, the process proceeds to step S1101. In step S1101, the authentication device 10 analyzes the facial image of the person to be authenticated extracted from the captured image, and acquires information about the accessory.

[0161] In step S1102, the authentication device 10 inputs the luminance information, ambient light information, distance, information related to the attachment, the reflection state of the illumination light, control information, and the authentication result into the learning model, and outputs the control information output from the learning model. The authentication device 10 controls the second camera 107B and the lighting device 108 based on the control information output from the learning model. Then, the process proceeds to step S108.

[0162] As described above, the authentication system 1 according to the eighth embodiment controls the second camera 107B and the lighting device 108 based on a learning model that learns the relationship between information about the attachment and the factors that the attachment affects on the illuminance of ambient light, and can acquire a biometric image suitable for biometric authentication. Note that the learning model may be generated based on the accuracy of biometric authentication so as to prioritize reducing one of the effects of shadows and reflections over the other. This allows a high-quality biometric image suitable for biometric authentication to be acquired. [Ninth embodiment]

[0163] The authentication system 1 according to the ninth embodiment will be described below. The following mainly focuses on the differences from the first embodiment, and explanations of common parts will be omitted or simplified.

[0164] Fig. 30 is a flowchart showing an outline of the processing executed by the authentication device 10 according to the ninth embodiment. In Fig. 30, the processing in steps S101 to S107 and steps S109 to S114 is the same as in the first embodiment, but the processing in steps S1201 to S1202 is different from the first embodiment.

[0165] When the processing of step S107 is completed, the processing proceeds to step S1201. In step S1201, the authentication device 10 calculates a quality value of the facial image generated in step S107. For example, the authentication device 10 can calculate the quality value of the facial image by comparing the luminance information of the acquired facial image with the luminance information of a registered facial image (registered luminance information) that has been previously confirmed to be suitable for facial authentication. In this case, the quality value of the biometric image can be easily calculated. Alternatively, the authentication device 10 may calculate the quality value of the facial image based on the pixel values ​​of the generated facial image. The method of calculating the quality value can be determined arbitrarily. The authentication device 10 may correct the quality value based on, for example, the amount of change in illuminance of the illumination light per unit time. In this case, deterioration in the quality of the biometric image due to a sudden change in illuminance of the illumination light can be suppressed.

[0166] In step S1202, the authentication device 10 determines whether the calculated quality value is equal to or greater than a predetermined threshold value.

[0167] If the authentication device 10 determines that the quality value is equal to or greater than a predetermined threshold (step S1202: YES), the process proceeds to step S109. Then, face authentication is performed (step S109). On the other hand, if the authentication device 10 determines that the quality value is less than the threshold (step S1202: NO), the process returns to step S101.

[0168] As described above, the authentication system 1 according to the ninth embodiment can perform control so that biometric authentication is performed when the quality value of the generated biometric image satisfies a predetermined condition, and is not performed when the quality value does not satisfy the same condition. This avoids unnecessary biometric authentication and allows for a quick transition to the next imaging process, thereby increasing the number of times biometric authentication is performed based on high-quality biometric images. [Tenth embodiment]

[0169] FIG. 31 is a functional block diagram showing the overall configuration of an information processing device 100 according to a tenth embodiment. The information processing device 100 includes an acquisition unit 100A and a control unit 100B. The acquisition unit 100A acquires the positional relationship between a subject of biometric authentication and an imaging device that captures the subject, ambient light information including at least one of illumination light irradiated on the subject by an illumination device and ambient light around the subject, and luminance information of the subject in the captured image captured by the imaging device. The control unit 100B controls at least one of the illumination device and the imaging device based on the positional relationship, ambient light information, and luminance information. The tenth embodiment provides an information processing device 100 that can acquire a biometric image suitable for biometric authentication. [Eleventh embodiment]

[0170] FIG. 32 is a functional block diagram showing the overall configuration of an information processing device 200 according to an eleventh embodiment. The information processing device 200 includes an acquisition unit 200A and a control unit 200B. The acquisition unit 200A acquires ambient light information that indicates factors that may affect the illuminance of a subject of biometric authentication. The control unit 200B controls at least one of an illumination device that irradiates the subject with illumination light and an imaging device that captures an image of the subject, based on the ambient light information. The eleventh embodiment provides an information processing device 200 that can acquire a biometric image suitable for biometric authentication. [Modified embodiment]

[0171] This disclosure is not limited to the above-described embodiments and can be modified as appropriate within the scope of the spirit of this disclosure. For example, an example in which part of the configuration of one embodiment is added to another embodiment, or an example in which part of the configuration of another embodiment is replaced with another embodiment, is also an embodiment of this disclosure.

[0172] In the above-described embodiment, a case where face authentication is performed in the authentication system 1 has been described, but the technology disclosed herein can also be applied to a system that performs iris authentication. If the authentication system 1 is an iris authentication system, the second camera 107B can be replaced with an iris camera. An iris image used for iris authentication can be obtained by irradiating a person's eyes with infrared light from the lighting device 108 and capturing an image of the infrared light reflected by the iris with the iris camera.

[0173] In the above-described embodiment, a configuration has been described in which the distance from the authentication device 10 to the person to be authenticated is measured based on a detection signal from the distance sensor 109, but the method of measuring the distance is not limited to this. For example, the actual distance from the authentication device 10 to the person to be authenticated may be estimated based on the distance between the eyes of the person to be authenticated in a captured image. In this case, the distance sensor 109 can be omitted, and manufacturing costs can be reduced.

[0174] Furthermore, in the above-described embodiment, the case where both the lighting device 108 and the second camera 107B are controlled has been described, but it is sufficient to control at least one of the lighting device 108 and the second camera 107B. In other words, it is also possible to control only one of the lighting device 108 and the second camera 107B.

[0175] Furthermore, authentication device 10 may control at least one of lighting device 108 and second camera 107B based on the positional relationship in previously performed biometric authentication, the illuminance of the illumination light emitted by the lighting device, and changes in brightness information. For example, if biometric authentication of a first person fails because the illumination light is too dim at a distance of 3 meters, control can be performed to brighten the illumination light irradiated on the second person at a distance of 3 meters in the next biometric authentication of a second person. This allows for the acquisition of high-quality biometric images suitable for biometric authentication.

[0176] Furthermore, the authentication device 10 may control at least one of the lighting device 108 and the second camera 107B based on the matching score (authentication result) and changes in the positional relationship in previously performed biometric authentication. Specifically, the authentication device 10 may record the matching score and distance displacement data of the authentication target for a first person previously performed, and when capturing a biometric image for biometric authentication of a second person after the first person, the authentication device 10 may control the lighting device 108 and the second camera 107B based on the matching score and distance displacement data for the first person. Furthermore, if the matching score for biometric authentication based on a biometric image of the first person captured at a point two meters away from the authentication device 10 is high, the point may be considered the best point, and the control parameters of the lighting device 108 at the point for the next second person may be corrected. This allows a high-quality biometric image suitable for biometric authentication to be acquired.

[0177] In the above-described embodiment, the imaging angle and imaging range are changed by driving second camera 107B itself, but the configuration for changing the imaging angle and imaging range is not limited to this. For example, a configuration using a rotating mirror may be applied.

[0178] 33 and 34 are schematic diagrams illustrating a method for changing the imaging angle and imaging range of authentication device 10 according to a modified embodiment. Rotating mirror 111 is provided so as to be movable in the vertical direction relative to the horizontal, and is a member that reflects light incident from outside authentication device 10 toward second camera 107B. As shown in FIGS. 33 and 34, first camera 107A is disposed in the horizontal direction.

[0179] On the other hand, the light receiving surface of second camera 107B is disposed vertically downward so as to face the reflective surface of rotating mirror 111. Rotating mirror 111 is connected to rotating shaft 110a of motor 110 and can rotate together with rotating shaft 110a. FIG. 33 shows the state before rotating mirror 111 is driven. On the other hand, FIG. 34 shows the state after rotating mirror 111 has rotated counterclockwise from the state shown in FIG. 33. The angle at which the optical axis Ax of second camera 107B intersects with the horizontal plane HP is -θ3. That is, the imaging angle of second camera 107B in FIG. 34 is -θ3.

[0180] Furthermore, the imaging range R10 of first camera 107A is wider than the imaging range R20 of second camera 107B. In other words, the angle of view of first camera 107A is wider than the angle of view of second camera 107B. Authentication device 10 moves imaging range R20 of second camera 107B by changing the tilt angle of rotating mirror 111 using motor 110. For example, authentication device 10 can move imaging range R20 of second camera 107B up and down along the vertical direction by rotating rotating mirror 111.

[0181] According to the authentication system 1 of the modified embodiment, there is no need to drive the second camera 107B itself in order to change the imaging angle of the second camera 107B. By driving the rotating mirror 111 instead of the second camera 107B, it is possible to capture a biometric image used for biometric authentication at an appropriate imaging angle. Note that the rotating mirror 111 does not have to be directly connected to the rotating shaft 110a of the motor 110. For example, the rotating mirror 111 may be indirectly connected to the rotating shaft 110a via a gear, a belt, or the like, and driven by the motor 110.

[0182] In the first embodiment described above, the authentication device 10 has been described in which the imaging devices (first camera 107A and second camera 107B) and the lighting device 108 are integrated. However, the imaging devices and the lighting device 108 may be provided independently of each other. FIG. 35 is a diagram showing an example of the positional relationship between the imaging devices, the lighting device 108, and the person to be authenticated T in a modified embodiment. Here, the lighting device 108 is installed at a point P10 that is a distance D10 away from a point P0 where the authentication device 10 is installed. In FIG. 35, the lighting device 108 is hung from a ceiling (not shown), but the installation method of the lighting device 108 is not limited to this. Furthermore, the positional relationship between the lighting device 108, the imaging devices, and the person to be authenticated T is not limited to the positional relationship shown in FIG. 35. For example, the lighting device 108 may be installed at a position farther from the person to be authenticated T than the imaging devices. Furthermore, the lighting device 108 may be installed above or below the imaging device. When the image capture device and the lighting device 108 are installed at a distance from each other, the authentication system may store ambient light information in a database in association with the positional relationship between the image capture device, the lighting device 108, and the person to be authenticated T. By determining the image capture conditions based on the ambient light information associated with the positional relationship, it is possible to keep the brightness of the biometric image captured by the image capture device within a predetermined range suitable for biometric authentication. Then, by performing biometric authentication using a biometric image captured at an appropriate brightness, it is possible to improve the authentication accuracy in biometric authentication.

[0183] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-described embodiments is recorded on a storage medium, the program recorded on the storage medium is read as code, and the program is executed on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. Furthermore, not only the storage medium on which the above-described program is recorded, but also the program itself is included in each embodiment. Furthermore, one or more components included in the above-described embodiments may be circuits such as ASICs or FPGAs configured to realize the functions of each component.

[0184] Examples of the storage medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disk)-ROM, a magnetic tape, a non-volatile memory card, and a ROM. Furthermore, the scope of each embodiment is not limited to programs that execute processing by themselves recorded on the storage medium, but also includes programs that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0185] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0186] It should be noted that the above-described embodiments are merely examples of specific embodiments for carrying out this disclosure, and the technical scope of this disclosure should not be interpreted as being limited by these embodiments. In other words, this disclosure can be carried out in various forms without departing from its technical idea or main features.

[0187] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0188] (Appendix 1) an acquisition unit that acquires a positional relationship between a subject of biometric authentication and an image capturing device that captures an image of the subject, ambient light information including at least one of illumination light irradiated onto the subject by an illumination device and ambient light around the subject, and luminance information of the subject in an image captured by the image capturing device; a control unit that controls at least one of the lighting device and the imaging device based on the positional relationship, the ambient light information, and the luminance information; An information processing device comprising:

[0189] (Appendix 2) the ambient light information is associated with the positional relationship in advance; the control unit updates the ambient light information based on an imaging condition of the captured image and a result of the biometric authentication performed based on the captured image. 2. The information processing device according to claim 1.

[0190] (Appendix 3) the control unit controls at least one of the illumination device and the imaging device based on imaging conditions of the captured image, the positional relationship at the time of imaging, and a result of the biometric authentication performed based on the captured image. 2. The information processing device according to claim 1.

[0191] (Appendix 4) the control unit turns on one of the plurality of lighting devices that corresponds to the type of biometric authentication to be performed in the positional relationship. 4. An information processing device according to any one of claims 1 to 3.

[0192] (Appendix 5) the control unit controls at least one of the lighting device and the imaging device based on the color of the body part of the subject indicated by the luminance information. 4. An information processing device according to any one of claims 1 to 3.

[0193] (Appendix 6) the control unit controls at least one of the lighting device and the imaging device based on at least one of a face position of the subject in a three-dimensional space identified from the captured image, a moving speed of the subject identified from the positional relationship, and a gait of the subject identified from the positional relationship. 5. An information processing device according to any one of appendices 1 to 4.

[0194] (Appendix 7) a determination unit that determines a transmission state of the illumination light through an article worn by the subject based on the captured image, the control unit controls at least one of an irradiation wavelength range of the illumination light of the illumination device and a light receiving wavelength range of the image capturing device based on the determination result of the transmission state. 7. An information processing device according to any one of appendices 1 to 6.

[0195] (Appendix 8) a determination unit that determines a reflection state of the illumination light from an article worn by the subject based on the captured image, the control unit controls at least one of the lighting device and the imaging device based on the reflection state. 7. An information processing device according to any one of appendices 1 to 6.

[0196] (Appendix 9) a determination unit that determines, based on the captured image, whether or not an article is worn by the subject and whether or not a shadow of the article is cast on a body part of the subject; a learning unit that learns the relationship between the presence or absence of the object, the positional relationship, and the presence or absence of the shadow; Further provided with the control unit controls at least one of the lighting device and the imaging device based on a learning result of the learning unit. 7. An information processing device according to any one of appendices 1 to 6.

[0197] (Appendix 10) a determination unit that determines, based on the captured image, whether or not an article is worn by the subject, a reflection state of the illumination light on the article, and whether or not a shadow of the article is cast on a body part of the subject; a learning unit that learns the relationship between the presence or absence of the object, the positional relationship, the reflection state, and the presence or absence of the shadow; Further provided with the control unit controls at least one of the lighting device and the imaging device based on the learning result of the learning unit so as to eliminate reflection. 7. An information processing device according to any one of appendices 1 to 6.

[0198] (Appendix 11) a learning unit configured to learn the relationship between a change in the imaging angle of the imaging device, a change in the luminance information, the presence or absence of an article worn by the subject, the presence or absence of a shadow of the article on a body part of the subject, and a reflection state of the illumination light on the article; the control unit controls at least one of the lighting device and the imaging device based on the learning result of the learning unit. 7. An information processing device according to any one of appendices 1 to 6.

[0199] (Appendix 12) the control unit calculates a quality value of the captured image based on a comparison result between the luminance information and predetermined registered luminance information, and requests the authentication device to perform the biometric authentication based on the quality value. 12. An information processing device according to any one of appendices 1 to 11.

[0200] (Appendix 13) the control unit corrects a quality value of the captured image based on a variation in illuminance of the illumination light per unit time, and requests the authentication device to perform the biometric authentication based on the quality value. 12. An information processing device according to any one of appendices 1 to 11.

[0201] (Appendix 14) the control unit controls at least one of the illumination device and the imaging device based on changes in the positional relationship in the biometric authentication that has already been performed, the illuminance of the illumination light emitted by the illumination device, and the luminance information. 14. An information processing device according to any one of appendices 1 to 13.

[0202] (Appendix 15) the control unit controls at least one of the illumination device and the imaging device based on changes in the matching score and the positional relationship in the biometric authentication that has already been performed. 14. An information processing device according to any one of appendices 1 to 13.

[0203] (Appendix 16) acquiring ambient light information including a positional relationship between a subject of biometric authentication and an image capturing device capturing an image of the subject, illumination light irradiated onto the subject by an illumination device, and / or ambient light around the subject, and luminance information of the subject in an image captured by the image capturing device; controlling at least one of the lighting device and the imaging device based on the positional relationship, the ambient light information, and the luminance information; An information processing method comprising:

[0204] (Appendix 17) On the computer, acquiring ambient light information including a positional relationship between a subject of biometric authentication and an image capturing device capturing an image of the subject, illumination light irradiated onto the subject by an illumination device, and / or ambient light around the subject, and luminance information of the subject in an image captured by the image capturing device; controlling at least one of the lighting device and the imaging device based on the positional relationship, the ambient light information, and the luminance information; An information processing method comprising: A recording medium on which a program for executing the above is recorded.

[0205] (Appendix 18) an acquisition unit that acquires ambient light information representing factors that may affect the illuminance of a person to be biometrically authenticated; a control unit that controls at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject based on the ambient light information; An information processing device comprising:

[0206] (Appendix 19) The ambient light information includes an image capture date and time. 19. The information processing device according to claim 18.

[0207] (Appendix 20) The ambient light information includes weather information. 20. The information processing device according to claim 18 or 19.

[0208] (Appendix 21) The ambient light information includes optical properties, shapes, sizes, and arrangements of structures surrounding the subject. 21. An information processing device according to any one of appendices 18 to 20.

[0209] (Appendix 22) obtaining ambient light information representative of factors that may affect illuminance at a subject for biometric authentication; controlling at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject based on the ambient light information; An information processing method comprising:

[0210] (Appendix 23) obtaining ambient light information representative of factors that may affect illuminance at a subject for biometric authentication; controlling at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject based on the ambient light information; A recording medium on which a program for executing the above is recorded. [Explanation of symbols]

[0211] 1. Authentication system 10. Authentication device 101 Processor 102 RAM 103···ROM 104 Storage 105 Communication I / F 106···Display 107A···1st camera 107B···Second camera 108 Lighting equipment 108A...1st lighting device 108B...Second lighting device 109...Distance sensor 110···Motor 111···Rotating mirror 20 Authentication Server 21 Authentication Engine 22. Environmental Light Information Database 23. Registered Person Information Database 24. Layout information database 30. Gate device 100, 200... Information processing device 100A...acquisition unit 100B···Control unit 200A...Acquisition part 200B···Control unit

Claims

1. an acquisition unit that acquires a positional relationship between a subject of biometric authentication and an image capturing device that captures an image of the subject, ambient light information including at least one of illumination light irradiated onto the subject by an illumination device and ambient light around the subject, and luminance information of the subject in an image captured by the image capturing device; a control unit that controls at least one of the illumination device and the imaging device based on the positional relationship, the ambient light information, and the luminance information; a learning unit that learns the relationship between a change in the imaging angle of the imaging device, a change in the luminance information, the presence or absence of an article worn by the subject, the presence or absence of a shadow of the article on a body part of the subject, and a reflection state of the illumination light on the article; Equipped with The control unit controls at least one of the lighting device and the imaging device based on a learning result of the learning unit.

2. the ambient light information is associated with the positional relationship in advance; the control unit updates the ambient light information based on an imaging condition of the captured image and a result of the biometric authentication performed based on the captured image. The information processing device according to claim 1 .

3. the control unit controls at least one of the illumination device and the imaging device based on imaging conditions of the captured image, the positional relationship at the time of imaging, and a result of the biometric authentication performed based on the captured image. The information processing device according to claim 1 .

4. An acquisition unit that acquires ambient light information including the positional relationship between a subject of biometric authentication and an imaging device that images the subject, illumination light irradiated by a lighting device onto the subject, and at least one of ambient light around the subject, and luminance information of the subject in an image captured by the imaging device; a control unit that controls at least one of the illumination device and the imaging device based on the positional relationship, the ambient light information, and the luminance information; Equipped with the control unit controls at least one of the illumination device and the imaging device based on changes in the positional relationship in the biometric authentication that has already been performed, the illuminance of the illumination light emitted by the illumination device, and the luminance information. Information processing device.

5. acquiring ambient light information including a positional relationship between a subject of biometric authentication and an image capturing device capturing an image of the subject, illumination light irradiated onto the subject by an illumination device, and / or ambient light around the subject, and luminance information of the subject in an image captured by the image capturing device; controlling at least one of the lighting device and the imaging device based on the positional relationship, the ambient light information, and the luminance information; learning a relationship between a change in the imaging angle of the imaging device, a change in the luminance information, the presence or absence of an article worn by the subject, the presence or absence of a shadow of the article on a body part of the subject, and a reflection state of the illumination light on the article; Equipped with In the controlling step, at least one of the lighting device and the imaging device is controlled based on a learning result in the learning step.

Images