Information processing device and information processing method

Abstract

Acquire high-quality biometric images suitable for biometric authentication. [Solution] The information processing device disclosed herein includes an acquisition unit that acquires the positional relationship between a person subject to biometric authentication and an imaging device that images the person subject, ambient light information including at least one of the illumination light irradiated onto the person by an illumination device and ambient light around the person subject, and brightness information of the person subject in an image captured by the imaging device; a determination unit that determines the state of the illumination light on an article worn by the person subject based on the image captured; and a control unit that controls at least one of the illumination device and the imaging device based on the positional relationship, the ambient light information, the brightness information, and the determination result of the determination unit.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] Patent Document 1 describes an authentication system that authenticates a target person using a face image generated by imaging the face of the target person.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] This disclosure aims to improve the technology described in the above prior art documents.

Means for Solving the Problems

[0005] According to one aspect of this disclosure, an acquisition unit that acquires at least one of the positional relationship between a subject for biometric authentication and an imaging device that images the subject, environmental light information including illumination light irradiated by an illumination device to the subject and ambient light around the subject, and luminance information of the subject in an imaging image captured by the imaging device; a control unit that controls at least one of the illumination device and the imaging device based on the positional relationship, the environmental light information, and the luminance information; and a learning unit that learns the relationship between the displacement of the imaging angle of the imaging device, the displacement of the luminance information, the presence or absence of an article worn by the subject, the presence or absence of a shadow of the article on the body part of the subject, and the reflection state of the illumination light on the article are provided. The control unit controls at least one of the illumination device and the imaging device based on the learning result of the learning unit.

[0006] According to another aspect of this disclosure, an information processing method is provided, comprising the steps of: acquiring the positional relationship between a person subject to biometric authentication and an imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by an illumination device and ambient light around the person, and brightness information of the person; controlling the illumination device and at least one of the imaging device based on the positional relationship, the ambient light information, and the brightness information; and learning the relationship between the displacement of the imaging angle of the imaging device, the displacement of the brightness information, the presence or absence of an article worn by the person, the presence or absence of a shadow of the article on a part of the person's body, and the reflection state of the illumination light on the article, wherein the control step controls at least one of the illumination device and the imaging device based on the learning result in the learning step.

[0007] In another aspect of this disclosure, a program is provided that causes a computer to perform the steps of: acquiring the positional relationship between a person subject to biometric authentication and an imaging device that images the person, ambient light information including at least one of illumination light irradiated onto the person by an illumination device and ambient light around the person, and brightness information of the person in an image captured by the imaging device; and controlling at least one of the illumination device and the imaging device based on the positional relationship, the ambient light information and the brightness information.

[0008] In another aspect of this disclosure, an information processing device is provided, comprising: an acquisition unit that acquires ambient light information representing factors that may affect the illuminance of a person subject to biometric authentication; and a control unit that controls at least one of an illumination device that irradiates the person with illumination light and an imaging device that images the person based on the ambient light information.

[0009] In another aspect of this disclosure, an information processing method is provided, comprising the steps of: acquiring ambient light information representing factors that may affect the illuminance of a person subject to biometric authentication; and controlling at least one of an illumination device that irradiates the person with illumination light and an imaging device that images the person, based on the ambient light information.

[0010] In another aspect of this disclosure, a recording medium is provided on which a program is recorded that causes a computer to perform the steps of: acquiring ambient light information representing factors that may affect the illuminance of a person subject to biometric authentication; and controlling at least one of an illumination device that irradiates the person with illumination light and an imaging device that images the person, based on the ambient light information. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing an example of the overall configuration of the authentication system according to the first embodiment. [Figure 2] This is a block diagram showing an example of the hardware configuration of an authentication device according to the first embodiment. [Figure 3] This is a schematic diagram illustrating the authentication system according to the first embodiment. [Figure 4] This is a front view showing the external appearance of the authentication device according to the first embodiment. [Figure 5] This is a perspective view illustrating the internal structure of the authentication device according to the first embodiment. [Figure 6] This figure shows an example of the information stored in the ambient light information database according to the first embodiment. [Figure 7] This figure shows an example of the information stored in the registrant information database according to the first embodiment. [Figure 8] This is a functional block diagram showing the overall configuration of the authentication device according to the first embodiment. [Figure 9] This is a flowchart illustrating the process performed by the authentication device according to the first embodiment. [Figure 10] This is a schematic diagram illustrating a method for changing the imaging angle and imaging range of the second camera in the authentication device according to the first embodiment. [Figure 11] This is a schematic diagram illustrating a method for changing the imaging angle and imaging range of the second camera in the authentication device according to the first embodiment. [Figure 12] This is a schematic diagram illustrating the authentication system according to the second embodiment. [Figure 13]It is a block diagram showing an example of the overall configuration of the authentication system according to the second embodiment. [Figure 14] It is a diagram showing an example of information stored in the ambient light information database according to the second embodiment. [Figure 15] It is a diagram showing an example of information stored in the layout information database according to the second embodiment. [Figure 16] It is a flowchart showing an outline of the processing executed by the authentication device according to the second embodiment. [Figure 17] It is a flowchart showing an outline of the processing executed by the authentication device according to the third embodiment. [Figure 18] It is a block diagram showing an example of the hardware configuration of the authentication device according to the fourth embodiment. [Figure 19] It is a flowchart showing an outline of the processing executed by the authentication device according to the fourth embodiment. [Figure 20] It is a flowchart showing an outline of the processing executed by the authentication device according to the fifth embodiment. [Figure 21] It is a flowchart showing an outline of the processing executed by the authentication device according to the sixth embodiment. [Figure 22] It is a functional block diagram showing the overall configuration of the authentication device according to the seventh embodiment. [Figure 23] It is a flowchart showing an outline of the processing executed by the authentication device according to the seventh embodiment. [Figure 24] It is a flowchart showing an outline of the processing executed by the authentication device according to the seventh embodiment. [Figure 25] It is a flowchart showing an outline of the processing executed by the authentication device according to the seventh embodiment. [Figure 26] It is a functional block diagram showing the overall configuration of the authentication device according to the eighth embodiment. [Figure 27] It is a schematic diagram explaining the neural network used for the learning process according to the eighth embodiment. [Figure 28] It is a flowchart showing an outline of the processing executed by the authentication device according to the eighth embodiment. [Figure 29] This is a flowchart illustrating the process performed by the authentication device according to the eighth embodiment. [Figure 30] This is a flowchart illustrating the process performed by the authentication device according to the ninth embodiment. [Figure 31] This is a functional block diagram showing the overall configuration of the information processing device according to the 10th embodiment. [Figure 32] This is a functional block diagram showing the overall configuration of the information processing device according to the 11th embodiment. [Figure 33] This is a schematic diagram illustrating a method for changing the imaging angle and imaging range of the second camera in an authentication device according to a modified embodiment. [Figure 34] This is a schematic diagram illustrating a method for changing the imaging angle and imaging range of the second camera in an authentication device according to a modified embodiment. [Figure 35] This figure shows an example of the positional relationship between the imaging device, the lighting device, and the person being authenticated in a modified embodiment. [Modes for carrying out the invention]

[0012] Illustrative embodiments of this disclosure will be described below with reference to the drawings. Similar or corresponding elements in the drawings are denoted by the same reference numerals, and their descriptions may be omitted or simplified.

[0013] [First Embodiment] Figure 1 is a block diagram showing an example of the overall configuration of the authentication system 1 according to the first embodiment. The authentication system 1 consists of an authentication device 10, an authentication server 20, and a gate device 30. Each device is connected to networks NW1 and NW2, such as a LAN (Local Area Network) or the Internet.

[0014] Authentication system 1 acquires biometric information of the person to be authenticated (hereinafter referred to as "person to be authenticated") and performs biometric authentication by comparing the acquired biometric information with pre-registered biometric information.

[0015] In the first embodiment, the term "biometric information" refers to a facial image and features extracted from the facial image; however, biometric information is not limited to facial images and facial features. That is, the authentication system 1 may perform biometric authentication using biometric images other than facial images (such as iris images, fingerprint images, palm print images, vein images, gait images, and auricle images) and features as the biometric information of the person to be authenticated.

[0016] Authentication System 1 can be applied, for example, to identity verification for entry and exit at airports, identity verification at government agencies, identity verification for entry and exit at factories and businesses, and identity verification for entry and exit at event venues. Furthermore, Authentication System 1 can be applied to biometric authentication-based payments. Authentication System 1 is applicable to authentication in various technological fields.

[0017] The authentication device 10 is a computer that captures an image of a person to be authenticated who is present in the authentication area and outputs the biometric image to the authentication server 20. In the first embodiment, the "authentication area" refers to a predetermined three-dimensional space located in front of the authentication device 10. In the first embodiment, the "biometric image" is a facial image.

[0018] Furthermore, the authentication device 10 controls at least one of the second camera 107B and the illumination device 108, described later, based on predefined information (hereinafter referred to as "ambient light information") regarding the ambient light irradiated onto the person to be authenticated from various light sources such as the sun, to image the person to be authenticated. Specific examples of ambient light information include the illuminance and direction of irradiation of the ambient light. In the first embodiment, unless otherwise specified, ambient light includes illumination light.

[0019] The authentication server 20 is a computer that performs biometric authentication. The authentication server 20 comprises an authentication engine 21, an ambient light information database 22, and a registered user information database 23. The authentication engine 21 performs a comparison process between the biometric image (or feature data) of the person to be authenticated, captured by the authentication device 10, and the registered biometric image (or feature data) of the registered user pre-registered in the registered user information database 23, and performs biometric authentication of the person to be authenticated based on the comparison result.

[0020] The ambient light information database 22 stores ambient light information for each location where biometric authentication is performed. The registered user information database 23 stores information about registered users who are permitted to pass through the gate device 30.

[0021] The gate device 30 is a passage control device that controls the passage of people by opening and closing a gate (not shown) based on control information from the authentication device 10. When the authentication device 10 successfully authenticates a person, the gate device 30 transitions from a closed state that blocks the passage of people to an open state that allows the passage of people. The gate type is not particularly limited and may include, for example, a flapper gate in which flappers provided on one or both sides of the passage open and close, or a turnstile gate in which three bars rotate.

[0022] Figure 2 is a block diagram showing an example of the hardware configuration of the authentication device 10 according to the first embodiment. The authentication device 10 includes a processor 101, RAM (Random Access Memory) 102, ROM (Read Only Memory) 103, storage 104, communication I / F (Interface) 105, display 106, first camera 107A, second camera 107B, lighting device 108, distance sensor 109, and motor 110 as a computer that performs calculations, control, and storage. Each device is interconnected via buses, wiring, drive devices, etc., which are not shown.

[0023] The processor 101 has the function of performing predetermined calculations according to programs stored in the ROM 103, storage 104, etc., and controlling each part of the authentication device 10. The processor 101 can be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), etc. Furthermore, one of the above examples may be used, or multiple may be used in parallel.

[0024] RAM 102 is composed of a volatile storage medium and provides a temporary memory area necessary for the operation of the processor 101. RAM 102 may be, for example, D-RAM (Dynamic RAM). ROM 103 is composed of a non-volatile storage medium and stores necessary information such as programs used for the operation of the authentication device 10. ROM 103 may be, for example, P-ROM (Programmable ROM).

[0025] The storage device 104 consists of a non-volatile storage medium and is used for storing data, operating programs for the authentication device 10, and the like. The storage device 104 is composed of, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0026] The communication interface 105 is a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), 4G, or 5G, and is a module for communicating with other devices.

[0027] The processor 101 loads programs stored in ROM 103, storage 104, etc., into RAM 102 and executes them.

[0028] Display 106 is a display device that displays videos, still images, text, etc. Display 106 may display, for example, guidance information regarding authentication or authentication results. A liquid crystal display, an OLED (Organic Light Emitting Diode) display, etc., can be used as the display 106.

[0029] The first camera 107A is an imaging device that captures the entire authentication area. The first camera 107A includes a photodetector configured to be sensitive to infrared light. As the first camera 107A, a digital camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, etc., is used, suitable for image processing in the authentication device 10.

[0030] The second camera 107B is an imaging device that images a predetermined body part of the person being authenticated. The second camera 107B includes a photodetector configured to be sensitive to infrared light. As the second camera 107B, a digital camera using a CMOS image sensor, a CCD image sensor, etc., is used. The first camera 107A and the second camera 107B differ from each other in terms of the image target, resolution, etc. The imaging range of the first camera 107A may include the imaging range of the second camera 107B.

[0031] Furthermore, in this embodiment, the first camera 107A and the second camera 107B are infrared cameras, but the combination patterns of the first camera 107A and the second camera 107B are not limited to this. For example, the first camera 107A may be a visible light camera and the second camera 107B may be an infrared camera. Alternatively, the first camera 107A may be an infrared camera and the second camera 107B may be a visible light camera. Similarly, both the first camera 107A and the second camera 107B may be visible light cameras.

[0032] The illumination device 108 includes a light-emitting element that emits infrared light, such as an infrared LED. The wavelength of the infrared light emitted from the illumination device 108 may be, for example, in the near-infrared region of about 800 nm. The timing of illumination light emission from the illumination device 108 is synchronized with the imaging timing of the first camera 107A and the second camera 107B. However, the timing of illumination light emission from the illumination device 108 may not be synchronized with the imaging timing of each camera.

[0033] Furthermore, in this embodiment, the illumination device 108 is an infrared light irradiator provided in correspondence with the second camera 107B, which is an infrared camera, but it is not limited to this. For example, the illumination device 108 may be a visible light irradiator. Also, multiple illumination devices 108 may be provided to irradiate infrared light and visible light, respectively.

[0034] The distance sensor 109 is a measuring device that measures the distance from the authentication device 10 to the person to be authenticated. The distance sensor 109 may be capable of optically measuring the distance from the authentication device 10 to the person to be authenticated. Examples of optically capable distance measuring distance sensors 109 include TOF (Time Of Flight) type sensors, triangulation type sensors, and LiDAR (Light Detection and Ranging). In addition, a proximity sensor that detects when an object approaches without contact can also be used as the distance sensor 109.

[0035] The motor 110 is a drive device that drives an object connected to a rotating shaft (not shown). In the first embodiment, the second camera 107B is connected to the rotating shaft of the motor 110. Note that the second camera 107B does not have to be directly connected to the rotating shaft of the motor 110. For example, the second camera 107B may be indirectly connected to the rotating shaft via gears, belts, etc., and driven by the motor 110.

[0036] Note that the hardware configuration shown in Figure 2 is an example, and other devices may be added, or some devices may be omitted. Also, some devices may be replaced with other devices having similar functions. Furthermore, some functions of the first embodiment may be provided by other devices via a network, or the functions of the first embodiment may be implemented by distributing them across multiple devices. The illustrated hardware configuration can be modified as appropriate.

[0037] Figure 3 is a schematic diagram illustrating the authentication system 1 according to the first embodiment. As shown in Figure 3, the authentication system 1 is a walk-through type biometric authentication system that uses a first camera 107A and a second camera 107B to image a person to be authenticated as they move within the authentication area toward the authentication device 10. Although Figure 3 illustrates a case where biometric authentication is performed on the person to be authenticated three times in the order of points P3, P2, and P1, the points where biometric authentication is performed are not limited to these. Distance D is the distance from the position P0 of the authentication device 10 to the point P1 where the person to be authenticated T is standing.

[0038] Figure 4 is a front view showing the external appearance of the authentication device 10 according to the first embodiment. In Figure 4, the positional relationships of the components constituting the authentication device 10 are explained using a three-dimensional coordinate system composed of mutually orthogonal X, Y, and Z axes. The X and Y axes are axes in the horizontal plane. The X and Y axes are mutually orthogonal. The Z axis is an axis perpendicular to the horizontal plane.

[0039] In Figure 4, the display 106, first camera 107A, distance sensor 109, and second camera 107B are arranged from top to bottom along the center line of the authentication device 10. A pair of lighting devices 108 are also arranged on either side of the second camera 107B.

[0040] Figure 5 is a perspective view illustrating the internal structure of the authentication device 10 according to the first embodiment. As shown in Figure 5, the second camera 107B and the illumination device 108 are arranged inside the case 112 together with the motor 110. The first camera 107A has a wider field of view than the second camera 107B. The first camera 107A is not provided inside the case 112 in order to keep the imaging angle constant. The second camera 107B and the illumination device 108 are connected to the rotation axis A of the motor 110. Therefore, the second camera 107B and the illumination device 108 rotate together with the rotation axis A. Due to the rotation around the rotation axis A, the optical axis Ax of the second camera 107B changes vertically. In the first embodiment, the imaging angle of the second camera 107B refers to the angle of the optical axis Ax with respect to the horizontal plane.

[0041] When the motor 110 is driven, the second camera 107B rotates around the rotation axis A in the direction of arrow UP or arrow DW. When the second camera 107B rotates in the direction of arrow UP, the imaging range of the second camera 107B moves upward. Conversely, when the second camera 107B rotates in the direction of arrow DW, the imaging range of the second camera 107B moves downward.

[0042] Figure 6 shows an example of information stored in the ambient light information database 22 according to the first embodiment. The ambient light information database 22 includes data items such as the distance from the authentication device 10 to the ambient light receiving point, the ambient light source, the ambient light illuminance, and the direction of ambient light irradiation. The light source is the sun, the lighting device 108, and lighting devices other than the lighting device 108 of the authentication device 10. By referring to the ambient light information based on distance, it is possible to determine, for example, what illuminance the sunlight and the lighting light from the lighting device 108 will have when irradiating the person being authenticated at a distance of 2 meters in front of the authentication device 10. It is also possible to determine how the illuminance of the lighting light changes depending on the distance. The ambient light information is generated in advance based on, for example, measured values ​​actually measured using an illuminance sensor in the authentication area, or predicted values ​​calculated by computer simulations.

[0043] Figure 7 shows an example of the information stored in the registrant information database 23 according to the first embodiment. The registrant information database 23 includes data items such as a registrant ID to identify the registrant, name, age, gender, registered biometric image, and feature quantities.

[0044] Figure 8 is a functional block diagram showing the overall configuration of the authentication device 10 according to the first embodiment. The authentication device 10 comprises a face detection unit 11, a distance acquisition unit 12, an ambient light information acquisition unit 13, a brightness information acquisition unit 14, and a control unit 15.

[0045] The face detection unit 11 analyzes the image of the authentication area captured by the first camera 107A and detects the face of the person to be authenticated from the captured image. In the first embodiment, the face detection unit 11 detects the position and size of the face of the person to be authenticated from the captured image.

[0046] The distance acquisition unit 12 acquires the distance from the authentication device 10 to the person to be authenticated based on the detection signal from the distance sensor 109. In the first embodiment, since the second camera 107B and the lighting device 108 are located inside the housing of the authentication device 10, the positional relationship between the second camera 107B, the lighting device 108, and the person to be authenticated can be determined by the distance from the authentication device 10 to the person. That is, the positional relationship between the second camera 107B, the lighting device 108, and the person to be authenticated in the first embodiment corresponds to the positional relationship between the authentication device 10 and the person to be authenticated.

[0047] Furthermore, in the first embodiment, information such as the illuminance and direction of ambient light is pre-associated in the ambient light information database 22 with distance, that is, the positional relationship between the authentication device 10 (camera) and the person being authenticated. Therefore, the ambient light information acquisition unit 13 refers to the ambient light information database 22 based on the distance from the authentication device 10 to the person being authenticated and acquires ambient light information corresponding to that distance. The ambient light information may differ, for example, between a first point 3 meters away from the authentication device 10 and a second point 2 meters away from the authentication device 10.

[0048] The luminance information acquisition unit 14 analyzes the image captured by the first camera 107A of the authentication area or the biometric image of the person to be authenticated captured by the second camera 107B, and acquires luminance information for a predetermined body part of the person to be authenticated within the image. Examples of body parts for which luminance information is acquired include the face and eyes of the person to be authenticated. The luminance of a body part may be, for example, the average luminance value of the pixel group in the skin area of ​​the face.

[0049] The control unit 15 controls at least one of the second camera 107B (authentication device 10) and the illumination device 108 based on the positional relationship between the second camera 107B and the person to be authenticated, ambient light information including at least one of the illumination light emitted from the illumination device 108 and the ambient light surrounding the person to be authenticated, and brightness information of the person to be authenticated in the image captured by the second camera 107B. For example, the control unit 15 can determine the skin color of the person to be authenticated based on the brightness information and adjust the illuminance and wavelength of the illumination light emitted from the second camera 107B, or the imaging conditions of the second camera 107B. Similarly, the control unit 15 may adjust the illuminance and wavelength of the illumination light based on the eye color of the person to be authenticated. This ensures that the brightness of the biometric image captured by the second camera 107B falls within a predetermined range suitable for biometric authentication. By performing biometric authentication using a biometric image captured with appropriate brightness, it is possible to improve the authentication accuracy in biometric authentication.

[0050] The processor 101 loads programs stored in the ROM 103, storage 104, etc., into RAM and executes them. As a result, the processor 101 realizes the functions of the face detection unit 11, distance acquisition unit 12, ambient light information acquisition unit 13, brightness information acquisition unit 14, and control unit 15 described above.

[0051] Figure 9 is a flowchart illustrating the process performed by the authentication device 10 according to the first embodiment.

[0052] In step S101, the authentication device 10 acquires an image of the entire authentication area by capturing an image of the authentication area in front of the device with the first camera 107A.

[0053] In step S102, the authentication device 10 performs face detection on the captured image. Specifically, the authentication device 10 detects a region in the captured image that matches the shape pattern of a person's face. When the authentication device 10 detects the face of the person to be authenticated, the process proceeds to step S103.

[0054] In step S103, the authentication device 10 analyzes the captured image to obtain brightness information of the face region of the person to be authenticated. This allows the authentication device 10 to obtain the skin color of the face region of the person to be authenticated.

[0055] In step S104, the authentication device 10 measures the horizontal distance from the authentication device 10 to the person to be authenticated based on the detection signal from the distance sensor 109.

[0056] In step S105, the authentication device 10 determines the position of the face in three-dimensional space based on the coordinates of the face in the captured image and the distance measured in step S104. By determining the actual position of the face, it becomes possible to align the direction of the second camera 107B and the illumination device 108 with the person being authenticated. By aligning the direction of the second camera 107B and the illumination device 108 with the person being authenticated, it becomes possible to acquire a biometric image that is more suitable for authentication.

[0057] In step S106, the authentication device 10 refers to the ambient light information database 22 based on the distance measured in step S104 and obtains ambient light information for that distance. For example, if the person to be authenticated is 2 meters away from the authentication device 10, ambient light information corresponding to 2 meters is obtained.

[0058] In step S107, the authentication device 10 controls the second camera 107B and the illumination device 108 based on ambient light information and brightness information. Specifically, the authentication device 10 controls the exposure conditions, gain, imaging timing, imaging angle, etc., of the second camera 107B. The authentication device 10 also controls the illuminance, wavelength, irradiation direction, irradiation timing, etc., of the illumination light emitted from the illumination device 108.

[0059] Figures 10 and 11 are schematic diagrams illustrating the imaging angle and imaging range of the second camera 107B according to the first embodiment. In Figure 10, the position of the face of the person being authenticated T1 is higher than the center position of the second camera 107B in the vertical direction. Therefore, the imaging angle of the second camera 107B is set so that the optical axis Ax of the second camera 107B is upward from the horizontal. The imaging range R1 of the second camera 107B in Figure 10 includes the face of the person being authenticated T1.

[0060] In Figure 11, the height of the person being authenticated, T2, is lower than the height of the person being authenticated, T1. Also, the position of the face of the person being authenticated, T2, is lower vertically than the center position of the second camera 107B. Therefore, the imaging angle is set so that the optical axis Ax of the second camera 107B is pointed downwards from the horizontal. Consequently, the imaging range R2 of the second camera 107B in Figure 11 is shifted vertically downwards compared to the imaging range R1 shown in Figure 11. The imaging range R2 includes the face of the person being authenticated, T2.

[0061] In step S108, the authentication device 10 captures the face of the person to be authenticated using the second camera 107B and generates a face image.

[0062] In step S109, the authentication device 10 transmits the face image generated in step S108 to the authentication engine 21, causing the authentication engine 21 to perform face authentication. The authentication engine 21 compares the received face image with the registered face image of the registered user registered in the registered user information database 23, performs face authentication, and then transmits the authentication result to the authentication device 10.

[0063] In step S110, the authentication device 10 records the distance at which the face image was captured and the authentication result received from the authentication engine 21 in a storage device such as RAM 102 or storage 104, relating them to the image capture time. This provides time-series data of the distance of the person to be authenticated as they move through the authentication area and the authentication results.

[0064] In step S111, the authentication device 10 determines whether or not to complete the authentication of the person to be authenticated. For example, the authentication device 10 may determine whether or not to complete the authentication based on whether or not the number of authentication attempts has reached a predetermined number, or whether or not authentication has been successful at least once.

[0065] If the authentication device 10 determines that it has completed the authentication of the person to be authenticated (step S111: YES), the process proceeds to step S112. Conversely, if the authentication device 10 determines that it has not completed the authentication of the person to be authenticated (step S111: NO), the process returns to step S101. As a result, the authentication process for the person to be authenticated as they move within the authentication area towards the authentication device 10 is repeatedly executed.

[0066] In step S112, the authentication device 10 determines whether or not to allow the person to pass through the gate device 30 based on the authentication result recorded in the storage device for the person to be authenticated.

[0067] If the authentication device 10 determines that it is permissible for the person to be authenticated to pass through the gate device 30 (step S112: YES), it opens the gate (step S113) and terminates the process.

[0068] In response, if the authentication device 10 determines that it will not allow the person to pass through the gate device 30 (step S112: NO), it displays authentication error information on the display 106 (step S114) and terminates the process.

[0069] Conventional authentication systems, such as those exemplified in Patent Document 1, did not take into account the influence of ambient light at the point where biometric images were captured. As a result, the captured biometric images were sometimes unsuitable for biometric authentication. In contrast, according to the authentication system 1 of the first embodiment, the second camera 107B and the lighting device 108 can be controlled based on ambient light information predetermined for each distance and luminance information obtained from the captured image. This allows for appropriate adjustment of the conditions when the second camera 107B captures the face of the person to be authenticated, and enables the acquisition of a face image suitable for face authentication performed by the authentication engine 21.

[0070] [Second Embodiment] The authentication system 1 according to the second embodiment will be described below. The following description will mainly focus on the differences from the first embodiment, and the common parts will be omitted or simplified in its explanation.

[0071] Figure 12 is a schematic diagram illustrating the authentication system 1 according to the second embodiment. Figure 12 shows that facial recognition is performed on a person T being authenticated who is standing at point P1. In addition, a window 41 is provided in the outer wall 40 located behind the authentication device 10, at a position higher than the authentication device 10, allowing ambient light to enter.

[0072] Figure 13 is a block diagram showing an example of the overall configuration of the authentication system 1 according to the second embodiment. Unlike the first embodiment, the authentication device 10 includes a face detection unit 11, an acquisition unit 16, and a control unit 15. The authentication server 20 further includes a layout information database 24.

[0073] The acquisition unit 16 acquires ambient light information that represents factors that may affect the illuminance of the person being authenticated. In the second embodiment, the acquisition unit 16 acquires ambient light information including the date and time the person being authenticated is imaged and the layout information of the facility where the authentication area is provided. The layout information includes the optical characteristics, shape, size, and arrangement of structures provided in the facility. The layout information is defined based on coordinates in three-dimensional space.

[0074] Figure 14 shows an example of information stored in the ambient light information database 22 according to the second embodiment. The ambient light information database 22 includes location ID, date and time, weather, light source, illuminance, and irradiation direction as data items. For example, if the date and time is 2 PM on August 1st, ambient light information including the illuminance and irradiation direction of sunlight irradiating location P1, which is the authentication area, can be obtained from the ambient light information database 22 using the date and time as the key. The illuminance and irradiation direction of ambient light vary depending on the type of light source, the date and time of imaging, and the weather. In the second embodiment, it is assumed that information such as ambient light illuminance and irradiation direction in the ambient light information database 22 is not associated with distance (positional relationship).

[0075] Figure 15 shows an example of information stored in the layout information database 24 according to the second embodiment. The layout information database 24 includes data items such as location ID, location of light-gathering opening, size of light-gathering opening, location of occluding object, and size of occluding object. Ambient light information of ambient light illuminating the authentication area varies depending on the location and size of light-gathering openings and occluding objects in the facility where the authentication area is installed. Therefore, by combining ambient light information and layout information, highly accurate ambient light information regarding the illuminance of ambient light in the authentication area can be obtained.

[0076] Figure 16 is a flowchart illustrating the process performed by the authentication device 10 according to the second embodiment.

[0077] In step S201, the authentication device 10 acquires an image of the entire authentication area by capturing an image of the authentication area in front of the device with the first camera 107A.

[0078] In step S202, the authentication device 10 performs face detection processing on the captured image. When the authentication device 10 detects the face of the person to be authenticated, the process proceeds to step S203.

[0079] In step S203, the authentication device 10 refers to the ambient light information database 22 based on the imaging date and time and obtains ambient light information. Note that since sunlight conditions differ between sunny and rainy days even at the same date and time, the device may further obtain weather information from an external system based on the location information of the imaging site and the imaging date and time (current time), and then refer to the ambient light information database 22 based on the imaging date and time and weather. By considering the location, date and time, weather, etc. of the imaging site, a more appropriate biological image can be obtained.

[0080] In step S204, the authentication device 10 obtains layout information from the layout information database 24 using the location ID corresponding to the authentication area in the authentication device 10 as the key.

[0081] In step S205, the authentication device 10 acquires the final ambient light information in the authentication area based on the ambient light information and layout information.

[0082] In step S206, the authentication device 10 controls the second camera 107B and the illumination device 108 based on the acquired ambient light information. Specifically, the authentication device 10 controls the exposure conditions, gain, imaging timing, and imaging angle of the second camera 107B. The authentication device 10 also controls the illuminance, wavelength, irradiation direction, and irradiation timing of the illumination light emitted from the illumination device 108.

[0083] In step S207, the authentication device 10 captures the face of the person to be authenticated using the second camera 107B and generates a face image.

[0084] In step S208, the authentication device 10 transmits the face image generated in step S207 to the authentication engine 21, causing the authentication engine 21 to perform face authentication. The authentication engine 21 compares the received face image with the registered face image of the registered user registered in the registered user information database 23, performs face authentication, and then transmits the authentication result to the authentication device 10.

[0085] In step S209, the authentication device 10 determines whether or not to allow the person to pass through the gate device 30 based on the authentication result received from the authentication engine 21.

[0086] If the authentication device 10 determines that it is permissible for the person to be authenticated to pass through the gate device 30 (step S209: YES), it opens the gate (step S210) and terminates the process.

[0087] In response, if the authentication device 10 determines that it will not allow the person to pass through the gate device 30 (step S209: NO), it displays authentication error information on the display 106 (step S211) and terminates the process.

[0088] As described above, according to the authentication system 1 of the second embodiment, ambient light information can be acquired based on the date and time of acquisition of the biometric image and the layout information of the facility where biometric authentication is performed, and the second camera 107B and the lighting device 108 can be controlled accordingly. This allows for appropriate adjustment of the conditions when the second camera 107B captures the face of the person to be authenticated, and a face image suitable for face authentication performed by the authentication engine 21 can be obtained. For example, if the acquisition date and time is 15:00 and sunlight is shone from the facility's window 41 at an irradiation angle θ onto the authentication point, the second camera 107B and the lighting device 108 can be controlled accordingly to suppress the influence of sunlight based on the positional relationship between the window 41 and the authentication point.

[0089] [Third Embodiment] The following describes the authentication system 1 according to the third embodiment. The following description will primarily focus on the differences from the first embodiment, while common parts will be omitted or simplified.

[0090] Figure 17 is a flowchart illustrating the process performed by the authentication device according to the third embodiment. In Figure 17, steps S101 to S110 and steps S112 to S114 are the same as in the first embodiment, but steps S301 to S303 differ from the first embodiment.

[0091] Once the process in step S110 is completed, the process moves on to step S301. In step S301, the authentication device 10 determines whether or not to complete the authentication of the person to be authenticated.

[0092] If the authentication device 10 determines that it has completed the authentication of the person to be authenticated (step S301: YES), the process proceeds to step S112. On the other hand, if the authentication device 10 determines that it has not completed the authentication of the person to be authenticated (step S301: NO), the process proceeds to step S302.

[0093] In step S302, the authentication device 10 acquires the imaging conditions of the second camera 107B when imaging the face of the person to be authenticated.

[0094] In step S303, the authentication device 10 updates the ambient light information in the ambient light information database 22 based on the imaging conditions, positional relationship, and authentication result. For example, if the illuminance of the ambient light during face imaging differs from what was expected and authentication fails, the authentication device 10 estimates the actual illuminance of the ambient light by analyzing the captured image. The authentication device 10 then updates the ambient light information database 22 with the estimated ambient light information. By considering the combination of imaging conditions, positional relationship, and authentication result, the authentication device 10 can control the second camera 107B and the lighting device 108 in detail and with high precision. When the processing in step S303 is completed, the process returns to step S101.

[0095] As described above, according to the authentication system 1 of the first embodiment, the ambient light information in the ambient light information database 22 can be flexibly changed based on the imaging conditions and authentication results, thereby improving the accuracy of subsequent biometric authentication.

[0096] [Fourth Embodiment] The following describes the authentication system 1 according to the fourth embodiment. The following description will primarily focus on the differences from the first embodiment, while common parts will be omitted or simplified.

[0097] Figure 18 is a block diagram showing an example of the hardware configuration of the authentication device 10 according to the fourth embodiment. Unlike in Figure 2, Figure 18 shows that the authentication device 10 includes a first illumination device 108A and a second illumination device 108B. The first illumination device 108A is an illumination device that irradiates infrared light toward the face of the person to be authenticated when capturing an image of the person's face. The second illumination device 108B is an illumination device that irradiates infrared light toward the eye of the person to be authenticated when capturing an image of the iris of the person to be authenticated.

[0098] Figure 19 is a flowchart illustrating the process performed by the authentication device 10 according to the fourth embodiment. In Figure 19, steps S101 to S105 and steps S110 to S114 are the same as in the first embodiment, but steps S401 to S407 differ from the first embodiment.

[0099] Once the process in step S105 is completed, the process moves to step S401. In step S401, the authentication device 10 determines whether or not to perform facial recognition on the person to be authenticated.

[0100] If the authentication device 10 determines to perform facial recognition on the person to be authenticated (step S401: YES), the process proceeds to step S402. Conversely, if it determines to perform iris recognition without performing facial recognition on the person to be authenticated (step S401: NO), the process proceeds to step S405.

[0101] In step S402, the authentication device 10 controls the first camera 107A and the first lighting device 108A based on ambient light information and brightness information.

[0102] In step S403, the authentication device 10 captures the face of the person to be authenticated using the first camera 107A and generates a face image.

[0103] In step S404, the authentication device 10 transmits the face image generated in step S403 to the authentication engine 21, causing the authentication engine 21 to perform face authentication. The authentication engine 21 compares the received face image with the registered face image of the registered user registered in the registered user information database 23, performs face authentication, and then transmits the authentication result to the authentication device 10. After that, the process proceeds to step S110.

[0104] In step S405, the authentication device 10 controls the second camera 107B and the second lighting device 108B based on ambient light information and brightness information.

[0105] In step S406, the authentication device 10 captures the iris of the person to be authenticated using the second camera 107B and generates an iris image.

[0106] In step S407, the authentication device 10 transmits the iris image generated in step S406 to the authentication engine 21, causing the authentication engine 21 to perform iris authentication. The authentication engine 21 compares the received iris image with the registered iris image of the registered user registered in the registered user information database 23, performs iris authentication, and then transmits the authentication result to the authentication device 10. The process then proceeds to step S110.

[0107] As described above, the authentication system 1 according to the fourth embodiment can achieve two-factor authentication of facial recognition and iris recognition, unlike the first embodiment. Furthermore, the first illumination device 108A is controlled when capturing the face of the person to be authenticated, and the second illumination device 108B is controlled when capturing the iris. Since the illumination light can be appropriately switched according to the type of biometric authentication to be performed, biometric images suitable for facial recognition and iris recognition can be obtained. [Fifth Embodiment]

[0108] The following describes the authentication system 1 according to the fifth embodiment. The following description will primarily focus on the differences from the first embodiment, while common parts will be omitted or simplified.

[0109] Figure 20 is a flowchart illustrating the process performed by the authentication device according to the fifth embodiment. In Figure 20, steps S101 to S105 and steps S108 to S114 are the same as in the first embodiment, but steps S501 to S505 differ from the first embodiment.

[0110] Once the processing in step S105 is complete, the process moves on to step S501. In step S501, the authentication device 10 determines whether or not the acquisition of time-series data related to distance and face position has been completed.

[0111] If the authentication device 10 determines that the acquisition of time-series data is complete (step S501: YES), the process proceeds to step S502. Conversely, if the authentication device 10 determines that the acquisition of time-series data is not complete (step S501: NO), the process returns to step S101.

[0112] In step S502, the authentication device 10 calculates the movement speed of the person to be authenticated based on the time-series data.

[0113] In step S503, the authentication device 10 estimates the position of the face after movement and the distance from the authentication device 10 based on the movement speed calculated in step S502. The position of the face refers to the position coordinates in three-dimensional space.

[0114] In step S504, the authentication device 10 obtains ambient light information from the ambient light information database 22 based on the estimated distance traveled.

[0115] In step S505, the authentication device 10 controls the second camera 107B and the lighting device 108 based on ambient light information and brightness information. The process then proceeds to step S108.

[0116] As described above, according to the authentication system 1 of the fifth embodiment, ambient light information at the destination can be acquired based on the estimated movement speed of the person to be authenticated. This allows the second camera 107B and the lighting device 108 to be appropriately adjusted, thereby further improving the quality of the acquired biological images.

[0117] [Sixth Embodiment] The authentication system 1 according to the sixth embodiment will be described below. The following description will mainly focus on the differences from the first embodiment, and the common parts will be omitted or simplified.

[0118] Figure 21 is a flowchart illustrating the process performed by the authentication device according to the sixth embodiment. In Figure 21, steps S101 to S105 and steps S108 to S114 are the same as in the first embodiment, but steps S601 to S605 differ from the first embodiment.

[0119] Once the processing in step S105 is complete, the process moves on to step S601. In step S601, the authentication device 10 determines whether or not the acquisition of time-series data related to distance and face position has been completed.

[0120] If the authentication device 10 determines that the acquisition of time-series data is complete (step S601: YES), the process proceeds to step S602. Conversely, if the authentication device 10 determines that the acquisition of time-series data is not complete (step S601: NO), the process returns to step S101.

[0121] In step S602, the authentication device 10 identifies the gait of the person to be authenticated based on time-series data. Gait is a person's walking pattern. Gait differs from person to person in terms of arm swing, stride length, posture, left-right asymmetry of movement, etc.

[0122] In step S603, the authentication device 10 estimates the position of the face and the distance from the authentication device 10 after movement, based on the gait of the person to be authenticated identified in step S602. That is, the authentication device 10 estimates the location of the person to be authenticated when the next biometric image is to be captured, as the person is moving within the authentication area.

[0123] In step S604, the authentication device 10 obtains ambient light information from the ambient light information database 22 based on the estimated distance traveled.

[0124] In step S605, the authentication device 10 controls the second camera 107B and the lighting device 108 based on ambient light information and brightness information. The process then proceeds to step S108.

[0125] As described above, according to the authentication system 1 of the sixth embodiment, ambient light information can be acquired based on the gait of the person being authenticated. This allows the second camera 107B and the lighting device 108 to be appropriately adjusted, thereby further improving the quality of the acquired biological images.

[0126] [Seventh Embodiment] The following describes the authentication system 1 according to the seventh embodiment. The following description will primarily focus on the differences from the first embodiment, while common parts will be omitted or simplified.

[0127] Figure 22 is a functional block diagram showing the overall configuration of the authentication device 10 according to the seventh embodiment. In Figure 22, the authentication device 10 includes a face detection unit 11, a distance acquisition unit 12, an ambient light information acquisition unit 13, a brightness information acquisition unit 14, and a control unit 15, in addition to a determination unit 17.

[0128] The determination unit 17 analyzes the captured image taken by the first camera 107A and determines whether or not an item worn by the person being authenticated (hereinafter referred to as "wearing item") has any effect on the biological image. Specifically, the determination unit 17 determines the presence or absence of a wearing item in the captured image, the type of wearing item, the size of the wearing item, the transmission state of illumination light (infrared light) or ambient light through the wearing item, the reflection state of illumination light or ambient light, the presence or absence of shadows cast by the wearing item, etc.

[0129] Figures 23, 24, and 25 are flowcharts illustrating the general processes performed by the authentication device according to the seventh embodiment.

[0130] Figure 23 shows the process for controlling the second camera 107B and the illumination device 108 based on the transmission state of illumination light through the wearable device worn by the person being authenticated. In Figure 23, steps S101 to S106 and steps S108 to S114 are the same as in the first embodiment, but steps S701 to S704 differ from the first embodiment.

[0131] Once the processing in step S106 is completed, the process moves on to step S701. In step S701, the authentication device 10 analyzes the captured image and determines whether or not there is an object being worn on the head of the person to be authenticated.

[0132] If the authentication device 10 determines that there is an object attached to the head of the person to be authenticated (step S701: YES), the process proceeds to step S702. Conversely, if the authentication device 10 determines that there is no object attached to the head of the person to be authenticated (step S701: NO), the process proceeds to step S704.

[0133] In step S702, the authentication device 10 captures an image of the person being authenticated along with any worn items. Examples of worn items include hats, glasses, sunglasses, and helmets.

[0134] In step S703, the authentication device 10 analyzes the facial image captured in step S702 and determines the transmission state of illumination light (infrared light) through the worn object. For example, if the worn object is eyeglasses or sunglasses, the transmission state of infrared light can be determined by the pixel values ​​in the lens portion.

[0135] In step S704, the authentication device 10 controls the second camera 107B and the illumination device 108 based on the determination result of the transmission state of the illumination light (infrared light) in step S703. For example, if the worn item is sunglasses with black lenses and it is determined that infrared light of a predetermined wavelength does not transmit through them, the authentication device 10 preferably changes the wavelength range of the infrared light emitted from the illumination device 108. Alternatively, the wavelength range of light received by the second camera 107B may be changed. By considering the transmission state of the illumination light, high-quality biometric images suitable for biometric authentication can be obtained.

[0136] Figure 24 shows the process for controlling the second camera 107B and the illumination device 108 based on the reflection state of illumination light on the attached object. The process in Figure 24 differs from that in Figure 23 in steps S801 and S802.

[0137] Once the processing in step S702 is complete, the process moves on to step S801. In step S801, the authentication device 10 analyzes the facial image captured in step S702 and determines the reflection state of the illumination light on the surface of the worn object.

[0138] In step S802, the authentication device 10 controls the second camera 107B and the illumination device 108 based on the determination result of the illumination light reflection state in step S801. For example, if the worn item is eyeglasses and reflects the illumination light, it is preferable for the authentication device 10 to control the imaging angle of the second camera 107B to be changed. The authentication device 10 may also change the illuminance and irradiation angle of the illumination light emitted from the illumination device 108. By considering the reflection state of infrared light, biological images can be acquired more appropriately.

[0139] Figure 25 shows the process for controlling the second camera 107B and the lighting device 108 based on the presence or absence of shadows cast by the attached object. The process in Figure 25 differs from that in Figure 23 in steps S901 and S902.

[0140] Once the processing in step S702 is completed, the process moves on to step S901. In step S901, the authentication device 10 analyzes the captured image and determines whether there is a shadow of the worn object on the body part of the person to be authenticated, as well as the position and size of the shadow.

[0141] In step S902, the authentication device 10 controls the second camera 107B and the illumination device 108 based on information regarding the shadow of the worn object. For example, if the worn object is a hat and the shadow of the hat overlaps with the face area of ​​the person being authenticated in the image, it is preferable for the authentication device 10 to control the imaging angle of the second camera 107B to change. The authentication device 10 may also change the illuminance and irradiation angle of the infrared light emitted from the illumination device 108. By considering information regarding the shadow of the worn object, a more appropriate biological image can be acquired.

[0142] [Eighth Embodiment] The authentication system 1 according to the eighth embodiment will be described below. The following description will mainly focus on the differences from the first embodiment, and the common parts will be omitted or simplified.

[0143] Figure 26 is a functional block diagram showing the overall configuration of the authentication device 10 according to the eighth embodiment. The authentication device 10 includes a face detection unit 11, a distance acquisition unit 12, an ambient light information acquisition unit 13, a brightness information acquisition unit 14, a control unit 15, and a determination unit 17, in addition to a learning unit 18.

[0144] The learning unit 18 takes as input the presence or absence of items worn by the person to be authenticated in the captured image, the presence or absence of shadows on the person's body parts, and the relationship between the size of the shadows, and generates a learning model that outputs control information for the second camera 107B and the illumination device 108. Because the learning model is generated using this learning data, high-quality biometric images suitable for biometric authentication can be obtained. Note that the data used for the learning process is not limited to these. The input data can be any element that changes the brightness information of the captured image. For example, the input data may include displacement data of the imaging angle of the second camera 107B when capturing a face, and displacement data of the positional relationship between the authentication device 10 (second camera 107B and illumination device 108) and the person to be authenticated during imaging. When learning displacement data of imaging angle and brightness information, it is possible to suppress the deterioration of biometric image quality due to rapid displacement of the imaging angle. Similarly, when learning displacement data of positional relationship and brightness information, it is possible to suppress the deterioration of biometric image quality due to displacement of the positional relationship.

[0145] Figure 27 is a schematic diagram illustrating a neural network used in the learning process according to the eighth embodiment. The neural network shown in Figure 27 comprises an input layer having multiple nodes, a hidden layer having multiple nodes, and an output layer having one node. Multiple types of biometric information, which are input values, are input to each node of the input layer. Each node of the hidden layer is connected to each node of the input layer. Each element of the input values ​​input to the nodes of the hidden layer is used in calculations at each node of the hidden layer. Each node of the hidden layer calculates a calculated value using, for example, the input values ​​input from each node of the input layer, a predetermined weighting coefficient, and a predetermined bias value. Each node of the hidden layer is connected to the output layer and outputs the calculated calculated value to the node of the output layer. The node of the output layer receives the calculated values ​​from each node of the hidden layer.

[0146] The output layer nodes output a value indicating optimal control information y using the calculated values ​​input from each node in the intermediate layer, weighting coefficients, and bias values. Examples of control information y for the second camera 107B include exposure conditions, gain, imaging timing, and imaging angle. Examples of control information y for the illumination device 108 include illuminance, wavelength, irradiation direction, and irradiation timing of the illumination light.

[0147] When training a neural network, methods such as backpropagation are used. Specifically, the output value when data is input to the input layer is compared with the output value obtained from the training data, and the error between the two output values ​​is fed back to the hidden layer. This is repeated until the error falls below a predetermined threshold. Through this training process, a learning model is generated that can output optimal control information y when data about the wearer of the person being authenticated is input.

[0148] Figures 28 and 29 are flowcharts illustrating the general processes performed by the authentication device 10 according to the eighth embodiment.

[0149] Figure 28 shows an example of the learning process in the learning unit 18. This process can be executed independently of the process shown in Figure 29, which will be described later.

[0150] In step S1001, the authentication device 10 inputs a biometric image (face image) of the person to be authenticated.

[0151] In step S1002, the authentication device 10 obtains the distance from the authentication device 10 to the person to be authenticated at the time of capturing the biometric image.

[0152] In step S1003, the authentication device 10 analyzes the biometric image and obtains brightness information of the face of the person to be authenticated.

[0153] In step S1004, the authentication device 10 analyzes the biometric image and determines whether there is a shadow of the worn object in the facial area of ​​the person to be authenticated, the position of the shadow, and the size of the shadow.

[0154] In step S1005, the authentication device 10 analyzes the biometric image and determines the reflection state of the illumination light on the worn object.

[0155] In step S1006, the authentication device 10 obtains ambient light information from the ambient light information database 22 based on distance.

[0156] In step S1007, the authentication device 10 acquires the imaging conditions of the second camera 107B, which were determined before imaging the face, and the illumination conditions of the illumination light in the illumination device 108.

[0157] In step S1008, the authentication device 10 obtains the authentication result of face recognition based on the face image captured under the conditions of step S1007.

[0158] In step S1009, the authentication device 10 generates a learning model based on luminance information, ambient light information, distance, information about the worn object, the reflection state of the illumination light, control information, and the authentication result.

[0159] Figure 29 shows an example of a process that controls the second camera 107B and the lighting device 108 based on the learning model generated by the learning process shown in Figure 28. In Figure 29, steps S101 to S106 and steps S108 to S114 are the same as in the first embodiment, but steps S1101 to S1102 are different from the first embodiment.

[0160] Once the processing in step S106 is completed, the process moves on to step S1101. In step S1101, the authentication device 10 analyzes the facial image of the person to be authenticated extracted from the captured image and obtains information about the worn items.

[0161] In step S1102, the authentication device 10 inputs luminance information, ambient light information, distance, information about the worn object, the reflection state of the illumination light, control information, and the authentication result into the learning model, and outputs the control information output from the learning model. Based on the control information output from the learning model, the authentication device 10 controls the second camera 107B and the illumination device 108. After that, the process proceeds to step S108.

[0162] As described above, according to the authentication system 1 of the eighth embodiment, the second camera 107B and the lighting device 108 are controlled based on a learning model that has learned the relationship between information about the wearable object and the factors that the wearable object has on the illuminance of ambient light, thereby enabling the acquisition of a biometric image suitable for biometric authentication. Furthermore, based on the accuracy of biometric authentication, the learning model may be generated to prioritize the reduction of one of the two effects, the effect of shadow generation and the effect of reflection, over the other. This enables the acquisition of a high-quality biometric image suitable for biometric authentication.

[0163] [Ninth Embodiment] The authentication system 1 according to the ninth embodiment will be described below. The following description will mainly focus on the differences from the first embodiment, and the common parts will be omitted or simplified in its explanation.

[0164] Figure 30 is a flowchart illustrating the process performed by the authentication device 10 according to the ninth embodiment. In Figure 30, steps S101 to S107 and steps S109 to S114 are the same as in the first embodiment, but steps S1201 to S1202 are different from those in the first embodiment.

[0165] Once the processing in step S107 is completed, the process moves to step S1201. In step S1201, the authentication device 10 calculates the quality value of the face image generated in step S107. For example, the authentication device 10 can calculate the quality value of the face image by comparing the brightness information of the acquired face image with the brightness information of a registered face image that has been previously confirmed to be suitable for face authentication (registered brightness information). In this case, the quality value of the biometric image can be easily calculated. Alternatively, the authentication device 10 may calculate the quality value of the face image based on the pixel values ​​of the generated face image. The method for calculating the quality value can be arbitrarily determined. The authentication device 10 may, for example, correct the quality value based on the amount of illuminance displacement of the illumination light per unit time. In this case, the deterioration of the biometric image quality due to rapid displacement of the illumination light can be suppressed.

[0166] In step S1202, the authentication device 10 determines whether the calculated quality value is equal to or greater than a predetermined threshold.

[0167] If the authentication device 10 determines that the quality value is above a predetermined threshold (step S1202: YES), the process proceeds to step S109. Then, facial recognition is performed (step S109). On the other hand, if the authentication device 10 determines that the quality value is below the threshold (step S1202: NO), the process returns to step S101.

[0168] As described above, according to the authentication system 1 of the ninth embodiment, biometric authentication can be performed if the quality value of the generated biometric image meets predetermined conditions, and not performed if the quality value does not meet the same conditions. Since unnecessary biometric authentication can be avoided and the system can quickly proceed to the next imaging process, the number of times biometric authentication based on high-quality biometric images can be performed can be increased.

[0169] [Tenth Embodiment] Figure 31 is a functional block diagram showing the overall configuration of the information processing device 100 according to the tenth embodiment. The information processing device 100 comprises an acquisition unit 100A and a control unit 100B. The acquisition unit 100A acquires the positional relationship between the subject of biometric authentication and the imaging device that images the subject, ambient light information including at least one of the illumination light irradiated onto the subject by the illumination device and the ambient light around the subject, and brightness information of the subject in the image captured by the imaging device. The control unit 100B controls at least one of the illumination device and the imaging device based on the positional relationship, ambient light information, and brightness information. According to the tenth embodiment, an information processing device 100 capable of acquiring a biometric image suitable for biometric authentication is provided.

[0170] [Embodiment No. 11] Figure 32 is a functional block diagram showing the overall configuration of the information processing device 200 according to the 11th embodiment. The information processing device 200 comprises an acquisition unit 200A and a control unit 200B. The acquisition unit 200A acquires ambient light information representing factors that may affect the illuminance of the person subject to biometric authentication. Based on the ambient light information, the control unit 200B controls at least one of an illumination device that irradiates the subject with illumination light and an imaging device that images the subject. According to the 11th embodiment, an information processing device 200 capable of acquiring a biometric image suitable for biometric authentication is provided.

[0171] [Modified Embodiment] This disclosure is not limited to the embodiments described above and can be modified as appropriate without departing from the spirit of this disclosure. For example, an example in which some components of one embodiment are added to another embodiment, or in which some components of another embodiment are replaced, is also an embodiment of this disclosure.

[0172] The above-described embodiment explains the case where facial recognition is performed in the authentication system 1, but the technology disclosed can also be applied to a system that performs iris recognition. If the authentication system 1 is an iris recognition system, the second camera 107B can be replaced with an iris camera. By irradiating the person's eye with infrared light from the illumination device 108 and capturing the infrared light reflected by the iris with the iris camera, an iris image used for iris recognition can be obtained.

[0173] In the embodiment described above, a configuration was described in which the distance from the authentication device 10 to the person to be authenticated is measured based on the detection signal from the distance sensor 109, but the method of measuring the distance is not limited to this. For example, the actual distance from the authentication device 10 to the person to be authenticated may be estimated based on the inter-eye distance of the person to be authenticated in the captured image. In this case, the distance sensor 109 can be omitted, and manufacturing costs can be reduced.

[0174] Furthermore, although the above-described embodiment described the case where both the lighting device 108 and the second camera 107B are controlled, it is sufficient to control at least one of the lighting device 108 and the second camera 107B. In other words, only the lighting device 108 or the second camera 107B may be controlled.

[0175] Furthermore, the authentication device 10 may control at least one of the illumination device 108 and the second camera 107B based on the positional relationship in the previously performed biometric authentication, the illuminance of the illumination light emitted by the illumination device, and changes in brightness information. For example, if biometric authentication for the first person fails at a distance of 3 meters because the illumination light is too dim, the next biometric authentication for the second person can be controlled to brighten the illumination light illuminating the second person at a distance of 3 meters. This makes it possible to obtain a high-quality biometric image suitable for biometric authentication.

[0176] Furthermore, the authentication device 10 may control at least one of the lighting device 108 and the second camera 107B based on the changes in the matching score (authentication result) and positional relationship in the previously performed biometric authentication. Specifically, the matching score and distance displacement data for the first person to be authenticated are recorded, and when capturing a biometric image for biometric authentication of the second person after the first person, the lighting device 108 and the second camera 107B can be controlled based on the matching score and distance displacement data for the first person. Also, if the matching score in biometric authentication based on the biometric image captured at a point 2 meters away from the authentication device 10 for the first person is high, that point can be considered the best point, and the control parameters of the lighting device 108 at that point can be modified for the next second person. This makes it possible to obtain high-quality biometric images suitable for biometric authentication.

[0177] In the embodiment described above, the imaging angle and imaging range were changed by driving the second camera 107B itself, but the configuration for changing the imaging angle and imaging range is not limited to this. For example, a configuration using a rotating mirror may be applied.

[0178] Figures 33 and 34 are schematic diagrams illustrating how to change the imaging angle and imaging range of the authentication device 10 according to a modified embodiment. The rotating mirror 111 is provided so as to be drivable in the vertical direction with respect to the horizontal, and is a member that reflects light incident from outside the authentication device 10 toward the second camera 107B. As shown in Figures 33 and 34, the first camera 107A is arranged in the horizontal direction.

[0179] On the other hand, the light-receiving surface of the second camera 107B is positioned vertically downward so as to face the reflective surface of the rotating mirror 111. The rotating mirror 111 is connected to the rotation axis 110a of the motor 110 and can rotate together with the rotation axis 110a. Figure 33 shows the state before the rotating mirror 111 is driven. On the other hand, Figure 34 shows the state after the rotating mirror 111 has rotated counterclockwise from the state in Figure 33. The angle at which the optical axis Ax of the second camera 107B intersects the horizontal plane HP is -θ3. That is, the imaging angle of the second camera 107B in Figure 34 is -θ3.

[0180] Furthermore, the imaging range R10 of the first camera 107A is wider than the imaging range R20 of the second camera 107B. In other words, the field of view of the first camera 107A is larger than the field of view of the second camera 107B. The authentication device 10 moves the imaging range R20 of the second camera 107B by changing the tilt angle of the rotating mirror 111 using the motor 110. For example, the authentication device 10 can move the imaging range R20 of the second camera 107B up and down along the vertical direction by rotating the rotating mirror 111.

[0181] According to the modified embodiment of the authentication system 1, it is not necessary to drive the second camera 107B itself in order to change the imaging angle of the second camera 107B. By driving the rotating mirror 111 instead of the second camera 107B, a biometric image used for biometric authentication can be captured at an appropriate imaging angle. Note that the rotating mirror 111 does not have to be directly connected to the rotation shaft 110a of the motor 110. For example, the rotating mirror 111 may be indirectly connected to the rotation shaft 110a via gears, belts, etc., and driven by the motor 110.

[0182] In the first embodiment described above, an authentication device 10 was described in which an imaging device (first camera 107A and second camera 107B) and an illumination device 108 were integrated. However, the imaging device and the illumination device 108 may be provided independently of each other. Figure 35 is a diagram showing an example of the positional relationship between the imaging device, the illumination device 108, and the person to be authenticated T in a modified embodiment. Here, the illumination device 108 is installed at point P10, which is a distance D10 from point P0 where the authentication device 10 is installed. In Figure 35, the illumination device 108 is suspended from the ceiling (not shown), but the method of installing the illumination device 108 is not limited to this. Also, the positional relationship between the illumination device 108, the imaging device, and the person to be authenticated T is not limited to the positional relationship shown in Figure 35. For example, the illumination device 108 may be installed at a position further from the person to be authenticated T than the imaging device. Also, the illumination device 108 may be installed above or below the imaging device. If the imaging device and the illumination device 108 are installed at a distance from each other, the authentication system can store ambient light information in a database in association with the positional relationship between the imaging device, the illumination device 108, and the person to be authenticated T. By determining the imaging conditions based on the ambient light information associated with the positional relationship, the brightness of the biometric image captured by the imaging device can be kept within a predetermined range suitable for biometric authentication. Furthermore, by performing biometric authentication using a biometric image captured with appropriate brightness, it is possible to improve the authentication accuracy in biometric authentication.

[0183] Each embodiment also includes a method for recording a program on a storage medium that operates the configuration of the embodiment to realize the functions of the embodiment described above, reading the program recorded on the storage medium as code, and executing it on a computer. In other words, a computer-readable storage medium is also included in the scope of each embodiment. Furthermore, not only the storage medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment. In addition, one or more components included in the above-described embodiment may be circuits such as ASICs or FPGAs configured to realize the functions of each component.

[0184] Examples of storage media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD (Compact Disk)-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that perform processing on the storage medium alone, but also includes programs that operate on an OS (Operating System) in cooperation with other software and the functions of expansion boards to perform processing.

[0185] The services realized by the functions of each of the above embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0186] Furthermore, the embodiments described above are merely examples of concrete implementations of this disclosure, and the technical scope of this disclosure should not be interpreted as being limited by them. In other words, this disclosure can be implemented in various ways without departing from its technical concept or its main features.

[0187] Some or all of the embodiments described above may also be described as follows, but are not limited to the following.

[0188] (Note 1) An acquisition unit that acquires the positional relationship between the person to be biometrically authenticated and the imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by the lighting device and the ambient light around the person, and brightness information of the person in the image captured by the imaging device, A control unit controls at least one of the illumination device and the imaging device based on the positional relationship, ambient light information and brightness information, An information processing device equipped with the following features.

[0189] (Note 2) The ambient light information is pre-associated with the positional relationship, The control unit updates the ambient light information based on the imaging conditions of the captured image and the result of the biometric authentication performed based on the captured image. The information processing device described in Appendix 1.

[0190] (Note 3) The control unit controls at least one of the illumination device and the imaging device based on the imaging conditions of the captured image, the positional relationship at the time of imaging, and the result of the biometric authentication performed based on the captured image. The information processing device described in Appendix 1.

[0191] (Note 4) The control unit illuminates the lighting device among the plurality of lighting devices that corresponds to the type of biometric authentication performed in the given positional relationship. An information processing device as described in any of the appendices 1 to 3.

[0192] (Note 5) The control unit controls at least one of the illumination device and the imaging device based on the color of the subject's body part indicated by the brightness information. An information processing device as described in any of the appendices 1 to 3.

[0193] (Note 6) The control unit controls the lighting device and at least one of the imaging device based on at least one of the position of the subject's face in three-dimensional space identified from the captured image, the subject's movement speed identified from the positional relationship, and the subject's gait identified from the positional relationship. An information processing device as described in any of the appendices 1 to 4.

[0194] (Note 7) The system further includes a determination unit that determines the transmission state of the illumination light through the item worn by the subject based on the captured image, The control unit controls at least one of the irradiation wavelength range of the illumination light from the illumination device and the light reception wavelength range in the imaging device based on the determination result of the transmission state. An information processing device as described in any of the appendices 1 to 6.

[0195] (Note 8) The system further includes a determination unit that determines the reflection state of the illumination light on the item worn by the subject based on the captured image, The control unit controls at least one of the illumination device and the imaging device based on the reflection state. An information processing device as described in any of the appendices 1 to 6.

[0196] (Note 9) A determination unit that determines, based on the captured image, whether or not the subject is wearing an item and whether or not the shadow of the item is on the subject's body part, A learning unit that learns the relationship between the presence or absence of the aforementioned article, the aforementioned positional relationship, and the presence or absence of the aforementioned shadow, Furthermore, The control unit controls at least one of the illumination device and the imaging device based on the learning results in the learning unit. An information processing device as described in any of the appendices 1 to 6.

[0197] (Note 10) A determination unit that determines, based on the captured image, whether or not the subject is wearing an item, the reflection state of the illumination light on the item, and whether or not the item casts a shadow on the subject's body part, A learning unit that learns the relationship between the presence or absence of the aforementioned article, the positional relationship, the reflection state, and the presence or absence of the aforementioned shadow, Furthermore, The control unit controls at least one of the illumination device and the imaging device to eliminate reflections based on the learning results of the learning unit. An information processing device as described in any of the appendices 1 to 6.

[0198] (Note 11) The system further comprises a learning unit that learns the relationship between the displacement of the imaging angle of the imaging device, the displacement of the brightness information, the presence or absence of an item worn by the subject, the presence or absence of a shadow of the item on the subject's body parts, and the reflection state of the illumination light on the item. The control unit controls at least one of the illumination device and the imaging device based on the learning results of the learning unit. An information processing device as described in any of the appendices 1 to 6.

[0199] (Note 12) The control unit calculates a quality value of the captured image based on the comparison result between the brightness information and predetermined registered brightness information, and requests the authentication device to perform the biometric authentication based on the quality value. An information processing device as described in any of the appendices 1 to 11.

[0200] (Note 13) The control unit corrects the quality value of the captured image based on the amount of illuminance displacement of the illumination light per unit time, and requests the authentication device to perform the biometric authentication based on the quality value. An information processing device as described in any of the appendices 1 to 11.

[0201] (Note 14) The control unit controls at least one of the lighting device and the imaging device based on the changes in the positional relationship in the completed biometric authentication, the illuminance of the illumination light emitted by the lighting device, and the brightness information. An information processing device as described in any of the appendices 1 to 13.

[0202] (Note 15) The control unit controls at least one of the illumination device and the imaging device based on the changes in the matching score and positional relationship in the biometric authentication that has been performed. An information processing device as described in any of the appendices 1 to 13.

[0203] (Note 16) The steps include obtaining the positional relationship between the person to be biometrically authenticated and the imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by the lighting device and the ambient light around the person, and brightness information of the person in the image captured by the imaging device. A step of controlling at least one of the illumination device and the imaging device based on the positional relationship, ambient light information and brightness information, An information processing method comprising the following:

[0204] (Note 17) On the computer, The steps include obtaining the positional relationship between the person to be biometrically authenticated and the imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by the lighting device and the ambient light around the person, and brightness information of the person in the image captured by the imaging device. A step of controlling at least one of the illumination device and the imaging device based on the positional relationship, ambient light information and brightness information, An information processing method comprising the following: A recording medium on which a program to execute a program is stored.

[0205] (Note 18) An acquisition unit that acquires ambient light information representing factors that may affect the illuminance for the person being authenticated, A control unit that controls at least one of an illumination device that irradiates the subject with illumination light and an imaging device that images the subject, based on the ambient light information, An information processing device equipped with the following features.

[0206] (Note 19) The ambient light information includes the date and time of imaging. The information processing device described in Appendix 18.

[0207] (Note 20) The aforementioned ambient light information includes weather information. The information processing device described in Appendix 18 or 19.

[0208] (Note 21) The aforementioned ambient light information includes the optical properties, shape, size, and arrangement of structures surrounding the subject. An information processing device as described in any of Appendix 18 to 20.

[0209] (Note 22) A step of obtaining ambient light information that represents factors that may affect the illuminance for the person being authenticated, A step of controlling at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject, based on the ambient light information, An information processing method comprising the following:

[0210] (Note 23) A step of obtaining ambient light information that represents factors that may affect the illuminance for the person being authenticated, A step of controlling at least one of an illumination device that irradiates illumination light onto the subject and an imaging device that images the subject, based on the ambient light information, A recording medium on which a program to execute a program is stored. [Explanation of symbols]

[0211] 1. Authentication System 10. Authentication device 101... Processor 102...RAM 103···ROM 104...Storage 105...Communication I / F 106...Display 107A... Camera 1 107B...Second camera 108...Lighting equipment 108A...1st lighting device 108B...Second lighting device 109... Distance sensor 110...motor 111... Rotating mirror 20. Authentication Server 21. Authentication Engine 22. Ambient Light Information Database 23. Registered User Information Database 24. Layout Information Database 30...Gate device 100, 200... Information processing device 100A... Acquisition section 100B... Control Unit 200A...Acquisition part 200B... Control Unit

Claims

1. An acquisition unit that acquires the positional relationship between the person to be biometrically authenticated and the imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by the lighting device and the ambient light around the person, and brightness information of the person in the image captured by the imaging device, A determination unit that determines the state of the illumination light on the item worn by the subject based on the captured image, A control unit controls at least one of the illumination device and the imaging device based on the positional relationship, ambient light information, brightness information, and the determination result of the determination unit. An information processing device equipped with the following features.

2. The ambient light information is pre-associated with the positional relationship, The control unit updates the ambient light information based on the imaging conditions of the captured image and the result of the biometric authentication performed based on the captured image. The information processing apparatus according to claim 1.

3. The control unit controls at least one of the illumination device and the imaging device based on the imaging conditions of the captured image, the positional relationship at the time of imaging, and the result of the biometric authentication performed based on the captured image. The information processing apparatus according to claim 1.

4. The determination unit determines, based on the captured image, the state of light transmission through the item worn by the subject, The control unit controls at least one of the irradiation wavelength of the illumination light in the illumination device and the light reception wavelength range in the imaging device based on the determination result of the transmission state. The information processing apparatus according to claim 1 or 2.

5. The determination unit determines, based on the captured image, the reflection state of the illumination light on the item worn by the subject, The control unit controls at least one of the illumination device and the imaging device based on the reflection state. The information processing apparatus according to claim 1 or 2.

6. An acquisition unit that acquires the positional relationship between the person to be biometrically authenticated and the imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by the lighting device and the ambient light around the person, and brightness information of the person in the image captured by the imaging device, A determination unit that determines, based on the captured image, whether or not the subject is wearing an item and whether or not the shadow of the item is on the subject's body part, A storage unit that records the learning results obtained by learning the relationship between the presence or absence of the aforementioned items, the positional relationship, and the presence or absence of the aforementioned shadows, A control unit controls at least one of the illumination device and the imaging device based on the positional relationship, ambient light information, brightness information, and learning results. An information processing device equipped with the following features.

7. The steps include obtaining the positional relationship between the person to be biometrically authenticated and the imaging device that images the person, ambient light information including at least one of the illumination light irradiated onto the person by the lighting device and the ambient light around the person, and brightness information of the person in the image captured by the imaging device. A step of determining the state of the illumination light on the item worn by the subject based on the captured image, A step of controlling at least one of the illumination device and the imaging device based on the positional relationship, the ambient light information, the brightness information, and the determination result of the determination step, An information processing method comprising the following:

Images