Authentication system, authentication method, and authentication device

Abstract

To provide an authentication system capable of performing user authentication according to the imaging environment.SOLUTION: The authentication system includes: a first imaging unit for picking up an image of a person to be authenticated irradiated with visible light; a second imaging unit for picking up an image of a person to be authenticated who is irradiated with invisible light; and an authentication unit capable of communicating with the first imaging unit and the second imaging unit. The authentication unit performs authentication of the person to be authenticated based on either a visible image of the person to be authenticated captured by the first imaging unit or a non-visible image of the person to be authenticated captured by the second imaging unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an authentication system, an authentication method, and an authentication device. [Background technology]

[0002] Patent Document 1 discloses a device for face recognition of an object. The device includes an optical image sensor element sensitive to an infrared spectrum with a peak wavelength between 925 nm and 955 nm to capture at least one photograph of the object under an ambient light condition with additional infrared conditions, an infrared light source providing the additional infrared light, and a processing unit providing an image or image code for face recognition obtained from the photograph. The device subtracts a sunlight photograph from the captured infrared light photograph to obtain an image or image code for face recognition. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2021-501401 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the use of face recognition using photographs (facial images) of people's faces in various indoor and outdoor locations has been increasing. However, when a device captures a photograph in an environment where a person's face is irradiated with sunlight or other lighting, where the person's face is backlit, or where there is low illumination such as at night or on a rainy day, the face of the person in the photograph may appear overexposed or backlit, and the facial features of the person used for face recognition may not be captured.

[0005] The present disclosure has been devised in view of the above-mentioned conventional circumstances, and has an object to provide an authentication system, an authentication method, and an authentication device that perform user authentication suited to an imaging environment. [Means for solving the problem]

[0006] The present disclosure provides an authentication system comprising a first imaging device that images a person to be authenticated when illuminated with visible light, a second imaging device that images the person to be authenticated when illuminated with invisible light, and an authentication device capable of communicating with the first imaging device and the second imaging device, wherein the authentication device authenticates the person to be authenticated based on either a visible image of the person to be authenticated captured by the first imaging device or a non-visible image of the person to be authenticated captured by the second imaging device.

[0007] The present disclosure also provides an authentication method performed by an authentication system including a first imaging device that images a person to be authenticated irradiated with visible light, a second imaging device that images the person to be authenticated irradiated with invisible light, and an authentication device capable of communicating with the first imaging device and the second imaging device, wherein the person to be authenticated is authenticated based on either a visible image of the person to be authenticated captured by the first imaging device or a non-visible image of the person to be authenticated captured by the second imaging device.

[0008] The present disclosure also provides an authentication device including an acquisition unit that acquires a visible image of a person to be authenticated irradiated with visible light or an invisible image of the person to be authenticated irradiated with invisible light, and an authentication unit that authenticates the person to be authenticated based on either the acquired visible image or the invisible image and a registered visible image of at least one person to be authenticated that has been registered in advance. Effect of the Invention

[0009] According to the present disclosure, user authentication can be performed according to the imaging environment. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining use cases of the face authentication system according to the first and second embodiments. [Diagram 2] FIG. 1 is a block diagram illustrating an example of the internal configuration of a face authentication system according to a first embodiment and a second embodiment. [Diagram 3] FIG. 1 is a sequence diagram showing an example of an operation procedure of the face authentication system according to the first embodiment. [Figure 4] FIG. 1 is a sequence diagram showing an example of an operation procedure of the face authentication system according to the first embodiment. [Diagram 5] FIG. 11 is a sequence diagram showing an example of an operation procedure of a face authentication system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, with reference to the drawings as appropriate, an embodiment that specifically discloses the configuration and operation of the authentication system, authentication method, and authentication device according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0012] (Embodiment 1) Face authentication systems 100, 100A (an example of an authentication system) according to the first embodiment will be described with reference to Fig. 1 and Fig. 2, respectively. Fig. 1 is a diagram illustrating a use case of the face authentication system 100 according to the first embodiment and the second embodiment. Fig. 2 is a block diagram illustrating an example of the internal configuration of the face authentication systems 100, 100A according to the first embodiment and the second embodiment. The face authentication system 100A according to the second embodiment will be described later.

[0013] In Figure 1, the authentication device P1 and face authentication server S1 are omitted in order to make it easier to understand the explanation of the invisible light illumination emitted by the lighting 23 and the internal scattered light reflected from the inside of the user's face (skin), incident on the second camera 32, and received therein.

[0014] The face authentication system 100 includes a first imaging device C1, a lighting device L1, a second imaging device C2, an authentication device P1, and a face authentication server S1. Note that the face authentication system 100 shown in FIG. 1 is an example and is not limited thereto.

[0015] For example, the face authentication system 100 may include a network. The first imaging device C1 and the second imaging device C2 may be realized by one imaging device. The authentication device P1 and the face authentication server S1 may be connected to each other via a network so as to be able to transmit and receive data, or may be integrally configured by one device or server. In addition, the registered face image database DB may be configured separately from the face authentication server S1.

[0016] The first imaging device C1 is controlled by the authentication device P1 to capture an image of an area including the user's face and obtain a first captured image (visible image) of the user to be used for face authentication. The first imaging device C1 includes a first camera .

[0017] The first camera 10 is configured to have at least an image sensor (not shown) and a lens (not shown). The image sensor is a solid-state imaging element including a Si photodiode and, for example, a Charged-Coupled Device (hereinafter, referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter, referred to as "CMOS"), and converts an optical image formed on an imaging surface by visible light into an electrical signal. The first camera 10 transmits the converted electrical signal (data of the first captured image) to the authentication device P1.

[0018] The illumination device L1 is controlled by the second imaging device C2 to irradiate a user, whose face is to be authenticated, with invisible light including near-infrared light having a wavelength of 940 nm. The illumination device L1 includes a communication unit 20, a control unit 21, an illumination power supply unit 22, an illumination 23, and a first polarizing plate 24.

[0019] The communication unit 20 is connected to the second camera 32 of the second imaging device C2 so as to be able to perform data communication therebetween, and outputs a control command transmitted from the second camera 32 to the control unit .

[0020] Based on the control command output from the communication unit 20, the control unit 21 executes ON / OFF control of the lighting power supply unit 22 (that is, control of turning on / off the lighting 23).

[0021] The lighting power supply unit 22 is controlled by the control unit 21 to turn on and off the power supply to the lighting 23 .

[0022] The illumination 23 is realized by at least one Light-Emitting Diode (LED). The illumination 23 is supplied with power from the illumination power supply unit 22 and irradiates invisible light, which is near-infrared light with a wavelength of 940 nm. The illumination 23 may be a point light source or a surface light source.

[0023] The first polarizing plate 24 is disposed orthogonal to the second polarizing plate 30 (i.e., crossed Nicols), and polarizes the invisible light emitted from the illumination 23 in a first direction. For example, in the example shown in FIG. 1, the first polarizing plate 24 transmits only the vertical component of the invisible light of the illumination 23.

[0024] The second imaging device C2 is controlled by the authentication device P1 to capture an image of the user's face and obtain a second captured image (an invisible image, for example, an IR image) of the user to be used for face authentication. The second imaging device C2 includes a second polarizing plate 30, a band-pass filter (hereinafter, referred to as "BPF") 31, and a second camera 32.

[0025] The second polarizer 30 is arranged orthogonal to the first polarizer 24 (i.e., crossed Nicols) and removes the specularly reflected light and internally scattered light reflected by the user's face and the components of other light (sunlight, visible illumination light, etc.) in a first direction (i.e., the specularly reflected light components), and transmits only the components of the internally scattered light and other light in a second direction.

[0026] The BPF 31 transmits only invisible light (i.e., near-infrared light with a wavelength of 940 nm) among the internally scattered light and other light transmitted through the second polarizing plate 30. As a result, the BPF 31 can transmit only invisible light with a wavelength of 940 nm, which is within the sensor sensitivity range that the Si photodiode (not shown) of the second camera 32 can receive and is largely attenuated by moisture in the air.

[0027] The second camera 32 is configured to have at least an image sensor (not shown) and a lens (not shown). The image sensor is a solid-state imaging element including a Si photodiode and, for example, a CCD or CMOS, and converts an optical image formed on the imaging surface by non-visible light (internally scattered light) into an electrical signal. The second camera 32 transmits the converted electrical signal (data of the second captured image) to the authentication device P1.

[0028] The authentication device P1 is realized by, for example, a personal computer (hereinafter, referred to as “PC”), a notebook PC, a tablet terminal, a smartphone, etc., and performs authentication of a user who is a face authentication target. The authentication device P1 includes a communication unit 40, a control unit 41, a memory 42, and a display unit 43.

[0029] The communication unit 40 (an example of an acquisition unit) is connected to the first camera 10 of the first imaging device C1 and the communication unit 50 of the face authentication server S1 so as to be capable of wireless or wired communication between them, and transmits and receives data. The communication unit 40 acquires data of the first captured image transmitted from the first camera 10, and outputs it to the control unit 41.

[0030] The communication unit 40 transmits the facial image output from the control unit 41 to the face authentication server S1. In addition, the communication unit 40 acquires an authentication score corresponding to the facial image transmitted from the face authentication server S1 and outputs the authentication score to the control unit 41.

[0031] The control unit 41 (an example of an authentication unit) is configured using, for example, a Central Processing Unit (hereinafter, referred to as "CPU") or a Field Programmable Gate Array (hereinafter, referred to as "FPGA"), and performs various processes and controls in cooperation with the memory 42. Specifically, the control unit 41 refers to the programs and data held in the memory 42, and executes the programs to realize the functions of the authentication device P1.

[0032] The control unit 41 executes an image analysis process on the first captured image captured by the first camera 10 to detect the user's face appearing in the first captured image. The control unit 41 cuts out an area including the detected user's face from the first captured image to generate a facial image. The control unit 41 outputs the generated facial image to the communication unit 40 and causes it to be transmitted to the facial authentication server S1.

[0033] The control unit 41 may calculate the luminance value of a pixel included in an area including the detected user's face or the face image. In such a case, the control unit 41 executes a determination as to whether or not the calculated luminance value is a luminance value suitable for face authentication, and transmits the face image to the face authentication server S1 if the control unit 41 determines that the calculated luminance value is a luminance value suitable for face authentication. In addition, if the control unit 41 determines that the calculated luminance value is not a luminance value suitable for face authentication, the control unit 41 may cause the second imaging device C2 to capture a second captured image (invisible light) and proceed to face authentication processing using the second captured image.

[0034] The control unit 41 determines whether the user appearing in the facial image (first captured image) is a registered user who has been registered in advance in the registered facial image database DB, based on whether the authentication score transmitted from the facial authentication server S1 is greater than or equal to a first threshold value.

[0035] When the control unit 41 determines that the authentication score is equal to or greater than the first threshold, it determines that the user is a registered user who has been registered in advance in the registered face image database DB, and that the face authentication is "OK" (i.e., successful). The control unit 41 executes a predetermined authentication success operation that has been set in advance.

[0036] The authentication success operation here is an operation (control) that is set in advance by a user or an administrator of the face authentication system 100 based on the purpose of the face authentication system 100. The authentication success operation is, for example, an unlocking or opening operation of a door, an entrance gate, etc., a payment operation for purchasing an item, and an operation of generating and displaying an authentication result screen that notifies that the face authentication result is successful.

[0037] On the other hand, if the control unit 41 determines that the authentication score is not greater than or equal to the first threshold, it generates a control command to cause the second imaging device C2 to capture a second captured image (non-visible light) and causes the communication unit 40 to transmit the control command to the second imaging device C2.

[0038] The control unit 41 executes an image analysis process on the second captured image captured by the second camera 32 to detect the face of the user appearing in the second captured image and generate a facial image. The control unit 41 outputs the generated facial image to the communication unit 40 and causes it to be transmitted to the facial authentication server S1.

[0039] When the control unit 41 determines that the authentication score transmitted from the face authentication server S1 is equal to or greater than the second threshold, it determines that the user is a registered user who has been registered in advance in the registered face image database DB, and that the face authentication is "OK" (i.e., successful). The control unit 41 executes a predetermined authentication success operation that has been set in advance. Note that the second threshold here is set to a score lower than the first threshold.

[0040] When the control unit 41 determines that the authentication score transmitted from the face authentication server S1 is not equal to or greater than the second threshold, it repeatedly performs re-imaging of the second captured image and authentication based on the authentication score corresponding to the re-imaging of the second captured image. When the control unit 41 determines that authentication using the second captured image has occurred a predetermined number of times (e.g., three times, five times, etc.) or more, it determines that the user is not a registered user registered in the registered face image database DB in advance and that face authentication has been "NG" (i.e., failed). The control unit 41 generates an authentication failure screen (not shown) that notifies the user of the failure of face authentication, and outputs it to the display unit 43.

[0041] The memory 42 has, for example, a random access memory (hereinafter, referred to as "RAM") as a work memory used when executing each process of the control unit 41, and a read only memory (hereinafter, referred to as "ROM") for storing programs and data that define the operation of the control unit 41. The RAM temporarily stores data or information generated or acquired by the control unit 41. The ROM has written therein a program that defines the operation of the control unit 41.

[0042] The display unit 43 may be configured using a display device such as a Liquid Crystal Display (hereinafter, referred to as "LCD") or an organic electroluminescence (hereinafter, referred to as "EL"). The display unit 43 displays (outputs) an authentication failure screen (not shown) output from the control unit 41.

[0043] The face authentication server S1 calculates an authentication score indicating the likelihood that the user appearing in the first captured image or the second captured image transmitted from the authentication device P1 is any one of the users registered in advance in the registered face image database DB. The face authentication server S1 includes a communication unit 50, a control unit 51, a memory 52, and a registered face image database DB. The registered face image database DB may be configured separately and connected to the face authentication server S1 so as to be able to communicate with it.

[0044] The communication unit 50 is connected to the authentication device P1 so as to be able to communicate wirelessly or via wired communication with the authentication device P1, and transmits and receives data. The communication unit 50 acquires face image (first captured image or second captured image) data transmitted from the authentication device P1, and outputs it to the control unit 51. The communication unit 50 transmits information on the authentication score output from the control unit 51 to the authentication device P1.

[0045] The control unit 51 is configured using, for example, a CPU or an FPGA, and performs various processes and controls in cooperation with the memory 52. ​​Specifically, the control unit 51 refers to the programs and data stored in the memory 52 and executes the programs to realize the functions of the face authentication server S1.

[0046] The control unit 51 compares the acquired face image with the registered face images of multiple users registered in advance in the registered face image database DB, and calculates (evaluates) an authentication score that is an index of whether the user shown in the face image is the same person as the registered user. The control unit 51 outputs information on the calculated authentication score to the communication unit 50, and causes it to be transmitted to the authentication device P1.

[0047] The memory 52 has, for example, a RAM as a work memory used when executing each process of the control unit 51, and a ROM for storing programs and data that define the operation of the control unit 51. The RAM temporarily stores data or information generated or acquired by the control unit 51. The ROM has written therein a program that defines the operation of the control unit 51.

[0048] The registered face image database DB is a so-called storage, and is configured using a storage medium such as a flash memory, a hard disk drive (hereinafter, referred to as "HDD"), a solid state drive (hereinafter, referred to as "SSD"), etc. The registered face image database DB stores (registers) a registered face image for each user, which is a visible image, in association with information about the user (for example, the user's name, date of birth, age, sex, address, telephone number, ID, etc.).

[0049] Next, a face authentication operation performed by the face authentication system 100 according to the first embodiment will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a sequence diagram showing an example of an operation procedure of the face authentication system 100 according to the first embodiment. Fig. 4 is a sequence diagram showing an example of an operation procedure of the face authentication system 100 according to the first embodiment.

[0050] The process of step St16 is not essential and may be omitted. The process of step St18 may be executed by the authentication device P1. In this case, the processes of steps St17 and St19 are omitted.

[0051] Moreover, the processes of steps St18 and St33 may each be executed by the authentication device P1.

[0052] The first imaging device C1 starts exposure of the first camera 10 based on a control command instructing the start of imaging transmitted from the authentication device P1 (St11). The first imaging device C1 ends the exposure after a predetermined exposure time has elapsed (St12).

[0053] After the exposure (imaging) is completed, the first imaging device C1 converts the optical image formed on the imaging surface by visible light into an electrical signal to obtain the captured image of the user (St13). The first imaging device C1 transmits the converted electrical signal (data of the first captured image) to the authentication device P1 (St14). Here, the first captured image is a visible image.

[0054] The authentication device P1 performs image analysis processing on the first captured image transmitted from the first imaging device C1 to detect the user's face. The authentication device P1 determines whether the user's face is detected from the first captured image (St15).

[0055] When it is determined in the process of step St15 that the user's face is detected from the first captured image (St15, YES), the authentication device P1 calculates the luminance value of the pixels included in the area where the user's face is detected. Note that the calculated luminance value may be the average value or the total value of the luminance values ​​of the multiple pixels included in the area where the user's face is detected.

[0056] On the other hand, if the authentication device P1 determines in the processing of step St15 that the user's face is not detected from the first captured image (St15, NO), it determines that it is difficult to obtain a first captured image (visible image) suitable for face authentication, and proceeds to the processing of step St22 (i.e., face authentication processing using the second captured image).

[0057] The authentication device P1 judges whether the calculated luminance value is equal to or less than a first predetermined value and equal to or more than a second predetermined value, which indicate a luminance value suitable for face authentication (St16). Note that the first predetermined value is the upper limit of the luminance value at which an image suitable for face authentication can be captured. The second predetermined value is the lower limit of the luminance value at which an image suitable for face authentication can be captured.

[0058] If the authentication device P1 determines in the processing of step St16 that the calculated brightness value is equal to or less than the first predetermined value and equal to or greater than the second predetermined value (St16, YES), it generates a face image by cutting out the area containing the user's face from the first captured image (visible image) and transmits the face image to the face authentication server S1 (St17).

[0059] On the other hand, if the authentication device P1 determines in the processing of step St16 that the calculated brightness value is equal to or less than the first predetermined value and not equal to or greater than the second predetermined value (St16, NO), it determines that it is difficult to obtain a first captured image (visible image) suitable for face recognition, and proceeds to the processing of step St22 (i.e., face recognition processing using the second captured image).

[0060] The face authentication server S1 compares the face image (visible image) sent from the authentication device P1 with the registered face images of the multiple users registered in the registered face image database DB, and calculates an authentication score (St18). The face authentication server S1 transmits information on the calculated authentication score to the authentication device P1 (St19).

[0061] The authentication device P1 acquires the authentication score transmitted from the face authentication server S1, and determines whether the authentication score is equal to or greater than a first threshold (St20).

[0062] If, in the processing of step St20, the authentication device P1 determines that the authentication score transmitted from the face authentication server S1 is equal to or greater than the first threshold value (St20, YES), it determines that the user is a registered user who has been registered in advance in the registered face image database DB and that the face authentication is "OK" (i.e., successful), and executes a predetermined authentication success operation that has been set in advance (St21).

[0063] On the other hand, if the authentication device P1 determines in the processing of step St20 that the authentication score is not greater than or equal to the first threshold value (St20, NO), it determines that it is difficult to obtain a first captured image (visible image) suitable for face authentication, and proceeds to the processing of step St22 (i.e., face authentication processing using the second captured image).

[0064] The authentication device P1 generates a control command to cause the second imaging device C2 to capture an image of the user, and transmits the generated control command to the second imaging device C2 (St22).

[0065] Based on the control command transmitted from the authentication device P1, the second imaging device C2 generates a control command requesting turning on the light 23. The second imaging device C2 transmits the generated control command to the lighting device L1, requesting turning on the light 23 (St23).

[0066] Based on the control command transmitted from the second imaging device C2, the lighting device L1 starts supplying power to the lighting device 23 from the lighting power supply unit 22, and turns on the lighting device 23 (St23).

[0067] The second imaging device C2 starts exposure of the second camera 32 (St25). The second imaging device C2 ends the exposure after a predetermined exposure time has elapsed (St26).

[0068] After the exposure (imaging) is completed, the second imaging device C2 converts the optical image formed on the imaging surface by the invisible light into an electrical signal to obtain the captured image captured by the user (St27). The second imaging device C2 generates a control command requesting turning off the illumination 23. The second imaging device C2 transmits the generated control command to the illumination device L1 to request turning off the illumination 23 (St28).

[0069] Based on the control command transmitted from the second imaging device C2, the lighting device L1 stops the power supply to the lighting 23 from the lighting power supply unit 22, and turns off the lighting 23 (St29).

[0070] The second imaging device C2 transmits the converted electrical signal (data of the second captured image) to the authentication device P1 (St30). Here, the second captured image is an invisible image.

[0071] The authentication device P1 performs image analysis processing on the second captured image transmitted from the second imaging device C2 to detect the user's face. The authentication device P1 determines whether the user's face is detected from the second captured image (St31).

[0072] If the authentication device P1 determines in the processing of step St31 that the user's face has been detected from the second captured image (St31, YES), it generates a face image by cutting out the area containing the user's face from the second captured image (invisible image) and transmits the face image to the face authentication server S1 (St32).

[0073] On the other hand, when the authentication device P1 determines in the process of step St31 that the user's face is not detected from the second captured image (St31, NO), the authentication device P1 returns to the process of step St22, whereby the authentication device P1 can perform face authentication processing using the invisible image again.

[0074] The face authentication server S1 compares the face image (invisible image) sent from the authentication device P1 with the registered face images (visible images) of the multiple users registered in the registered face image database DB, and calculates an authentication score (St33). The face authentication server S1 transmits information on the calculated authentication score to the authentication device P1 (St34).

[0075] The authentication device P1 acquires the authentication score transmitted from the face authentication server S1, and determines whether the authentication score is equal to or greater than the second threshold (St35). Note that the second threshold may be a score different from the first threshold, and may be, for example, a score smaller than the first threshold.

[0076] If, in the processing of step St35, the authentication device P1 determines that the authentication score transmitted from the face authentication server S1 is equal to or greater than the first threshold value (St35, YES), it determines that the user is a registered user who has been registered in advance in the registered face image database DB and that the face authentication is "OK" (i.e., successful), and executes a predetermined authentication success operation that has been set in advance (St36).

[0077] On the other hand, when it is determined in the process of step St35 that the authentication score is not equal to or greater than the second threshold (St35, NO), the authentication device P1 determines whether the number of NG attempts is equal to or greater than a predetermined number (for example, 3 times, 5 times, etc.) (St37).

[0078] When the authentication device P1 determines in the process of step St37 that the number of NG attempts is equal to or greater than a predetermined number of attempts (St37, YES), the authentication device P1 determines that the face authentication is “NG” (that is, failed). The authentication device P1 generates an authentication failure screen (not shown) that notifies the user of the failure of the face authentication, and outputs (displays) it on the display unit 43 (St38).

[0079] On the other hand, when it is determined in the process of step St37 that the number of NG attempts is not equal to or greater than the predetermined number of attempts (St37, NO), the authentication device P1 returns to the process of step St22.

[0080] As described above, when the face of the user is not detected from the first captured image (visible image) or when it is determined that the face authentication using the first captured image has failed (i.e., the face authentication is "NG"), the face authentication system 100 according to the first embodiment can stop the acquisition process of the first captured image suitable for face authentication or the face authentication using the first captured image, and switch to acquiring the second captured image (invisible image) suitable for face authentication and performing face authentication using the second captured image. As a result, even when it is difficult to capture the first captured image (visible image) suitable for face authentication, the face authentication system 100 can more efficiently perform face authentication of the user by acquiring the second captured image (invisible image) suitable for face authentication and performing face authentication.

[0081] In addition, when the facial recognition system 100 determines that it is difficult to capture a first captured image (visible image) suitable for facial recognition based on the luminance values ​​of pixels included in the area in which the user's face is detected in the first captured image, it cancels the process of acquiring the first captured image suitable for facial recognition or facial recognition using the first captured image, and switches to acquiring a second captured image (invisible image) suitable for facial recognition and performing facial recognition using the second captured image, thereby enabling more efficient performance of user facial recognition.

[0082] Furthermore, the face authentication system 100 captures a first captured image (visible image) suitable for face authentication at different timings from a second captured image (invisible image) suitable for face authentication. This allows the face authentication system 100 to more effectively suppress the incidence of invisible light on the imaging plane of the first camera 10 when capturing (obtaining) the first captured image, and the incidence of visible light on the imaging plane of the second camera 32 when capturing (obtaining) the second captured image.

[0083] 3 and 4, the authentication device P1 transmits a face image based on the first captured image or the second captured image to the face authentication server S1, but is not limited thereto. For example, the authentication device P1 may extract features indicating the individuality of the user from the face image based on the first captured image or the second captured image, and transmit data of the extracted features to the face authentication server S1. In such a case, the face authentication server S1 calculates an authentication score based on the features transmitted from the authentication device P1.

[0084] In addition, in the face authentication system 100, when the first imaging device C1 includes a filter capable of blocking invisible light and the light incident on this filter is received by the imaging surface of the first camera 10, the first captured image and the second captured image may be captured at substantially the same time. In such a case, the process of steps St11 to St13 and the process of steps St22 to St27 are executed substantially in parallel. The face authentication system 100 may execute face authentication using the first captured image with priority out of the captured first captured image and the captured second captured image, or may execute face authentication using the first captured image and face authentication using the second captured image substantially in parallel.

[0085] (Embodiment 2) The face authentication system 100 according to the first embodiment has been described with respect to an example in which face authentication is performed using an invisible face image when it is determined that a face cannot be detected from a visible face image or that face authentication using a visible face image has failed. The face authentication system 100A according to the second embodiment will be described with respect to an example in which face authentication is performed using either a visible face image or an invisible face image based on the illuminance value or luminous intensity value measured by the sensor SS near the first imaging device C1.

[0086] The face authentication system 100A according to the second embodiment has substantially the same configuration as the face authentication system 100 according to the first embodiment. Therefore, in the following description of the face authentication system 100A according to the second embodiment, the same components as those in the face authentication system 100 according to the first embodiment are denoted by the same reference numerals and description thereof will be omitted.

[0087] A face authentication system 100A according to the second embodiment will be described with reference to FIG.

[0088] The face authentication system 100A includes a first imaging device C1, a lighting device L1, a second imaging device C2, an authentication device P1A, a face authentication server S1, and a sensor SS. Note that the face authentication system 100A shown in FIG. 2 is an example and is not limited thereto.

[0089] The authentication device P1A is connected to the first imaging device C1, the second imaging device C2, the lighting device L1, the face authentication server S1, and the sensor SS so as to be able to transmit and receive data between them. The authentication device P1A includes a communication unit 40A, a control unit 41A, a memory 42, and a display unit 43.

[0090] The communication unit 40A (an example of an acquisition unit) is connected to the first camera 10 of the first imaging device C1, the communication unit 50 of the face authentication server S1, and the sensor SS so as to be capable of wireless or wired communication, respectively, and executes transmission and reception of data. The communication unit 40A acquires data of the first captured image transmitted from the first camera 10 and outputs it to the control unit 41A, and outputs information on the illuminance value or luminous intensity value transmitted from the sensor SS to the control unit 41A.

[0091] The communication unit 40A transmits the face image output from the control unit 41A to the face authentication server S1. The communication unit 40A also obtains an authentication score corresponding to the face image transmitted from the face authentication server S1 and outputs it to the control unit 41A.

[0092] The control unit 41A (an example of an authentication unit) is configured using, for example, a CPU or an FPGA, and performs various processes and controls in cooperation with the memory 42. Specifically, the control unit 41A refers to the programs and data held in the memory 42 and executes the programs to realize the functions of the authentication device P1A.

[0093] The control unit 41A acquires information on the illuminance value or luminance value transmitted from the sensor SS. The control unit 41A determines whether to perform face authentication using a visible face image or an invisible face image, based on whether the acquired illuminance value or luminance value is equal to or less than a third predetermined value and equal to or greater than a fourth predetermined value. The third predetermined value here is the upper limit of the illuminance value or luminance value at which an image suitable for face authentication can be captured. The fourth predetermined value is the lower limit of the illuminance value or luminance value at which an image suitable for face authentication can be captured.

[0094] When performing face authentication using a visible face image, the control unit 41A generates a control command for making the first imaging device C1 capture a first captured image (visible image) and transmits the control command to the first imaging device C1. When performing face authentication using an invisible face image, the control unit 41A generates a control command for making the second imaging device C2 capture a second captured image (invisible image) and transmits the control command to the second imaging device C2.

[0095] The sensor SS is an illuminance meter capable of measuring illuminance or a photometer capable of measuring light intensity. The sensor SS is installed near the first imaging device C1 and measures the illuminance or light intensity near the face (head) of the user imaged by the first imaging device C1. The sensor SS is connected to the authentication device P1A so as to be able to transmit and receive data therebetween, and transmits the measured illuminance or light intensity value.

[0096] Next, a face authentication operation executed by the face authentication system 100A according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a sequence diagram showing an example of an operation procedure of the face authentication system 100A according to the second embodiment.

[0097] The sensor SS measures the illuminance or luminous intensity near the face (head) of the user imaged by the first imaging device C1 (St41). The sensor SS transmits information on the measured illuminance or luminous intensity value to the authentication device P1A (St42).

[0098] The authentication device P1A judges whether the illuminance value or the luminous intensity value transmitted from the sensor SS is equal to or less than a third predetermined value and equal to or greater than a fourth predetermined value (St43).

[0099] When the authentication device P1A determines in the process of step St43 that the illuminance value or luminance value is equal to or less than the third predetermined value and equal to or greater than the fourth predetermined value (St43, YES), it determines that the illuminance value or luminance value near the user's face is suitable for capturing a visible image. The authentication device P1A generates a control command to capture a first captured image (visible image) and transmits it to the first imaging device C1 (St10). The face authentication system 100A executes the processes of steps St11 to St14 (capturing process of the first captured image) described in FIG. 3, and the processes of step St15 and step St17 to step St21 (face authentication process) using the first captured image (visible image).

[0100] On the other hand, when the authentication device P1A determines in the process of step St43 that the illuminance value or luminance value is equal to or less than the third predetermined value and is not equal to or greater than the fourth predetermined value (St43, NO), it determines that the illuminance value or luminance value near the user's face is not suitable for capturing a visible image. The authentication device P1A generates a control command to capture a second captured image (invisible image) and transmits it to the second imaging device C2 (St22). The face authentication system 100A executes the processes of steps St23 to St30 (capturing process of the second captured image) described in FIG. 4 and the processes of steps St31 to St38 (face authentication process) using the second captured image (invisible image).

[0101] As described above, the face authentication system 100A according to the second embodiment can determine whether the current imaging environment is suitable for face authentication using either a visible image or a non-visible image, based on the illuminance value or luminous intensity value at the time of capturing the first captured image or the second captured image.

[0102] As a result, when the current imaging environment is not suitable for capturing a visible image, the facial recognition system 100A can omit the processing of steps St11 to St14 (capturing processing of the first captured image) and the processing of steps St15 and St17 to St21 (facial recognition processing), thereby more effectively reducing the time required for facial recognition per user.

[0103] As described above, the face authentication systems 100 and 100A (examples of authentication systems) according to the first and second embodiments include a first imaging device C1 that captures an image of a user (example of a person to be authenticated) illuminated with visible light, a second imaging device C2 that captures an image of the user illuminated with invisible light, and authentication devices P1 and P1A that can communicate with the first imaging device C1 and the second imaging device C2. The face authentication systems 100 and 100A authenticate a user based on either a visible image (first captured image) of the user captured by the first imaging device C1 or an invisible image (second captured image) of the user captured by the second imaging device.

[0104] As a result, the face authentication systems 100, 100A according to the first and second embodiments can perform face authentication of a user more efficiently by using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for face authentication.

[0105] Furthermore, the authentication devices P1, P1A in the face authentication systems 100, 100A according to the first and second embodiments execute authentication of a user using a non-visible image (second captured image) based on a result of authentication of the user using a visible image (first captured image). As a result, the face authentication systems 100, 100A according to the first and second embodiments can switch between acquiring a second captured image (non-visible image) suitable for face authentication and executing face authentication using the second captured image based on a result of face authentication using the first captured image. Therefore, the face authentication systems 100, 100A can more efficiently execute face authentication of a user using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for face authentication.

[0106] Furthermore, when the authentication device P1, P1A in the face authentication system 100, 100A according to the first and second embodiments determines not to authenticate the user based on the visible image (first captured image), it authenticates the user based on the invisible image (second captured image). As a result, when the face authentication system 100, 100A according to the first and second embodiments determines that the face authentication using the first captured image has failed (that is, the face authentication is "NG"), it can stop the acquisition process of the first captured image suitable for face authentication or the face authentication using the first captured image, and switch to acquiring the second captured image (invisible image) suitable for face authentication and performing face authentication using the second captured image. Therefore, even if it is difficult to capture the first captured image (visible image) suitable for face authentication, the face authentication system 100 can more efficiently perform face authentication of the user by acquiring the second captured image (invisible image) suitable for face authentication and performing face authentication.

[0107] Moreover, the authentication devices P1, P1A in the face authentication systems 100, 100A according to the first and second embodiments detect a biological part (e.g., a face) used for authenticating a user from a visible image (first captured image), and execute authentication of the user using either the visible image (first captured image) or the non-visible image (second captured image) based on the detection result of the biological part. As a result, the face authentication systems 100, 100A according to the first and second embodiments can execute authentication of the user more efficiently by determining an image more suitable for face authentication from the visible image (first captured image) or the non-visible image (second captured image) based on the presence or absence of detection of the user's face.

[0108] Furthermore, when the authentication devices P1, P1A in the face authentication systems 100, 100A according to the first and second embodiments determine that no biological part is detected from the visible image, they authenticate the user based on the invisible image (second captured image). As a result, the face authentication systems 100, 100A according to the first and second embodiments can authenticate the user by using the invisible image even when the user's face is difficult to detect because, for example, the user's face is illuminated by sunlight or illumination light, or the user's face is backlit, or has low illuminance at night or in rainy weather.

[0109] Moreover, the authentication device P1 in the face authentication system 100 according to the first embodiment detects a biological part (e.g., a face) used for authenticating a user from a visible image (first captured image), calculates a luminance value of the biological part, and performs authentication of the user using either the visible image (first captured image) or the non-visible image (second captured image) based on the luminance value. This allows the authentication device P1 in the face authentication system 100 according to the first embodiment to perform face authentication of a user more efficiently using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for face authentication.

[0110] Furthermore, when the authentication device P1 in the face authentication system 100 according to the first embodiment determines that the luminance value is within a first predetermined range (i.e., equal to or less than a first predetermined value and equal to or more than a second predetermined value), it authenticates the user based on the visible image (first captured image), and when it determines that the luminance value is not equal to or more than the first predetermined range, it authenticates the user based on the non-visible image (second captured image). This allows the authentication device P1 in the face authentication system 100 according to the first embodiment to more efficiently perform face authentication of the user by using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for face authentication.

[0111] Moreover, the face authentication system 100A according to the second embodiment further includes a sensor SS capable of measuring an imaging environment in which a user is imaged. The authentication device P1A performs authentication of the user using either a visible image (first captured image) or a non-visible image (second captured image) based on the imaging environment measured by the sensor SS. This allows the face authentication system 100A according to the second embodiment to perform face authentication of the user more efficiently using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for face authentication.

[0112] In addition, in the face authentication system 100A according to the second embodiment, the imaging environment is illuminance or luminance. When the authentication device P1A determines that the illuminance or luminance is within a second predetermined range (i.e., equal to or less than a third predetermined value and equal to or more than a fourth predetermined value), the authentication device P1A authenticates the user based on the visible image (first captured image), and when the authentication device P1A determines that the illuminance or luminance is not within the second predetermined range, the authentication device P1A authenticates the user based on the non-visible image (second captured image). This allows the face authentication system 100A according to the second embodiment to more efficiently perform face authentication of the user using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for face authentication.

[0113] Furthermore, when the authentication device P1A in the face authentication system 100A according to the second embodiment determines that the illuminance or luminance is within a second predetermined range (i.e., equal to or less than a third predetermined value and equal to or more than a fourth predetermined value), it causes the first imaging device C1 to capture an image of the user, and when it determines that the illuminance or luminance is not within the second predetermined range, it causes the second imaging device C2 to capture an image of the user. This allows the face authentication system 100A according to the second embodiment to determine which of the visible image (first captured image) and the non-visible image (second captured image) is more suitable for face authentication, and to make authentication per user more efficient.

[0114] Furthermore, the first imaging device C1 and the second imaging device C2 in the face authentication systems 100, 100A according to the first and second embodiments capture images of the user at different times. This allows the face authentication systems 100, 100A according to the first and second embodiments to more effectively suppress the incidence of invisible light on the imaging plane of the first camera 10 when capturing (obtaining) the first captured image, and the incidence of visible light on the imaging plane of the second camera 32 when capturing (obtaining) the second captured image.

[0115] Moreover, the authentication devices P1, P1A in the face authentication systems 100, 100A according to the first and second embodiments authenticate a user based on either a visible image or a non-visible image and a registered visible image of at least one user registered in advance. As a result, the face authentication systems 100, 100A according to the first and second embodiments can perform user authentication even if a non-visible image of the user has not been registered in advance. Therefore, when a visible image of the user has been captured and registered in advance by a general-purpose imaging device (e.g., a camera or a smartphone, a tablet terminal, etc. equipped with a camera), the face authentication systems 100, 100A according to the first and second embodiments can perform face authentication of the user more efficiently by acquiring a second captured image (non-visible image) suitable for face authentication and performing face authentication, even if it is difficult to capture a first captured image (visible image) suitable for face authentication.

[0116] As described above, the authentication devices P1, P1A according to the first and second embodiments include a communication unit 40, 40A (an example of an acquisition unit) that acquires a visible image (first captured image) of a user (an example of a person to be authenticated) illuminated with visible light, or an invisible image (second captured image) of a user illuminated with invisible light, and a control unit 41, 41A (an example of an authentication unit) that authenticates a user based on either the acquired visible image (first captured image) or invisible image (second captured image) and a registered visible image of at least one user that has been registered in advance.

[0117] As a result, the authentication devices P1, P1A according to the first and second embodiments can perform facial authentication of a user more efficiently by using either the visible image (first captured image) or the non-visible image (second captured image), whichever is more suitable for facial authentication.

[0118] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the various embodiments described above may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0119] The present disclosure is useful as providing an authentication system, an authentication method, and an authentication device that perform user authentication corresponding to an imaging environment. [Explanation of symbols]

[0120] 10 First Camera 20, 40, 40A, 50 Communication section 21, 41, 41A, 51 Control unit 23 Lighting 24 First polarizing plate 30 Second polarizing plate 31 BPF 32 Second Camera 42,52 Memory 43 Display section 100,100A Face Recognition System C1 First imaging device C2 Second imaging device DB Registered face image database L1 lighting system P1, P1A authentication device S1, S1A Face recognition server SS Sensor

Claims

1. a first imaging device that images a person to be authenticated irradiated with visible light; a second imaging device that images the person to be authenticated irradiated with invisible light; an authentication device capable of communicating with the first imaging device and the second imaging device, The authentication device includes: authenticating the person to be authenticated based on either a visible image of the person to be authenticated captured by the first imaging device or a non-visible image of the person to be authenticated captured by the second imaging device; Authentication system.

2. The authentication device includes: performing authentication of the person to be authenticated using the invisible image based on a result of authentication of the person to be authenticated using the visible image; The authentication system according to claim 1 .

3. The authentication device includes: when it is determined not to authenticate the person to be authenticated based on the visible image, authenticating the person to be authenticated based on the invisible image. The authentication system according to claim 2 .

4. The authentication device includes: detecting a body part used for authenticating the person to be authenticated from the visible image; performing authentication of the person to be authenticated using either the visible image or the invisible image based on a result of the detection of the body part; The authentication system according to claim 1 .

5. The authentication device includes: when it is determined that the biological part is not detected from the visible image, authenticating the person to be authenticated based on the invisible image. The authentication system according to claim 4.

6. The authentication device includes: detecting a body part used for authenticating the person to be authenticated from the visible image, and calculating a luminance value of the body part; performing authentication of the person to be authenticated using either the visible image or the invisible image based on the luminance value; The authentication system according to claim 1 .

7. The authentication device includes: when it is determined that the luminance value is within a first predetermined range, authenticating the person to be authenticated based on the visible image; when it is determined that the luminance value is not within the first predetermined range, authenticating the person to be authenticated based on the invisible image. The authentication system according to claim 6.

8. The authentication apparatus further includes a sensor capable of measuring an imaging environment in which the person to be authenticated is imaged, The authentication device includes: performing authentication of the person to be authenticated using either the visible image or the non-visible image based on the imaging environment measured by the sensor; The authentication system according to claim 1 .

9. The imaging environment is illuminance or luminous intensity, The authentication device includes: When it is determined that the illuminance or the luminous intensity is within a second predetermined range, authenticating the person to be authenticated based on the visible image; when it is determined that the illuminance or the luminous intensity is not within the second predetermined range, authenticating the person to be authenticated based on the invisible image. The authentication system according to claim 8.

10. The authentication device includes: When it is determined that the illuminance or the luminous intensity is within the second predetermined range, the first imaging device is caused to capture an image of the person to be authenticated, when it is determined that the illuminance or the luminous intensity is not within the second predetermined range, causing the second imaging device to capture an image of the person to be authenticated; The authentication system according to claim 9.

11. the first imaging device and the second imaging device image the person to be authenticated at different timings, The authentication system according to claim 1 .

12. The authentication device includes: authenticating the person to be authenticated based on either the visible image or the invisible image and a registered visible image of at least one person to be authenticated that has been registered in advance; The authentication system according to claim 1 .

13. a first imaging device that images a person to be authenticated irradiated with visible light; a second imaging device that images the person to be authenticated irradiated with invisible light; An authentication method performed by an authentication system including an authentication device capable of communicating with the first imaging device and the second imaging device, authenticating the person to be authenticated based on either a visible image of the person to be authenticated captured by the first imaging device or a non-visible image of the person to be authenticated captured by the second imaging device; Authentication method.

14. an acquisition unit that acquires a visible image of the person to be authenticated irradiated with visible light or an invisible image of the person to be authenticated irradiated with invisible light; an authentication unit that authenticates the person to be authenticated based on either the acquired visible image or the acquired invisible image and a registered visible image of at least one person to be authenticated that has been registered in advance; Authentication device.

Images