Face authentication gate system, control device, control method, and program

Abstract

To enable improvement in the accuracy of authentication regardless of surrounding environments.SOLUTION: A face authentication gate system includes a gate device, a control device, and a face authentication server, and determines whether a user is permitted to pass a gate device or not on the basis of a result of face collation processing between registered face information registered in advance and captured face information based on a captured image. The face authentication server transmits a face collation score, which is a score for face collation between registered face information and captured face information based on an image of a user who is trying to pass the gate device, to the control device. The control device controls a parameter related to imaging on the basis of illuminance of a captured image when the face collation score is at an illuminance evaluation threshold or more and is less than an evaluation upper limit value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a face recognition gate system, a control device, a control method, and a program. [Background technology]

[0002] In stations, offices, and the like, there are areas where entry is restricted to people other than certain individuals. Gates are installed at the entrances and exits of such areas, and the entry of people who pass through the gates to enter the area and the exit of people who pass through the gates to leave the area are managed. For example, ticket gates are installed at the entrances and exits of stations, and the ticket gates manage the entry of people into the station premises based on whether or not they have a ticket.

[0003] In recent years, facial recognition ticket gates that use facial recognition technology to determine whether or not a person can pass through the gate have been under consideration.Facial recognition ticket gates perform facial recognition using images of the person to be authenticated (for example, a person using a station), and there are studies underway to control the light that is irradiated onto the person to be authenticated when the image is taken.

[0004] Patent document 1 describes a face authentication device that captures the face of a person to be authenticated as an input image and performs identity authentication by comparing the input image with a registered image, and that includes an ambient light illuminance detection means that detects the illuminance of ambient light, an illumination means that irradiates light onto the face of the person to be authenticated, and an illumination control means that controls the light irradiated by the illumination means to be weaker when the illuminance of ambient light detected by the ambient light illuminance detection means is low, and controls the light irradiated by the illumination means to be stronger when the illuminance of ambient light is high. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2007 / 135735 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the technology described in Patent Document 1, depending on the surrounding environment (e.g., the intensity and direction of external light), the image captured of the person to be authenticated may not be suitable for facial recognition, which may result in a deterioration in authentication accuracy.

[0007] Non-limiting examples of the present disclosure contribute to providing a face recognition gate system, a control device, a control method, and a program that can improve the accuracy of authentication regardless of the surrounding environment. [Means for solving the problem]

[0008] A facial recognition gate system according to one embodiment of the present disclosure includes a gate device, a control device, and a facial recognition server, and determines whether or not a user can pass through the gate device based on the results of a facial matching process between pre-registered facial information and photographed facial information based on a captured image.The facial recognition server transmits a facial matching score, which is a score for the facial matching between the registered facial information and photographed facial information based on an image captured of a user attempting to pass through the gate device, to the control device, and the control device controls parameters related to the shooting based on the illuminance of the captured image when the facial matching score is greater than or equal to an illuminance evaluation threshold and less than an upper evaluation limit value.

[0009] A control device according to one embodiment of the present disclosure includes a communication unit that acquires a face matching score, which is a score for matching photographed face information based on an image of a user attempting to pass through a gate device with pre-registered registered face information, and a control unit that controls parameters related to the photographing based on the illuminance of the image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an upper evaluation limit value.

[0010] In a control method according to one embodiment of the present disclosure, a control device acquires a face matching score, which is a score obtained by matching photographed face information based on an image of a user attempting to pass through a gate device with pre-registered registered face information, and controls parameters related to the photographing based on the illuminance of the image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an upper evaluation limit value.

[0011] A program according to one embodiment of the present disclosure causes a control device to execute a process of acquiring a face matching score, which is a score for face matching between captured face information based on an image of a user attempting to pass through a gate device and pre-registered registered face information, and controlling parameters related to the photographing based on the illuminance of the image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an upper evaluation limit value.

[0012] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0013] Non-limiting examples of the present disclosure can improve authentication accuracy regardless of the surrounding environment.

[0014] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing a first example of a face authentication ticket gate system 1 according to an embodiment. [Figure 2] FIG. 1 is a diagram showing a second example of the face authentication ticket gate system 1 according to an embodiment. [Figure 3]1 is a flowchart showing a first example of the flow of processing for accumulating execution results of face authentication processing according to an embodiment. [Figure 4] 1 is a flowchart showing an example of a flow of control of a lighting device according to an embodiment. [Figure 5] Diagram showing variations in facial illuminance measurements [Figure 6] A diagram showing examples of facial image quality check items [Figure 7] FIG. 10 is a diagram showing a first example of parameter control of a lighting device. [Figure 8] FIG. 10 is a diagram showing a second example of parameter control of a lighting device. [Figure 9] A diagram showing an example of the relative positions of users and the zones of facial recognition ticket gates [Figure 10] 10 is a flowchart showing a second example of the processing flow for accumulating the execution results of face authentication processing in the embodiment. [Figure 11] 1 is a flowchart showing an example of a control flow of a camera according to an embodiment. [Figure 12] FIG. 1 shows a first example of camera parameter control. [Figure 13] FIG. 2 shows a second example of camera parameter control. [Figure 14A] Flowchart for controlling lighting device parameters for each zone [Figure 14B] 1 is a flowchart showing an example of determining parameters of lighting devices for each zone. [Figure 15] FIG. 1 shows an example of determining parameters of lighting devices for each zone. [Figure 16] A diagram showing an example of a change in face frame size [Figure 17] FIG. 10 is a diagram showing an example of the correspondence between changes in the face frame size of a user and changes in face illuminance. [Figure 18] FIG. 1 is a diagram showing an overview of control of a lighting device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0017] (One embodiment) <Knowledge that led to this disclosure> Ticket gates are installed at the entrances and exits of railway stations to manage the entry and exit of people using the station. If the ticket gate determines that a person attempting to enter or exit the station through the ticket gate is authorized to pass through the ticket gate, the ticket gate controls the person to pass through. Also, if the ticket gate determines that a person attempting to enter or exit the station through the ticket gate is unauthorized to pass through, the ticket gate controls the person not to pass through.

[0018] In a ticket gate, control that allows a person to pass includes, for example, at least one of door control that is provided in the passage of the ticket gate and that opens doors that prevent the person from passing through, display control that displays a message indicating that passage is permitted, and audio control that outputs a sound indicating that passage is permitted. Control that does not allow a person to pass includes, for example, at least one of door control that is provided in the passage of the ticket gate and that closes doors that prevent the person from passing through, display control that displays a message indicating that passage is not permitted, and audio control that outputs a sound indicating that passage is not permitted.

[0019] In recent years, facial recognition ticket gates that use facial recognition technology to determine whether or not a person can pass through the ticket gate have been considered. In the following, a person who is about to pass through a facial recognition ticket gate may be referred to as a user of the facial recognition ticket gate or simply as a user.

[0020] The facial recognition process in a facial recognition ticket gate involves taking a picture of the user's face, detecting a facial image (a region containing the face cut out from the captured image) that includes the user's face from within the captured image (face detection), and then performing facial recognition based on the detected facial image to determine whether the user is a person permitted to pass through the facial recognition ticket gate.

[0021] Capturing an image including the user's face, detecting the face from within the captured image, and performing facial recognition based on the detected facial image are performed by a facial recognition ticket gate control device and / or a facial recognition server that controls the facial recognition ticket gate in the facial recognition ticket gate system.

[0022] For example, depending on the installation location of a facial recognition ticket gate, there are times when external light (sunlight) and / or illumination light enters. As such, since facial recognition ticket gates are installed at ticket gates in train stations, the illuminance of the light irradiated on the person to be authenticated (e.g., a user using the station), that is, the level of illuminance in an image containing a facial image, changes depending on the surrounding environment (e.g., the strength and direction of external light such as sunlight), which may result in a deterioration in the accuracy of authentication.

[0023] For example, it is expected that illuminance will be higher during the daytime than at night due to the incidence of sunlight. Furthermore, it is expected that illuminance will be higher on sunny days with plenty of sunlight than on cloudy days. Furthermore, depending on the direction of sunlight incidence, it is expected that overexposure will occur only in certain directions. For example, if a facial recognition ticket gate is configured to allow entry and exit in both directions, it is expected that the illuminance due to sunlight incident on a user entering the facial recognition ticket gate from one side will be higher than that from the other side. In this way, it is expected that the illuminance in an image containing a facial image will change depending on the time of day and / or the direction of the facial recognition ticket gate. For example, if the illuminance changes depending on the time of day, it is desirable to control the lighting according to the time of day. Furthermore, if the illuminance changes depending on the direction of the facial recognition ticket gate, it is desirable to control the lighting according to the direction.

[0024] Facial recognition requires exposure that produces clear images, including facial images. Therefore, facial recognition ticket gates must be able to control the exposure (e.g., illuminance) to be suitable for facial recognition under any conditions of the surrounding environment, including the influence of external light, etc.

[0025] For example, it is necessary to be able to capture the surrounding image, including the facial image, in a wide dynamic range, rather than focusing on the brightness or darkness of the entire captured image that shows the user and the background behind the user.

[0026] Therefore, in this embodiment, a face recognition ticket gate system that controls parameters related to image capture so that an image suitable for face recognition is captured will be described. For example, in this embodiment, the brightness (hereinafter referred to as illuminance) of the face frame area (peripheral image including the face image) obtained by the face detection function and the face recognition score obtained by face recognition are evaluated in image capture, thereby determining the situation of incident external light and controlling parameters related to image capture, including the brightness of the lighting.

[0027] Below, a configuration example and an operation example of a face authentication ticket gate system including a face authentication ticket gate, a face authentication control device, and a face authentication server will be described.

[0028] In the following description, the terms "face authentication" and "face matching" may be used interchangeably. For example, the terms "face authentication score" and "face matching score" may be used interchangeably.

[0029] <System configuration example> 1 is a diagram showing a first example of a facial recognition ticket gate system 1 according to the present embodiment. The facial recognition ticket gate system 1 shown in FIG. 1 includes a facial recognition control device 11, a facial recognition ticket gate 12, a lighting device 13, a camera 14, and a facial recognition server 15.

[0030] The facial recognition ticket gate system 1 shown in FIG. 1 identifies the person attempting to pass through the facial recognition ticket gate 12 and determines whether the identified person is permitted to pass through the facial recognition ticket gate 12.

[0031] A person who wishes to use the facial recognition ticket gate system 1 shown in Fig. 1 registers information in advance. For example, a person who wishes to use the facial recognition ticket gate system 1 registers an image including their own face (hereinafter referred to as a facial image) and personal information (e.g., name, etc.) in the facial recognition server 15. Hereinafter, a person who registers information in the management server 15 will be referred to as a registrant. Note that personal information does not have to be registered in the facial recognition server 15.

[0032] Information about registrants is stored in the face authentication server 15. The information about registrants includes facial images and personal information about the registrants, as well as information about whether the registrants are permitted to pass through the face authentication ticket gate 12.

[0033] Note that information regarding whether or not a registrant is permitted to pass through the facial recognition ticket gate 12 does not have to be stored in the facial recognition server 15. For example, the determination of whether or not a registrant is permitted to pass through the facial recognition ticket gate 12 may be referred to as ticket gate determination. The ticket gate determination may be performed by a device (e.g., a ticket gate determination device) different from the facial recognition server 15. In this case, information regarding whether or not a registrant is permitted to pass through the facial recognition ticket gate 12 may be stored in the ticket gate determination device. When the facial recognition server 15 identifies a registrant corresponding to the user who is the subject of face matching in face matching, it requests the ticket gate determination device to determine whether or not the identified registrant is permitted to pass through the facial recognition ticket gate 12. Based on this request and the stored information, the ticket gate determination device transmits the determination result, regarding whether or not the identified registrant is permitted to pass through the facial recognition ticket gate 12, to the facial recognition server 15. The facial recognition server 15 transmits the judgment result obtained from the ticket gate judgment device to the facial recognition control device 11, and the facial recognition control device 11 may control whether to allow the person to pass through the facial recognition ticket gate 12 or not to allow the person to pass through the facial recognition ticket gate 12 based on the judgment result.

[0034] For ease of explanation, the following description will be given assuming that each of the registrants registered in the face authentication server 15 is a person who is permitted to pass through the face authentication ticket gate 12. In other words, if the face authentication process can identify the person who is about to pass through the face authentication ticket gate 12, the person is determined to be a person who is permitted to pass through the face authentication ticket gate 12.

[0035] In the face recognition ticket gate system 1, by referring to the information registered in the face recognition server 15, if the user is included in the registered persons, it is identified which of the registered persons the user is.

[0036] The facial recognition ticket gate 12 is installed, for example, at an entrance / exit of a station, and allows people who are permitted to pass through the facial recognition ticket gate 12 to pass, and blocks people who are not permitted to pass through the facial recognition ticket gate 12. The facial recognition ticket gate 12 has at least a lighting control unit 121.

[0037] 1 is installed on the ceiling above the face authentication ticket gate 12 and emits light downward from the ceiling. For example, the lighting device 13 is connected to a lighting control unit 121, and the emitted light is controlled by the lighting control unit 121.

[0038] The light emitted from the lighting device 13 may be controlled in both the entrance direction and the exit direction of the face recognition ticket gate 12.

[0039] The camera 14 is installed in the facial recognition ticket gate 12. The camera 14 is provided at a position where it can photograph a user approaching the facial recognition ticket gate 12 from both the entrance and exit directions of the facial recognition ticket gate 12, and also at a position where it can photograph the user from both the right and left sides of the user. The camera 14 outputs the photographed images to the facial recognition control device 11.

[0040] The face authentication control device 11 controls the face authentication ticket gate 12 based on face authentication processing. The face authentication control device 11 is connected to a camera 14 and a face authentication server 15. The face authentication control device 11 may be mounted on the face authentication ticket gate 12. Alternatively, the face authentication control device 11 may be configured as a device separate from the face authentication ticket gate 12 and connected to the face authentication ticket gate 12 by wire or wirelessly.

[0041] The face authentication control device 11 has a face matching request processing unit 111, a ticket gate control unit 112, and a communication unit 113. The face matching request processing unit 111 and the ticket gate control unit 112 may be collectively referred to as a processing unit or a processor.

[0042] The face matching request processing unit 111 controls the camera 14 to capture an image including the user's face. The face matching request processing unit 111 detects the user's face image from the captured image and transmits a face matching request to the face authentication server 15 based on the detected face image. For example, the face matching request includes the detected face image of the user (or feature amounts extracted from the user's face image by the face matching request processing unit 111), identification information for identifying the face authentication ticket gate 12 (for example, a face authentication ticket gate ID), and entry / exit direction information. The entry / exit direction information may be information indicating from which direction the user is attempting to enter the face authentication ticket gate 12. Note that, hereinafter, the face image or feature amounts extracted from the face image will be referred to as face information. The face matching request includes the user's face information.

[0043] The ticket gate control unit 112 controls the face authentication ticket gate 12 based on the face matching response transmitted from the face authentication server 15. For example, the ticket gate control unit 112 controls the opening and closing of a door (omitted in FIG. 1) that is provided in a passage of the face authentication ticket gate 12 and blocks passage through the passage. For example, if the user is a person permitted to pass through the face authentication ticket gate 12, the ticket gate control unit 112 controls the door to be in an open state, and if the user is not a person permitted to pass through the face authentication ticket gate 12, the ticket gate control unit 112 controls the door to be in a closed state.

[0044] For example, the ticket gate control unit 112 controls the display unit, the audio output unit, etc. so as to provide information to a user of the facial recognition ticket gate 12 via a display unit, an audio output unit, etc., which are omitted in Fig. 1. For example, information indicating whether the user is a person permitted to pass through the facial recognition ticket gate 12 may be provided to the user via the display unit, the audio output unit, etc.

[0045] The communication unit 113 is an interface that transmits signals containing information to other devices (for example, the face authentication server 15) included in the face authentication ticket gate system 1. The communication unit 113 is also an interface that receives signals containing information provided from other devices included in the face authentication ticket gate system 1. The communication unit 113 may be divided into a transmission unit that performs signal transmission processing and a reception unit that performs signal reception processing.

[0046] The face authentication control device 11 repeats image capture, face detection, and sending of face matching requests while the face matching response received from the face authentication server 15 is "face authentication NG" (for example, the face matching score obtained as a result of face matching is less than the authentication threshold). For example, if the number of times that the face authentication control device 11 receives a face matching result indicating "face authentication NG" exceeds a specified number (for example, an integer greater than or equal to 1) or if the number of times that image capture, face detection, and sending of face matching requests is repeated exceeds a specified number, the face authentication control device 11 may stop sending of face matching requests and treat the user as having "face authentication NG."

[0047] The face authentication server 15 is connected to one or more face authentication control devices 11, and performs face authentication processing based on a face authentication request received from the face authentication control device 11. The face authentication server 15 includes a face authentication processing unit 151, a communication unit 152, and an illuminance measurement unit 153. The face authentication processing unit 151 and the illuminance measurement unit 153 may be collectively referred to as a processing unit or a processor.

[0048] The face matching processing unit 151 performs face matching based on a face matching request received via the communication unit 152 .

[0049] In face matching, the face matching processing unit 151 identifies a registrant corresponding to the user by comparing the face image of the user included in the face matching request with face images of registrants registered in the face authentication server 15. For example, the face matching processing unit 151 calculates a face matching score indicating the degree of similarity between the face image of the user and the face image of the registrant. The face matching score may be calculated between the face image of the user and the face image of the registrant, or may be calculated between feature amounts of the face image of the user and the feature amounts of the face image of the registrant. Then, in face matching, a face matching score between the face images of the registrant and the face image of the registrant is calculated for each registrant. In other words, each face matching score is associated with a registrant.

[0050] In face matching, if the largest face matching score among the calculated face matching scores is equal to or greater than the authentication threshold, face matching processing unit 151 determines that the registrant corresponding to the largest face matching score corresponds to the user. If the face matching score is equal to or greater than the authentication threshold, face matching processing unit 151 determines that the registrant corresponding to the face matching score equal to or greater than the authentication threshold corresponds to the user of the face image included in the face matching request.

[0051] The face matching processing unit 151 transmits to the face authentication control device 11 via the communication unit 152 a face matching response including information about the registrant corresponding to the user of the face image included in the face matching request and the face matching score calculated for the registrant.

[0052] The illuminance measurement unit 153 measures the illuminance of the face image that is the target of face matching. For example, the illuminance measurement unit 153 measures the facial illuminance of the face image in the illuminance measurement. When the largest face matching score among the face matching scores calculated for the face images of a certain user is within a predetermined range, the illuminance measurement unit 153 stores the facial illuminance of the face image. For example, the facial illuminance of the face image is stored in association with the time when the face image was captured. Note that variations in the measurement of facial illuminance will be described later.

[0053] The communication unit 152 is an interface that transmits signals containing information to other devices (for example, the face recognition control device 11) included in the face recognition ticket gate system 1. The communication unit 152 is also an interface that receives signals containing information provided from other devices included in the face recognition ticket gate system 1. The communication unit 152 may be divided into a transmission unit that performs signal transmission processing and a reception unit that performs signal reception processing.

[0054] The ticket gate control unit 112 of the face authentication control device 11 controls the lighting device 13 via the lighting control unit 121 based on information (e.g., facial illuminance) stored in the face authentication server 15. For example, the ticket gate control unit 112 retrieves information stored in the face authentication server 15 and calculates a value indicating the statistical properties of the facial illuminance (e.g., the average value of the facial illuminance within a specified time period). The ticket gate control unit 112 determines parameters to be used for controlling the lighting device 13 based on the comparison result between the value indicating the statistical properties of the facial illuminance and predetermined conditions (e.g., one or more thresholds), and instructs the lighting control unit 121 of the determined parameters. The lighting control unit 121 controls the lighting device 13 based on the instruction. The control flow of the lighting device 13 will be described later.

[0055] 1 shows an example in which the lighting device 13 is installed on the ceiling, but the present disclosure is not limited to this. Another installation example of the lighting device 13 is shown in FIG.

[0056] Fig. 2 is a diagram showing a second example of the face authentication ticket gate system 1 according to the present embodiment. In Fig. 2, the same components as those in Fig. 1 are given the same numbers, and their explanation may be omitted.

[0057] 1 and 2 is that the lighting device 13 is provided on the ceiling in FIG. 1, but the lighting device 13 is provided in the face authentication ticket gate 12 in FIG.

[0058] 2 is provided in the face authentication ticket gate 12, and irradiates light toward a user moving toward the face authentication ticket gate 12. For example, the illumination device 13 is connected to an illumination control unit 121, and the irradiated light is controlled by the illumination control unit 121.

[0059] <Overall flow> In the face recognition ticket gate system 1 of this embodiment, a process of accumulating the execution results of face recognition processing and a process of controlling the lighting device based on the execution results of face recognition processing that have been accumulated are carried out.

[0060] In storing the results of the face recognition process, the face recognition server 15 stores the results of the face recognition process (for example, face matching score) that is executed when a user attempts to pass through the face recognition ticket gate 12, the face illuminance measured from the captured face image used in the face recognition process, and the time when the face recognition process was executed. The stored information is referred to as face illuminance information.

[0061] A limit may be placed on the information to be accumulated. For example, cases in which the face matching score is equal to or greater than the authentication threshold may be excluded. A case in which the face matching score is equal to or greater than the authentication threshold may correspond to a case in which face authentication is successful. Furthermore, cases in which the face matching score is less than a predetermined threshold may be excluded. Cases in which the face matching score is less than the predetermined threshold may include cases in which, in the face authentication process, the face information of a person to be subjected to face authentication is matched with the face information of a registered person different from the person.

[0062] In controlling the lighting device, the face authentication control device 11 controls lighting by referring to face illuminance information. For example, it periodically (for example, every minute) refers to face illuminance information within a specified period of time, evaluates the face illuminance, and controls lighting according to the evaluation. For example, if the face illuminance evaluation determines that the face illuminance is insufficient, it increases the brightness of the lighting, and if the face illuminance evaluation determines that the face illuminance is excessive, it decreases the brightness of the lighting.

[0063] <Process flow for accumulating the results of face recognition processing> Next, an example of the flow of processing for accumulating the execution results of the face recognition processing executed in the face recognition ticket gate system 1 will be described.

[0064] 3 is a flowchart showing a first example of the processing flow for accumulating the execution results of face authentication processing in this embodiment. The flowchart shown in FIG. 3 starts every time camera 14 captures an image at a fixed cycle.

[0065] The camera 14 captures an image (S101). The image captured by the camera 14 is output to the face authentication control device 11.

[0066] The face authentication control device 11 performs face detection processing on the captured image (S102). In the face detection processing, a frame (face image detection frame, or simply face frame) surrounding an image corresponding to a human face (hereinafter referred to as a face image) is detected.

[0067] The face authentication control device 11 determines whether or not a face has been detected from the captured image (S103).

[0068] If a face is not detected (NO in S103), the flow returns to S101.

[0069] If a face is detected (YES in S103), the face authentication server 15 performs face matching processing and illuminance measurement based on a face image including the detected face of the user (S104). For example, the face authentication control device 11 transmits a face matching request including the detected face information and face image to the face authentication server 15, requesting face matching processing. The face authentication server 15 performs face matching processing and illuminance measurement of the face image based on the face matching request. Then, the face authentication control device 11 receives a face matching response to the face matching request from the face authentication server 15. The face matching response includes a face matching score calculated for the detected face information. In the illuminance measurement, the facial illuminance of the face image is measured. For example, the average brightness value of the entire face image or at least a portion of the pixel of the face image corresponds to the facial illuminance. Variations in measuring facial illuminance will be described later.

[0070] The face authentication server 15 determines whether the face matching score is within a predetermined range (S105). For example, the predetermined range is defined by an upper evaluation limit and an illuminance evaluation threshold. The face authentication control device 11 determines whether the face matching score is equal to or greater than the illuminance evaluation threshold and less than the upper evaluation limit. The upper evaluation limit may be the same as the authentication threshold or may be different from the authentication threshold. Furthermore, the upper evaluation limit need not be set. In this case, the predetermined range may be a range equal to or greater than the illuminance evaluation threshold.

[0071] If the face matching score is not within the predetermined range (NO in S105), for example, if the face matching score is less than the illuminance evaluation threshold value or equal to or greater than the evaluation upper limit value, the flow proceeds to S201.

[0072] If the face matching score is within a predetermined range (YES in S105), the face recognition server 15 stores the time when the face image was captured and the measured face illuminance (S106). Note that the face matching score may be stored in addition to the time and face illuminance. Then, the flow proceeds to S201.

[0073] The face authentication control device 11 determines whether the face matching score is equal to or greater than the authentication threshold (S201).

[0074] If the face matching score is not equal to or greater than the authentication threshold (NO in S201), the flow returns to S101.

[0075] If the face matching score is equal to or greater than the authentication threshold (YES in S201), the face authentication control device 11 determines that face authentication of the user corresponding to the face image is successful (S202). In other words, if the face matching score is equal to or greater than the authentication threshold (YES in S201), the face authentication control device 11 determines that the user corresponding to the face image is a person permitted to pass through the face authentication ticket gate 12. Then, the face authentication control device 11 performs control to allow the user corresponding to the face image to pass through the face authentication ticket gate 12. Then, the flow ends.

[0076] 3 shows an example in which the illuminance measurement is performed by the face authentication server 15, but the present disclosure is not limited to this. For example, the illuminance measurement may be performed by the face authentication control device 11. By having the face authentication control device 11 perform the illuminance measurement, it is possible to reduce the load on the face authentication server 15 that has received the face matching request.

[0077] While FIG. 3 shows an example in which the illuminance of each face image that is the target of face matching processing is measured, the present disclosure is not limited to this. Illuminance measurement may be omitted for some of the face images that are the target of face matching processing. For example, a quality check of the face images may be performed, and illuminance measurement may be performed for face images with high quality (for example, face images with quality equal to or higher than a predetermined level), while illuminance measurement may not be performed for face images with low quality (for example, face images with quality below a predetermined level). Furthermore, the illuminance of face images that are not the target of face matching processing may be measured. The quality check of face images will be described later.

[0078] As shown in FIG. 3, the illuminance of a face image is measured when the face matching score is within a predetermined range. For example, the illuminance of a face image is measured when the face matching score is equal to or greater than the illuminance evaluation threshold and less than the evaluation upper limit. By limiting the illuminance to be evaluated for controlling the lighting device based on the face matching score in this manner, appropriate illuminance evaluation can be performed. Also, as shown in FIG. 3, the illuminance of a face image is accumulated when the face matching score is equal to or greater than the illuminance evaluation threshold and less than the evaluation upper limit, and the illuminance of a face image is not accumulated when the face matching score is equal to or greater than the evaluation upper limit. Because the illuminance evaluation threshold is greater than the evaluation upper limit, the condition that the face matching score is equal to or greater than the evaluation upper limit (e.g., the "second condition") is a stronger condition than the condition that the face matching score is equal to or greater than the illuminance evaluation threshold (e.g., the "first condition"). That is, in the example shown in FIG. 3, the illuminance of a face image is accumulated when the face matching score satisfies the first condition that the face matching score is equal to or greater than the illuminance evaluation threshold, but does not satisfy the second condition that the face matching score is equal to or greater than the evaluation upper limit. 3, if the first condition that the face matching score is equal to or greater than the illuminance evaluation threshold is met, and the second condition that the face matching score is equal to or greater than the evaluation upper limit is met, the illuminance of the face image is not accumulated. In this way, by limiting the illuminance to be accumulated as an evaluation target for controlling the lighting device based on the face matching score, it is possible to exclude illuminance that is inappropriate for control, and therefore to evaluate the illuminance appropriately.

[0079] For example, even if the illuminance of a captured face image is appropriate for face recognition processing, if the captured face image and the registered face image are not of the same person, the face matching score will be small. If the illuminance of a face image in such a case is included in the illuminance to be evaluated, the illuminance itself is appropriate for face recognition, so it will be impossible to evaluate the illuminance appropriately. In the example shown in Figure 3, by not measuring the illuminance of a face image when the face matching score is below the illuminance evaluation threshold, the illuminance of a face image in a case where the captured face image and the registered face image are not of the same person can be excluded from the illuminance to be evaluated.

[0080] For example, if the illuminance of a captured face image is appropriate for face recognition processing and the captured face image and the registered face image are of the same person, the face matching score will be sufficiently large. If the illuminance of a face image in such a case is included in the illuminance to be evaluated, the illuminance itself is appropriate for face recognition, so appropriate illuminance evaluation cannot be performed. In the example shown in Figure 3, by not measuring the illuminance of a face image when the face matching score is equal to or greater than the upper evaluation limit, the illuminance of a face image in a case where the illuminance of the captured face image is appropriate for face recognition processing and the captured face image and the registered face image are of the same person can be excluded from the illuminance to be evaluated.

[0081] In other words, as shown in Fig. 3, by measuring the illuminance of a face image when the face matching score is within a predetermined range, it is possible to evaluate the illuminance of a face image in cases where the captured face image and the registered face image are of the same person but the illuminance of the captured face image is not appropriate for face recognition processing. In this way, by limiting the illuminance to be evaluated for controlling the lighting device based on the face matching score, it is possible to evaluate appropriate illuminance.

[0082] <Lighting device control flow> Next, an example of the flow of control of the lighting device 13 executed in the face recognition ticket gate system 1 will be described.

[0083] Fig. 4 is a flowchart showing an example of the flow of control of lighting device 13 in this embodiment. The flowchart shown in Fig. 4 is executed, for example, at regular intervals (for example, every minute).

[0084] The face authentication control device 11 extracts information stored in the face authentication server 15 (for example, the time when the face image was captured and the measured face illuminance stored in S106 of FIG. 3) (S301).

[0085] The face authentication control device 11 calculates the average value of the facial illuminance of the data within a specified time period (S302). The specified time period is not particularly limited, but may be, for example, the one minute between the current time and one minute before the current time period.

[0086] The face authentication control device 11 determines whether the calculated average value is smaller than a first threshold value (S303). The first threshold value may be, for example, the lower limit of the specified face illuminance.

[0087] If the calculated average value is smaller than the first threshold value (YES in S303), the face authentication control device 11 performs a light increasing process (S304). Then, the flow ends. An example of the light increasing process will be described later.

[0088] If the calculated average value is not smaller than the first threshold value (NO in S303), the face authentication control device 11 determines whether the calculated average value is larger than a second threshold value (S305). The second threshold value may be, for example, the upper limit of the specified facial illuminance. The first threshold value and the second threshold value may be the same.

[0089] If the calculated average value is greater than the second threshold value (YES in S305), the face authentication control device 11 performs a dimming process (S306). Then, the flow ends. An example of the dimming process will be described later.

[0090] If the calculated average value is not greater than the second threshold value (NO in S305), the flow ends without increasing or decreasing the brightness of the illumination device 13.

[0091] As shown in FIG. 4, the illumination device 13 can be controlled to increase or decrease the brightness based on the illuminance of the face image when the face matching score is within a predetermined range, thereby controlling the illumination device 13 so that the illuminance of the face image is suitable for face recognition.

[0092] <Variations in measuring facial illuminance> Variations in measuring facial illuminance will now be described. There are no limitations on the method for measuring facial illuminance. For example, if a face frame is detected by face detection on a captured image, the illuminance of at least a part of the area surrounded by the face frame (hereinafter referred to as the face frame area) may be measured. The area surrounded by the face frame corresponds to a face image.

[0093] Fig. 5 is a diagram showing variations in measuring facial illuminance, showing a face frame detected in a captured image and four variations of measurement targets for the area surrounded by the face frame (face frame area).

[0094] In variation 1, the entire face frame area is the subject of illuminance measurement. In variation 1, the average brightness of the pixels within the face frame is calculated as the facial illuminance.

[0095] In variation 2, a specific area within the face frame is the target for measuring illuminance. In variation 2, the average brightness of the pixels in the specific area, calculated from the coordinates of the face frame, is calculated as the facial illuminance. For example, the specific area is calculated from the ratio of the top, bottom, left, and right based on the coordinates of the face frame.

[0096] In Variation 3, a specific region within the face frame is the target for illuminance measurement. In Variation 3, the average brightness of the pixels in the specific region obtained relative to the coordinates of the face frame and the coordinates of the eyes is calculated as the facial illuminance. In Variation 3, for example, the region corresponding to the cheeks is included in the specific region. The X coordinate of the region corresponding to the cheeks corresponds to the X coordinate of the lower part of the eye centered on the X coordinate of the eye, and the Y coordinate of the region around the cheeks is determined from the ratio of the coordinate of the eye to the coordinate of the face frame. Also, in Variation 3, for example, the region corresponding to the forehead of the face is included in the specific region. The X coordinate of the region corresponding to the forehead corresponds to the X coordinate of the upper part of the X coordinate of the eye, and the Y coordinate of the region corresponding to the forehead is included between the Y coordinate of the eye and the Y coordinate of the upper part of the face frame. For example, the region corresponding to the forehead may be determined to be a trapezoidal region above the X coordinate of the eye.

[0097] The cheeks and / or forehead include a larger area of ​​skin than the eyes, mouth, etc., and therefore the accuracy of measuring the illuminance on the face can be increased.

[0098] In Variation 4, when the person to be measured is wearing an attachment on their face, the area avoiding the attachment is the object of illuminance measurement. In Variation 4, for example, the area avoiding the mask is the object of illuminance measurement. For example, whether or not a mask is being worn is determined, and if it is determined that a mask is being worn, the mask area is excluded from the area within the face frame, and the average brightness of the pixels in the area excluding the mask area is calculated as the facial illuminance. Note that the method for determining whether or not a mask is being worn is not particularly limited. Furthermore, the method for determining the mask area to be excluded is not particularly limited. For example, a lower portion of the area surrounded by the face frame (for example, the lower half of the area surrounded by the face frame) may be determined to be the mask area.

[0099] In a facial image in which a person is wearing a white mask, if the average brightness of each pixel in the entire area, including the mask area, is calculated, the brightness may be extremely high. Also, in a facial image in which a person is wearing a black mask, if the average brightness of each pixel in the entire area, including the mask area, is calculated, the brightness may be extremely low. Because brightness varies greatly depending on the color of the mask, excluding the mask area allows for more accurate calculation of facial illuminance.

[0100] Although the above-described variation 4 shows an example in which the masked area is excluded, the present disclosure is not limited to this. For example, if the person to be detected is wearing sunglasses, the area avoiding the sunglasses may be used as the illuminance measurement target.

[0101] The variation of the facial illuminance measurement may be changed for each face image to be measured. For example, if the face image to be measured includes an attachment, variation 4 may be applied, and if the face image does not include an attachment, any of variations 1 to 3 may be applied.

[0102] It should be noted that the variations in measuring the facial illuminance are not limited to the four mentioned above.

[0103] <Facial image quality check> The above-described measurement of face illuminance does not have to be performed for all face images corresponding to face matching scores within a predetermined range. Face illuminance measurement may be performed for a portion of face images corresponding to face matching scores within a predetermined range. Furthermore, instead of treating all face images corresponding to face matching scores within a predetermined range equally, face illuminance may be measured by weighting each face image. Below, an example will be described in which the quality of face images is checked and face illuminance measurement is performed according to the quality.

[0104] The face authentication server 15 may perform a quality check on the face image that is the target of the face matching process. By performing the quality check, quality check information indicating the quality of the face image is obtained. This quality check information may be used to determine face images to be excluded from the measurement target of facial illuminance. For example, face images whose quality level is equal to or higher than a predetermined level may be determined as the measurement target of facial illuminance, and face images whose quality level is lower than the predetermined level may be excluded from the measurement target of facial illuminance. Furthermore, this quality check information may be used to weight the measured face illuminance. For example, among the measurement target of facial illuminance, a weight may be set higher for face images whose quality level is higher than a predetermined level than for face images whose quality level is lower than the predetermined level.

[0105] The face authentication server 15 may assign identification information (e.g., ID) to a face image that has undergone a quality check. The face authentication server 15 may then determine whether or not a face image that is the target of face matching processing is a face image to which identification information has been assigned, so as not to perform a quality check again on a face image that has already undergone a quality check. For example, if the face image that is the target of face matching processing is a face image to which identification information has been assigned, the quality check may not be performed, and if the face image that is the target of face matching processing is not a face image to which identification information has been assigned, the quality check may be performed. This reduces the processing load for quality checks.

[0106] Although an example has been shown in which the face authentication server 15 checks the quality of a face image, the present disclosure is not limited to this. For example, the face authentication control device 11 may check the quality of a face image. The face authentication control device 11 may check the quality of a face image, determine a face image whose quality level is equal to or higher than a predetermined level as a target for measuring the face illuminance, and perform illuminance measurement of the face image.

[0107] FIG. 6 is a diagram showing examples of quality check items for a facial image. In the example shown in FIG. 6, a total of 12 check items are shown, including "wearing a hat," "face obscured by hair," etc. A quality check of the items shown in FIG. 6 is performed on a facial image, and a numerical value (e.g., a score) indicating the quality of the facial image is determined. For example, for the check item "wearing a hat," one point is deducted if a hat is worn, and one point is added if a hat is not worn. Based on the quality numerical value determined according to the result of each check item, it may be determined whether or not the facial image corresponding to that numerical value is to be subject to facial illuminance measurement, and the magnitude of weighting may be determined.

[0108] Note that the check items shown in Fig. 6 are merely examples, and the present disclosure is not limited thereto. Quality checks may be performed using at least one of the check items shown in Fig. 6, or check items not included in Fig. 6 may be used.

[0109] <Parameter control of lighting device 13> Next, an example of parameter control of the lighting device 13 will be described. As the parameter control of the lighting device 13, for example, the luminance of the lighting device 13 is controlled. Note that a single luminance for one lighting device 13 may be controlled. Alternatively, one lighting device 13 may be divided into multiple blocks, and the luminance of each of the multiple blocks may be controlled.

[0110] In the first example of parameter control of the lighting device 13 shown below, the lighting device 13 is divided into two lighting blocks, lighting block A and lighting block B, and the brightness of each is controlled. For example, lighting block A is a lighting block that extends vertically from the face recognition ticket gate 12 and irradiates light directly onto the face. Lighting block B is a lighting block that extends horizontally and irradiates light onto the face from above.

[0111] Fig. 7 is a diagram showing a first example of parameter control of the lighting device 13. The example in Fig. 7 is an example in which the brightness of two lighting devices 13, lighting block A and lighting block B, is controlled in 10 stages. Fig. 7 shows, in table format, the control values ​​for the brightness of lighting block A and lighting block B in each of the 10 stages from stage #0 to stage #9.

[0112] Of the 10 levels, level #9 has the smallest control value and level #0 has the largest control value. In other words, the brightness of lighting device 13 controlled using the control value of level #9 is the smallest, and the brightness of lighting device 13 controlled using the control value of level #0 is the largest. For example, the smallest average value of facial illuminance corresponds to level #0, and the largest average value of facial illuminance corresponds to level #9.

[0113] In the first example, when it is determined that dimming processing is to be performed based on the average value of the facial illuminance (for example, S306 in FIG. 4), the lighting device 13 is controlled using a control value that is one step higher than the current control value so as to dim the light from the lighting device 13. When it is determined that dimming processing is to be performed based on the average value of the facial illuminance (for example, S304 in FIG. 4), the lighting device 13 is controlled using a control value that is one step lower than the current control value so as to increase the illumination light.

[0114] In the example of Figure 7, if the current control value is the control value of stage #5 and it is determined that dimming processing is to be performed based on the average facial illuminance, the brightness of lighting block A and lighting block B is controlled by increasing the stage by one and using the control value of stage #6.

[0115] In the example of Figure 7, if the current control value is the control value of stage #3 and it is determined that brightening processing should be performed based on the average facial illuminance, the brightness of lighting block A and lighting block B is controlled by lowering the stage by one and using the control value of stage #2.

[0116] As described above, in the first example, when it is determined that dimming processing should be performed, the control value is increased by one step from the current control value, and when it is determined that brightening processing should be performed, the control value is decreased by one step from the current control value, that is, the control value is changed in steps from the current control value. This causes the illumination light to change in steps, making it possible to avoid a situation in which the illumination light suddenly becomes brighter or darker, causing discomfort to the user.

[0117] In the second example of parameter control of the lighting device 13 shown below, similar to the first example, the lighting device 13 is divided into two lighting blocks, lighting block A and lighting block B, and the brightness of each is controlled. In the second example, control is performed to directly change the control value, regardless of the current control value.

[0118] Fig. 8 is a diagram showing a second example of parameter control of the lighting device 13. Similar to the example of Fig. 7, the example of Fig. 8 is an example in which the brightness of two lighting devices 13, lighting block A and lighting block B, is controlled in 10 levels. Fig. 8 shows, in table format, the ranges of average values ​​of facial illuminance corresponding to each of the 10 levels from level #0 to level #9, and the control values ​​of lighting block A and lighting block B corresponding to these ranges of average values.

[0119] Among the 10 levels, level #9 has the highest average value of facial illuminance, and level #0 has the lowest average value of facial illuminance. Level #9 has the lowest control value, and level #0 has the highest control value. In other words, the brightness of lighting device 13 controlled using the control value of level #9 is the lowest, and the brightness of lighting device 13 controlled using the control value of level #0 is the highest.

[0120] In the second example, the lighting device 13 is controlled based on the average value of the facial illuminance using a control value associated with the average value of the facial illuminance.

[0121] In the example of Figure 8, if the current control value is the control value of stage #3 and the average face illuminance is 69, the lighting device 13 is controlled using the control value of stage #6, which corresponds to the average face illuminance being 69.

[0122] As described above, in the second example, regardless of the current control value level, control is performed to irradiate the face with illumination light appropriate for the average face illumination level. This allows control to be performed so that appropriate illumination light is irradiated when the face illumination level suddenly drops or rises, thereby preventing a situation in which the quality of face recognition processing suddenly deteriorates.

[0123] <Variations involving zone determination> On the path to the facial recognition ticket gate 12, the facial illuminance may change depending on the distance from the facial recognition ticket gate 12. For example, when a user is located far from the facial recognition ticket gate 12, the influence of the lighting device 13 is low, and the closer the user is to the facial recognition ticket gate 12, the higher the facial illuminance becomes due to the light emitted by the lighting device 13, and the increase in illuminance by the lighting device becomes more noticeable.

[0124] The facial recognition ticket gate 12 measures the distance between the facial recognition ticket gate 12 and the user, and may start facial recognition when the distance reaches a predetermined value. The area defined according to the distance from the facial recognition ticket gate 12 may be called a zone.

[0125] Fig. 9 is a diagram showing an example of the positional relationship between the zones of the facial recognition ticket gate and users. Fig. 9 shows, as an example, three zones according to the distance from the facial recognition ticket gate.

[0126] Of the three zones, authentication start zone 1 corresponds to the zone where facial authentication starts. For example, when a user enters authentication start zone 1, facial authentication for the user starts.

[0127] Authentication start zone 2 corresponds to a zone located closer to the face authentication ticket gate 12 than authentication start zone 1. The effect of illumination light irradiated on a user present in authentication start zone 2 is more pronounced than the effect of illumination light on a user present in authentication start zone 1.

[0128] When a user enters through the entrance of facial recognition ticket gate 12 and it is determined through facial authentication that the user is permitted to pass, a charge confirmation process is carried out. As shown in Fig. 9, the zone inside facial recognition ticket gate 12 beyond the entrance where the charge confirmation process is carried out corresponds to the charge confirmation zone. In the charge confirmation zone, the influence of illumination light is more pronounced than in authentication start zone 1 and authentication start zone 2.

[0129] It is desirable for the facial authentication ticket gate 12 to start facial authentication in authentication start zone 1 and notify the user of the results of facial authentication while the user is present in authentication start zone 1 or authentication start zone 2. Therefore, it is desirable to improve the accuracy of facial authentication in authentication start zone 1 and authentication start zone 2. As described above, the influence of illumination light is more pronounced in authentication start zone 2 than in authentication start zone 1, and therefore control may be performed in authentication start zone 2 to ensure appropriate illuminance by the lighting device.

[0130] In order to ensure that the illumination device 13 provides appropriate illumination in the authentication start zone 2, the zone in which the user is present is determined. Then, control is executed according to the zone in which the user is present. For example, the facial illumination of a facial image when the user is present in a specific zone may be measured, and facial images when the user is present in a zone other than the specific zone may be excluded from the measurement of facial illumination.

[0131] The method for determining the zone in which the user is present is not particularly limited. Because the size of the face in the captured image varies depending on the zone in which the user is present, the zone in which the user is present may be determined based on the size of the face frame detected by face detection processing. Alternatively, the zone in which the user is present may be determined based on the distance between the left and right eyes of the detected face. Alternatively, when two cameras 14 are used for capturing images, the zone in which the user is present may be determined based on the images captured by the two cameras.

[0132] Fig. 10 is a flowchart showing a second example of the processing flow for accumulating the execution results of face authentication processing in this embodiment. In Fig. 10, the same processes as those in Fig. 3 are denoted by the same reference numerals, and their explanations may be omitted.

[0133] In the example of FIG. 10, when a face is detected (YES in S103), the face authentication control device 11 performs zone determination to determine in which zone the user is present (S401).

[0134] After the face matching process and the illuminance measurement (after S104), the face authentication control device 11 determines whether the user is present in a specific zone (S402). In the example of Fig. 9 described above, the authentication start zone 2 may be set as a specific zone, or both the authentication start zone 1 and the authentication start zone 2 may be set as specific zones.

[0135] If the user is not present in the specific zone (NO in S402), the flow returns to S101.

[0136] If the user is present in a specific zone (YES in S402), the process of S105 is executed.

[0137] If the face matching score is within the predetermined range (YES in S105), the face authentication server 15 stores the zone in which the face image was captured, the time when the face image was captured, and the measured face illuminance (S403). Then, the processes from S201 onwards are executed.

[0138] 10, the face authentication server 15 stores the facial illuminance of the face image of the user when the face matching score is within a predetermined range and the user is present in a specific zone. In this case, the face authentication control device 11 controls the parameters related to photography (for example, the parameters of the lighting device 13) based on the facial illuminance of the face image of the user when the face matching score is within a predetermined range and the user is present in a specific zone.

[0139] 10, the illuminance of a face image is measured when the user is present in a specific zone and the face matching score is within a predetermined range. In this way, by limiting the illuminance to be evaluated for controlling the lighting device based on the zone and the face matching score, it is possible to perform an appropriate illuminance evaluation.

[0140] <Camera parameter control> In the above example, an example of controlling parameters (e.g., brightness) of the lighting device 13 has been described, but the present disclosure is not limited to this. Below, an example of controlling parameters of the camera 14 instead of the lighting device 13 will be described. Note that, as parameter control of the camera 14, the exposure time and gain of the camera 14 are exemplarily controlled.

[0141] Fig. 11 is a flowchart showing an example of the flow of control of camera 14 in this embodiment. The flowchart shown in Fig. 11 is executed, for example, at regular intervals (for example, every minute). In Fig. 11, the same processes as those in Fig. 4 are given the same reference numerals, and their explanation may be omitted.

[0142] If the average value calculated in S302 is smaller than the first threshold value (YES in S303), the face authentication control device 11 executes a sensitization process to control the camera parameters (S501). For example, the sensitization process increases the exposure time and / or increases the gain. Then, the flow ends.

[0143] If the average value calculated in S302 is not smaller than the first threshold value (NO in S303), the face authentication control device 11 determines whether the calculated average value is larger than the second threshold value (S305). Note that the first threshold value and the second threshold value may be the same. Alternatively, the second threshold value may be larger than the first threshold value.

[0144] If the average value calculated in S302 is greater than the second threshold value (YES in S305), the face authentication control device 11 executes desensitization processing as a control of the camera parameters (S502). For example, the desensitization processing reduces the exposure time and / or reduces the gain. Then, the flow ends.

[0145] If the exposure time of the camera is extended, the image captured by the camera may be blurred, which may result in an image that is not suitable for face recognition. Therefore, an upper limit may be set for the exposure time.

[0146] Furthermore, if the gain of the camera is increased, noise in the image captured by the camera increases, which may result in an image that is not suitable for face recognition. Therefore, an upper limit may be set for the gain.

[0147] As the camera parameter control, both the exposure time and the gain of the camera may be controlled, or only one of them may be controlled.

[0148] When combining exposure time and gain, combinations of exposure time and gain values ​​may be stored in advance, and the exposure time and gain values ​​may be controlled according to the average value of the facial illuminance.

[0149] Below, two examples of controlling the exposure time and gain value will be described.

[0150] Fig. 12 is a diagram showing a first example of camera parameter control. The example in Fig. 12 is an example in which two parameters, exposure time and gain value of the camera, are controlled in 10 stages. Fig. 12 shows the control values ​​of exposure time and gain value in a table format for each of the 10 stages from stage #0 to stage #9.

[0151] Of the 10 levels, level #9 has the smallest control value and level #0 has the largest control value. In other words, the sensitivity of the camera controlled using the control value of level #9 is the lowest, and the sensitivity of the camera controlled using the control value of level #0 is the highest. For example, the smallest average value of facial illuminance corresponds to level #0, and the largest average value of facial illuminance corresponds to level #9.

[0152] In the first example, when it is determined that desensitization processing is to be performed based on the average value of facial illuminance, the camera is controlled using a control value that is one step higher than the current control value so that the camera sensitivity is reduced. Also, when it is determined that sensitization processing is to be performed based on the average value of facial illuminance, the camera is controlled using a control value that is one step lower than the current control value so that the camera sensitivity is increased.

[0153] In the example of Figure 12, if the current control value is the control value of stage #5 and it is determined that desensitization processing is to be performed based on the average face illuminance, the next stage is raised and the control value of stage #6 is used to control the camera's exposure time and gain value.

[0154] In the example of Figure 12, if the current control value is the control value of stage #3 and it is determined that sensitization processing is to be performed based on the average facial illuminance, the stage is lowered by one and the camera exposure time and gain value are controlled using the control value of stage #2.

[0155] As described above, in the first example, if it is determined that desensitization processing is to be performed, the control value is increased by one step from the current control value, and if it is determined that sensitization processing is to be performed, the control value is decreased by one step from the current control value, that is, the control value is changed in steps relative to the current control value.

[0156] In the second example of camera parameter control shown below, the exposure time and gain value of the camera are controlled in the same way as in the first example. In the second example, the control value is changed directly, regardless of the current control value.

[0157] Fig. 13 is a diagram showing a second example of camera parameter control. Similar to the example of Fig. 12, the example of Fig. 13 is an example in which two camera parameters, exposure time and gain value, are controlled in 10 stages. Fig. 13 shows, in table form, the range of average values ​​of facial illuminance corresponding to each of the 10 stages from stage #0 to stage #9, and the control values ​​of exposure time and gain value corresponding to this range of average values.

[0158] Among the 10 levels, level #9 has the highest average face illuminance, and level #0 has the lowest average face illuminance. Level #9 also has the lowest control value, and level #0 has the highest control value. In other words, the brightness of the auxiliary lighting controlled using the control value of level #9 is the lowest, and the brightness of the auxiliary lighting controlled using the control value of level #0 is the highest.

[0159] In a second example, the auxiliary lighting is controlled based on the average value of the facial illuminance using a control value associated with the average value of the facial illuminance.

[0160] In the example of Figure 13, if the current control value is the control value of stage #3 and the average facial illuminance is 78, the auxiliary lighting is controlled using the control value of stage #7, which corresponds to the average facial illuminance being 78.

[0161] As described above, in the second example, the exposure time and gain value are controlled to be appropriate for the average face illuminance, regardless of the current control value level. This allows the exposure time and gain value to be controlled to achieve appropriate sensitivity when the face illuminance suddenly drops or rises, thereby preventing a sudden deterioration in the quality of face recognition processing.

[0162] As described above, by controlling the exposure time and gain value of the camera and changing the sensitivity of the camera, it is possible to capture an image suitable for face recognition. For example, when the external light (e.g., sunlight) is very strong and the illuminance on the face is higher than appropriate even when the lighting device is turned off, it is possible to capture an image suitable for face recognition by changing the sensitivity of the camera. Also, when no lighting device is provided, it is possible to capture an image suitable for face recognition by changing the sensitivity of the camera.

[0163] The lighting device parameter control and the camera parameter control described above may be combined or used selectively. For example, at least two of the brightness of the lighting device, the exposure time of the camera, and the gain value of the camera may be controlled based on the average value of facial illuminance. Furthermore, either the lighting device parameter control or the camera parameter control may be selected depending on the time of day, weather, etc. For example, during sunny daytime hours, the lighting device may be turned off and the camera parameter control may be executed, while during cloudy days or at night, the camera parameters may be fixed and the lighting device parameters may be controlled.

[0164] <Lighting control by zone> In each of the above examples, the lighting device parameters and / or camera parameters may be constant while a certain user passes through the facial recognition ticket gate. In the variations shown below, the lighting device parameters and / or camera parameters are changed while a certain user passes through the facial recognition ticket gate. For example, the following variations show an example in which zones are defined according to the distance from the facial recognition ticket gate, and the lighting device parameters are changed for each zone.

[0165] For example, similar to the example shown in Fig. 9, three zones are defined: authentication start zone 1, authentication start zone 2, and billing confirmation zone. When controlling the parameters of the lighting device for each zone, the parameters of the lighting device are controlled so that appropriate facial illuminance is achieved in each of authentication start zone 1, authentication start zone 2, and billing confirmation zone.

[0166] For example, when controlling the parameters of the lighting device for each zone, information on the face illuminance for each zone is accumulated, and the parameters (for example, brightness) of the lighting device for each zone are determined based on the accumulated face illuminance for each zone.

[0167] Fig. 14A is a flowchart for controlling parameters of lighting devices for each zone. The flowchart shown in Fig. 14A is started every time the camera 14 captures an image at a fixed cycle. In Fig. 14A, the same processes as those in Fig. 3 and Fig. 10 are denoted by the same reference numerals, and their explanation may be omitted.

[0168] As shown in FIG. 14A, when a face is detected (YES in S103), a zone determination is performed (S601).

[0169] Then, lighting control for each zone is performed according to the determined zone (S602). For example, a brightness control value for the lighting device is determined for each zone, and the lighting device is controlled using the brightness control value corresponding to the determined zone. Note that a method for setting the brightness control value corresponding to the determined zone will be described later.

[0170] Then, the camera 14 captures an image again (S603).

[0171] Then, the face authentication control device 11 executes the face detection process (S604) in the same manner as in S102 of Fig. 3. Then, the face authentication control device 11 executes the face matching process and the illuminance measurement (S104).

[0172] If the face matching score is within a predetermined range (YES in S105), the face authentication server 15 stores the time and the face illuminance for each zone (S605), and then the processes from S201 onwards are executed.

[0173] Next, a method for setting the brightness control value corresponding to the determined zone will be described.

[0174] Fig. 14B is a flowchart showing an example of determining parameters of lighting devices for each zone. The flowchart shown in Fig. 14B is executed, for example, at regular intervals (for example, every minute).

[0175] The face authentication control device 11 sets k, which corresponds to the zone index, to an initial value of 1 (S701). In this example, the zones are divided into N zones (N is an integer equal to or greater than 2), and each zone is assigned an index of zone #1 to zone #N. As shown in FIG. 9, if there are three zones, authentication start zone 1, authentication start zone 2, and charging confirmation zone may correspond to zones #1, #2, and #3, respectively.

[0176] The face authentication control device 11 extracts the face illuminance information of zone #k (S702).

[0177] The face authentication control device 11 calculates the average value of the face illuminance in zone #k (S703).

[0178] The face authentication control device 11 determines the parameters for zone #k based on the calculated average value (S704).

[0179] The face authentication control device 11 adds 1 to k corresponding to the zone index (S705).

[0180] The face authentication control device 11 determines whether k is greater than N (S706).

[0181] If k is not greater than N (NO at S706), the flow returns to S702 to determine the parameters of the next zone.

[0182] If k is greater than N (YES in S706), the flow ends.

[0183] 14B, the brightness parameters of the lighting device 13 in each zone are determined based on the average value of the face illuminance in each zone. Here, the determined parameters are used in S602 in FIG. 14A.

[0184] Fig. 15 is a diagram showing an example of determining parameters for a lighting device for each zone. The example in Fig. 15 shows an example in which the brightness of lighting device 13 in each of authentication start zone 1, authentication start zone 2, and charging confirmation zone is controlled in 10 levels. Fig. 15 shows in table format the control values ​​of lighting device 13 corresponding to authentication start zone 1, authentication start zone 2, and charging confirmation zone in each of the 10 levels from level #0 to level #9, associated with the range of average values ​​of facial illuminance.

[0185] Of the 10 levels, level #9 has the smallest control value and level #0 has the largest control value. In other words, the brightness of lighting device 13 controlled using the control value of level #9 is the smallest, and the brightness of lighting device 13 controlled using the control value of level #0 is the largest. For example, the smallest average value of facial illuminance corresponds to level #0, and the largest average value of facial illuminance corresponds to level #9.

[0186] In the example of FIG. 15, the lighting device 13 is controlled based on the average value of the facial illuminance using a control value associated with the average value of the facial illuminance.

[0187] In the example of FIG. 15, when the average value of the facial illuminance in authentication start zone 1 is 57, the lighting device 13 is controlled using the control value 55 of stage #5 which corresponds to the average value of the facial illuminance being 57.

[0188] In the example of FIG. 15, when the average value of the face illuminance in the authentication start zone 2 is 75, the lighting device 13 is controlled using the control value 30 of stage #6 which corresponds to the average value of the face illuminance being 75.

[0189] In the example of FIG. 15, when the average value of the facial illuminance in the charge confirmation zone is 92, the lighting device 13 is controlled using the control value 3 of the stage #8 that corresponds to the average value of the facial illuminance being 92.

[0190] As described above, in the example of Fig. 15, control is performed for each zone so that illumination light appropriate for the average face illuminance is emitted, regardless of the current control value level. This allows control so that appropriate illumination light is emitted when the face illuminance suddenly drops or rises, preventing a situation in which the quality of face recognition processing suddenly deteriorates.

[0191] Furthermore, as described above, by performing brightening or dimming processing of the lighting device 13 for each zone based on the illuminance of the face image for each zone when the face matching score is within a predetermined range, the lighting device 13 can be controlled so that the illuminance of the face image for each zone is an illuminance suitable for face recognition.

[0192] <Lighting control for each face frame size> Fig. 16 is a diagram showing an example of changes in face frame size. Fig. 16 shows photographed images #1 to #4 and examples of the size of the face frame detected in each photographed image. The photographed images were taken in the order of photographed image #1 to #4. The user for whom the face frame is to be detected is located at the farthest position from the face authentication ticket gate 12 in photographed image #1, and is located at the closest position to the face authentication ticket gate 12 in photographed image #4.

[0193] As shown in FIG. 16, the closer you are to the face authentication ticket gate 12, the closer you are to the camera 14, and therefore the larger the size of the face frame becomes.

[0194] Fig. 17 is a diagram showing an example of the correspondence between changes in face frame size and changes in facial illuminance for users #1 to #5. Fig. 17 shows an example of the detected face frame size and the corresponding changes in facial illuminance for each of users #1 to #5.

[0195] For example, the face authentication control device 11 may store the face illuminance for each size of the face frame, and may control the lighting device 13 for each size of the face frame.

[0196] As shown in Figures 16 and 17, by performing brightening or dimming processing of the lighting device 13 for each face frame size based on the illuminance of the face image for each face frame size when the face matching score is within a predetermined range, the lighting device 13 can be controlled so that the illuminance of the face image is suitable for face recognition for each face frame size.

[0197] FIG. 18 is a diagram showing an outline of control of the lighting device in this embodiment.

[0198] As shown in Fig. 18, the face recognition ticket gate system 1 in this embodiment accumulates facial illuminance, subject distance, face frame evaluation, face matching score, passing result, and current control value. The subject distance may be indicated, for example, by the zone as described above, or by the size of the face frame. The face frame evaluation may be an evaluation based on the check items shown in Fig. 6, for example. The passing result may indicate whether or not the user corresponding to the face image whose facial illuminance was measured was able to pass.

[0199] Then, a control value for each distance is set based on the accumulated information, and the lighting device is controlled for each distance based on the set control value, as shown in Fig. 18. Note that the control for each distance may correspond to control for each zone or control for each size of the face frame.

[0200] Then, as shown in FIG. 18, information such as facial illuminance obtained as a result of controlling the lighting device is accumulated as feedback information.

[0201] As shown in FIG. 18, by storing, controlling, and feeding back information by distance (for example, by zone or by size of face frame), the lighting device 13 can be controlled so that the illuminance of the face image is suitable for face recognition by distance.

[0202] In the present embodiment described above, the facial recognition ticket gate system 1 (an example of a facial recognition gate system) includes a facial recognition ticket gate 12 (an example of a gate device), a facial recognition control device 11 (an example of a control device), and a facial recognition server 15. The facial recognition ticket gate system 1 determines whether or not a user is permitted to pass through the facial recognition ticket gate 12 based on the result of a face matching process between pre-registered face information and photographed face information based on a captured image. The facial recognition server 15 transmits the result of the face matching process between the registered face information and photographed face information based on a captured image of a user attempting to pass through the facial recognition ticket gate 12 (e.g., a face matching score) to the facial recognition control device 11. If the result of the face matching process satisfies a first condition (e.g., if the face matching score is within a predetermined range), the facial recognition control device 11 controls parameters related to photography (e.g., at least one of parameters of the lighting device 13 and parameters of the exposure or sensitivity of the camera 14) based on the captured image.

[0203] In the present embodiment described above, variations on the control of the lighting device 13 may be applied to the control of the camera 14. For example, the camera 14 may be controlled for each zone or for each size of the face frame.

[0204] In each of the above-described embodiments, instead of placing the facial recognition server centrally so as to be connected to facial recognition ticket gates at a plurality of stations, the facial recognition server may be placed at each station, or the configuration of the facial recognition server may be mounted inside the facial recognition ticket gate. For example, when the performance of information processing devices (e.g., PCs) and / or processors (e.g., GPUs) included in the information processing devices improves and further becomes inexpensive and versatile, the facial recognition server may be placed at each station, or the configuration of the facial recognition server may be mounted inside the facial recognition ticket gate.

[0205] However, if the facial recognition server is installed inside the facial recognition ticket gate, the facial recognition servers inside the facial recognition ticket gates that are installed at the same ticket gate may share cache information.

[0206] <Summary of the embodiment> A facial recognition gate system according to one embodiment of the present disclosure comprises a gate device, a control device, and a facial recognition server, and determines whether or not a user can pass through the gate device based on the results of a facial matching process between pre-registered facial information and photographed facial information based on a photographed image, wherein the facial recognition server transmits the results of the facial matching process between the registered facial information and photographed facial information based on an image of a user attempting to pass through the gate device to the control device, and the control device controls parameters related to the photographing based on the illuminance of the photographed image when the results of the facial matching process satisfy a first condition.

[0207] In this face recognition gate system, the parameter related to the image capture is at least one of a parameter of an illumination device that irradiates illumination light onto the face of the user, and an exposure or sensitivity parameter of a capture unit that captures the image.

[0208] In this facial recognition gate system, a plurality of zones are defined in the direction away from the entrance of the gate device according to the distance from the entrance, and when the result of the face matching process satisfies the first condition and when the user is present in a specific zone among the plurality of zones, the control device controls the parameters related to the photographing based on the illuminance of the photographed image.

[0209] In this facial recognition gate system, multiple zones are defined in the direction away from the entrance of the gate device according to the distance from the entrance, and the control device controls the parameters related to the photography according to the zone in which the user is located.

[0210] In this facial recognition gate system, if the result of the facial matching process satisfies the first condition but does not satisfy a second condition that is higher than the first condition, the control device accumulates the illuminance to be used to control the parameters related to the photography.

[0211] In this facial recognition gate system, if the result of the facial matching process satisfies the first condition and a second condition that is higher than the first condition, the control device does not accumulate the illuminance used to control the parameters related to the photography.

[0212] In this face recognition gate system, the control device or face recognition server measures the illuminance of at least a part of the face area extracted from the captured image.

[0213] In this face recognition gate system, the control device controls the parameters related to the photographing in accordance with the size of the face area extracted from the photographed image.

[0214] A control device according to one embodiment of the present disclosure includes a communication unit that acquires the results of a face matching process between captured face information based on an image of a user attempting to pass through a gate device and pre-registered face information, and a control unit that controls parameters related to the photographing based on the illuminance of the image when the result of the face matching process satisfies a first condition.

[0215] In a control method according to one embodiment of the present disclosure, a control device acquires the results of a face matching process between captured face information based on an image of a user attempting to pass through a gate device and pre-registered registered face information, and controls parameters related to the photographing based on the illuminance of the image when the result of the face matching process satisfies a first condition.

[0216] A program according to one embodiment of the present disclosure causes a control device to acquire the results of a face matching process between captured face information based on an image of a user attempting to pass through a gate device and pre-registered registered face information, and controls parameters related to the photographing based on the illuminance of the image when the result of the face matching process satisfies a first condition.

[0217] The present disclosure can be realized in software, hardware, or software in conjunction with hardware.

[0218] Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0219] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0220] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility.

[0221] The present disclosure may be implemented in any type of apparatus, device, or system (collectively referred to as a communications apparatus) that has a communications function. The communications apparatus may include a wireless transceiver and processing / control circuitry. The wireless transceiver may include a receiver and a transmitter, or both functions. The wireless transceiver (transmitter and receiver) may include a radio frequency (RF) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of communication devices include telephones (e.g., cell phones, smartphones), tablets, personal computers (PCs) (e.g., laptops, desktops, notebooks), cameras (e.g., digital still / video cameras), digital players (e.g., digital audio / video players), wearable devices (e.g., wearable cameras, smartwatches, tracking devices), game consoles, digital book readers, telehealth / telemedicine devices, communication-enabled vehicles or mobile transportation (e.g., cars, airplanes, ships), and combinations of the above devices.

[0222] Communications equipment is not limited to portable or mobile equipment, but also includes non-portable or fixed equipment, devices, and systems of any kind, such as smart home devices (such as appliances, lighting equipment, smart meters or metering devices, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0223] Furthermore, in recent years, in the field of IoT (Internet of Things) technology, CPS (Cyber ​​Physical Systems) has been attracting attention as a new concept that creates new added value by linking information between physical space and cyberspace. This CPS concept can also be adopted in the above-mentioned embodiments.

[0224] That is, as a basic configuration of a CPS, for example, an edge server located in physical space and a cloud server located in cyberspace can be connected via a network, and processing can be distributed and performed by processors installed on both servers. Here, it is preferable that each piece of processing data generated on the edge server or cloud server is generated on a standardized platform, and the use of such a standardized platform can improve the efficiency of building a system that includes a variety of sensor groups and IoT application software.

[0225] Communications include data communications via cellular systems, wireless LAN systems, communications satellite systems, etc., as well as data communications via combinations of these.

[0226] A communications apparatus also includes devices such as controllers and sensors connected or coupled to a communications device that performs the communications functions described in this disclosure, such as controllers and sensors that generate control and data signals used by the communications device to perform the communications functions of the communications apparatus.

[0227] The communication apparatus also includes infrastructure facilities, such as base stations, access points, and any other apparatus, device, or system that communicates with or controls the various apparatuses listed above, but are not limited to these.

[0228] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the disclosure.

[0229] Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Industrial Applicability]

[0230] An embodiment of the present disclosure is suitable for a face authentication system. [Explanation of symbols]

[0231] 1. Facial recognition ticket gate system 11 Face recognition control device 12 Facial recognition ticket gates 13 Lighting equipment 14 Camera 111 Face matching request processing unit 112 Ticket gate control unit 113, 152 Communications Department 121 Lighting control unit 151 Face matching processing unit

Claims

1. A facial recognition gate system includes a gate device, a control device, and a facial recognition server, and determines whether or not a person can pass through the gate device based on the result of a facial matching process between pre-registered face information and photographed face information based on a photographed image, The face authentication server a face matching score, which is a score of face matching between the registered face information and photographed face information based on an image of the user attempting to pass through the gate device, transmitted to the control device; The control device controlling the parameters related to the photographing based on the illuminance of the photographed image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an evaluation upper limit value; Facial recognition gate system.

2. The parameter related to the photographing is at least one of a parameter of an illumination device that irradiates illumination light onto the face of the user and a parameter of exposure or sensitivity of an image capturing unit that captures the image. The facial recognition gate system according to claim 1 .

3. a plurality of zones according to distances from the entrance of the gate device are defined in a direction away from the entrance, The control device controlling the parameters related to the photographing based on the illuminance of an image of the user when the face matching score is equal to or greater than the illuminance evaluation threshold value and less than the evaluation upper limit value, and when the user is present in a specific zone among the plurality of zones; The facial recognition gate system according to claim 1 .

4. a plurality of zones according to distances from the entrance of the gate device are defined in a direction away from the entrance, The control device controlling parameters related to the photographing depending on the zone in which the user is present; The facial recognition gate system according to claim 1 .

5. The control device When the face matching score is equal to or greater than the illuminance evaluation threshold value and less than the evaluation upper limit value, the illuminance to be used for controlling the parameters related to the photographing is stored. The facial recognition gate system according to claim 1 .

6. The control device When the face matching score is less than the illuminance evaluation threshold value or is equal to or greater than the evaluation upper limit value, the illuminance to be used for controlling the parameters related to the photographing is not accumulated. The facial recognition gate system according to claim 1 .

7. The control device or the face authentication server measuring the illuminance of at least a portion of a face region extracted from the captured image; The facial recognition gate system according to claim 1 .

8. The control device controlling the parameters related to the photographing in accordance with the size of the face area extracted from the photographed image; The facial recognition gate system according to claim 1 .

9. a communication unit that acquires a face matching score, which is a score of face matching between captured face information based on an image of a user who is about to pass through the gate device and pre-registered face information; a control unit that controls parameters related to the photographing based on the illuminance of the image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an evaluation upper limit value; A control device comprising:

10. The control device A face matching score is obtained, which is a score of face matching between photographed face information based on an image of a user who is about to pass through the gate device and pre-registered face information; controlling the parameters related to the photographing based on the illuminance of the image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an evaluation upper limit value; Control method.

11. The control device A face matching score is obtained, which is a score of face matching between photographed face information based on an image of a user who is about to pass through the gate device and pre-registered registered face information; controlling the parameters related to the photographing based on the illuminance of the image when the face matching score is equal to or greater than an illuminance evaluation threshold value and less than an evaluation upper limit value; A program that executes a process.

Images