Gate device, method and program for controlling gate device

Abstract

To provide a gate device that accurately determines a user to be authenticated.SOLUTION: A gate device includes an acquisition section, a setting section, and a gate control section. The acquisition section acquires an upper image and a lower image, respectively, from an upper camera and a lower camera installed on a vertically lower side than the upper camera. When a plurality of users are captured in both the upper image and the lower image, the setting section identifies a user in the very front row among the plurality of users by calculating the distance between each of the plurality of users and a camera installation surface, and sets the identified user as a person to be authenticated. The gate control section controls a gate on the basis of the authentication result of the person to be authenticated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a gate device, an authentication system, a control method for a gate device, and a storage medium. [Background technology]

[0002] Services using facial recognition are beginning to become widespread (see Patent Documents 1 to 3). For example, facial recognition is beginning to be applied to various procedures at airports (e.g., check-in, baggage drop-off, security check, etc.). Services using facial recognition are often provided using gate devices. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-236244 A [Patent Document 2] JP 2018-128970 A [Patent Document 3] JP 2018-109935 A Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, services using face recognition have begun to be provided at airports and the like. For example, security checks using face recognition are performed at security checkpoints at airports. However, existing airports have limited space for installing equipment, and the face recognition-enabled gate devices are often installed in small spaces.

[0005] Therefore, when the gate device captures an image (an image that includes a facial area), the image may contain facial images of not only the users in the front row (users to be authenticated) but also users behind them. In other words, because a gate device that supports facial recognition is installed in a small space, the distance between users lined up at the gate device is short, and multiple people may end up in the same image.

[0006] In this case, the gate device must select one face image from the multiple face images and set the selected face image as the authentication target. For example, the gate device may calculate the area (size) of each of the multiple face images and set the face image with the largest area as the authentication target. This approach is based on the premise that the face images of users lined up in the front row of the gate device are larger than the face images of users lined up behind them. However, there are large individual differences in face size, and the above approach (determining the authentication target based on the area of ​​the face image) may result in a user lined up behind the user lined up in the front row being set as the authentication target.

[0007] A primary object of the present invention is to provide a gate device, an authentication system, a control method for a gate device, and a storage medium that contribute to accurately determining a user to be authenticated. [Means for solving the problem]

[0008] According to a first aspect of the present invention, there is provided a gate device comprising: an acquisition unit that acquires an upper image from an upper camera, and acquires a lower image having a common area that is partially common to the upper image from a lower camera installed vertically below the upper camera; a setting unit that, when multiple users are shown in the upper and lower images, identifies the user in the front row by calculating the distance between each of the multiple users and the camera installation surface and sets the user in the front row as the person to be authenticated; and a gate control unit that controls a gate based on the authentication result of the person to be authenticated, wherein the setting unit sets an error when multiple users' faces are shown in the upper and lower images in areas excluding the common area.

[0009] According to a second aspect of the present invention, there is provided a method for controlling a gate device, in which an upper image is acquired from an upper camera, a lower image having a common area that is partially common to the upper image is acquired from a lower camera installed vertically below the upper camera, and if multiple users are captured in the upper and lower images, a user in the front row is identified by calculating the distance between each of the multiple users and a surface on which the cameras are installed and the user in the front row is set as a person to be authenticated, and if multiple users' faces are captured in areas of the upper and lower images excluding the common area, an error is set, and the gate is controlled based on the authentication result of the person to be authenticated.

[0010] According to a third aspect of the present invention, there is provided a program for causing a computer mounted on a gate device to execute the following processes: acquiring an upper image from an upper camera, and acquiring a lower image having a common area that is partially common to the upper image from a lower camera installed vertically below the upper camera; if multiple users are captured in the upper and lower images, identifying a user in the front row by calculating the distance between each of the multiple users and a surface on which the cameras are installed and setting the user in the front row as a person to be authenticated; setting an error if multiple user faces are captured in the upper and lower images in areas excluding the common area; and controlling a gate based on the authentication result of the person to be authenticated. Effect of the Invention

[0011] According to each aspect of the present invention, a gate device, an authentication system, a control method for a gate device, and a storage medium are provided that contribute to accurately determining a user to be authenticated. Note that the effects of the present invention are not limited to those described above. The present invention may achieve other effects instead of or in addition to the effects described above. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining an overview of an embodiment. [Diagram 2]1 is a diagram illustrating an example of a schematic configuration of an authentication system according to a first embodiment. [Diagram 3] 1 is a diagram showing an example of an external appearance of a gate apparatus according to a first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a plan view of the gate apparatus according to the first embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of a processing configuration of a gate device according to the first embodiment. [Figure 6] 5 is a diagram for explaining an operation of a biological information acquiring unit according to the first embodiment. FIG. [Figure 7] 10 is a flowchart showing an example of an operation of an authentication target setting unit according to the first embodiment. [Figure 8] FIG. 4 is a diagram for explaining the operation of an authentication target setting unit according to the first embodiment. [Figure 9] FIG. 4 is a diagram for explaining the operation of an authentication target setting unit according to the first embodiment. [Figure 10] FIG. 4 is a diagram for explaining the operation of an authentication target setting unit according to the first embodiment. [Figure 11] 11 is a flowchart showing an example of a context specifying process performed by the authentication target setting unit according to the first embodiment. [Figure 12] FIG. 4 is a diagram for explaining the operation of an authentication target setting unit according to the first embodiment. [Figure 13] FIG. 4 is a diagram for explaining the operation of an authentication target setting unit according to the first embodiment. [Figure 14] 11 is a diagram showing an example of a graph illustrating a correspondence relationship between the distance between the camera installation surface and the object and the position of the object in the common area. FIG. [Figure 15] FIG. 11 illustrates an example of an authentication request. [Figure 16] 5 is a flowchart showing an example of the operation of the gate device according to the first embodiment. [Figure 17] FIG. 2 is a diagram illustrating an example of a processing configuration of a server device according to the first embodiment. [Figure 18] FIG. 4 is a diagram illustrating an example of a user database. [Figure 19]FIG. 4 is a sequence diagram showing an example of the operation of the authentication system according to the first embodiment. [Figure 20] 11 is a diagram for explaining the relationship between the actual position of a user and the position of the user shown in an image. [Figure 21] 13 is a flowchart showing an example of an operation of an authentication target setting unit according to the second embodiment. [Figure 22] FIG. 2 is a diagram illustrating an example of a hardware configuration of a gate device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] First, an outline of one embodiment will be described. The reference numerals in the drawings are added to each element for convenience as an example to aid in understanding, and the description of the outline is not intended to be limiting. Furthermore, unless otherwise specified, the blocks shown in each drawing represent functional units rather than hardware units. The connection lines between the blocks in each drawing include both bidirectional and unidirectional. The unidirectional arrows are used to diagrammatically indicate the flow of the main signal (data) and do not exclude bidirectionality. In this specification and the drawings, elements that can be similarly described may be given the same reference numerals to avoid redundant description.

[0014] A gate device 100 according to an embodiment includes an acquisition unit 101, a setting unit 102, and a gate control unit 103 (see FIG. 1). The acquisition unit 101 acquires an upper image and a lower image from an upper camera and a lower camera installed vertically below the upper camera, respectively. When multiple users are captured in both the upper image and the lower image, the setting unit 102 calculates the distance between each of the multiple users and the surface on which the cameras are installed to identify the user in the front row from among the multiple users, and sets the identified user as the person to be authenticated. The gate control unit 103 controls the gate based on the authentication result of the person to be authenticated.

[0015] The gate device 100 acquires images from two cameras (an upper camera and a lower camera). When multiple users are captured in an image, the gate device 100 identifies the person to be authenticated. For example, the gate device 100 calculates the distance between each user and the surface on which the cameras are installed, using the parallax of an object (a user who appears in both images) in the upper and lower images. The gate device 100 identifies the user closest to the device based on the calculated distance, and sets the identified user as the person to be authenticated. In this way, the gate device 100 identifies the user in the front row regardless of the size of the face area, which varies greatly from person to person, and can therefore accurately determine the user to be authenticated.

[0016] Specific embodiments will be described in more detail below with reference to the drawings.

[0017] [First embodiment] The first embodiment will be described in more detail with reference to the drawings.

[0018] 2 is a diagram showing an example of a schematic configuration of the authentication system according to the first embodiment. Referring to FIG. 2, the authentication system includes a plurality of gate devices 10-1 to 10-3 and a server device 20. As shown in FIG.

[0019] In the following description, unless there is a particular reason to distinguish between the gate devices 10-1 to 10-3, they will simply be referred to as "gate device 10." Similarly, for other components, the left-hand symbols separated by hyphens will be used to represent the components.

[0020] 2 is merely an example and is not intended to limit the number of gate devices 10. The authentication system may include at least one gate device 10.

[0021] The gate device 10 and the server device 20 are configured to be able to communicate with each other via wired or wireless communication means. The server device 20 may be installed in the same building as the gate device 10, or may be installed on a network (cloud).

[0022] The gate device 10 is a device installed in, for example, an airport or a station. The gate device 10 controls the passage of users. The gate device 10 allows the passage of users who have been successfully authenticated by the server device 20 (hereinafter, referred to as "authenticated successfully"). The gate device 10 does not allow the passage of users who have not been authenticated by the server device 20 or users who have failed authentication.

[0023] The server device 20 is a device that performs authentication processing. Specifically, the server device 20 stores biometric information of users (system users) who are allowed to pass through the gate device 10. The server device 20 executes authentication processing using the biometric information acquired from the gate device 10 and pre-stored biometric information, and notifies the gate device 10 of the authentication result (authentication success, authentication failure).

[0024] When a person who has been successfully authenticated enters the internal area of ​​the gate device 10 (the area surrounded by the main body of the gate device 10), the gate device 10 opens the gate and allows the person who has been successfully authenticated to pass through.

[0025] Examples of the biometric information of a user include data (feature amounts) calculated from physical characteristics unique to an individual, such as the pattern of a face or iris. Alternatively, the biometric information of a user may be image data such as a face image or iris image. The biometric information of a user may be any information that includes the physical characteristics of the user.

[0026] Next, a detailed description will be given of each device included in the authentication system according to the first embodiment. In the following description, a "face image" of a user will be taken as an example of biometric information.

[0027] [Gate device] FIG. 3 is a diagram showing an example of the external appearance of the gate apparatus 10 according to the first embodiment.

[0028] 3, the gate device 10 includes a face recognition module 11. The face recognition module 11 is attached to the main body of the gate device 10. The face recognition module 11 includes two cameras (upper camera 12-1 and lower camera 12-2). Note that the "main body" of the gate device 10 is a structure forming the center of the gate device 10, which is in contact with the floor and to which the face recognition module and the like are attached.

[0029] The face recognition module 11 is installed at a predetermined angle with respect to the direction in which a user walks toward the gate device 10. Fig. 4 is a diagram (plan view) showing the gate device 10 as viewed from above.

[0030] 4, the face recognition module 11 is installed at an angle θ with respect to the moving direction of the user (the direction perpendicular to the gate 17). The angle θ is selected (adjusted) so that the upper camera 12-1 and the lower camera 12-2 can capture an image of the user U in the authentication area.

[0031] If the angle θ is made small, only the vicinity of the gate device 10 can be imaged, and there is a possibility that the user will enter the inside (internal area) of the gate device 10 before the authentication process using the face image is completed. In this case, the user needs to stop and wait until the authentication process is completed, and it cannot be said that a comfortable walk-through is realized. On the other hand, if the angle θ is made large, it is possible to image the far side of the gate device 10. In this case, it is advantageous to realize a comfortable walk-through, but there is a possibility that even users who are not walking toward the gate device itself will be authenticated. For example, in the example of FIG. 2, the gate device 10-1 images a user who is not walking toward the gate device 10-1 and requests authentication from the server device 20. In this way, if the angle θ is too large, there is a possibility that a useless authentication process will occur. It is desirable for the system administrator or operator to determine the optimal angle by considering the advantages and disadvantages caused by the setting angle θ of the face recognition module 11 as described above. Note that, as a result of the inventors' earnest study, it was found that the angle θ is desirably selected from the range of 30 to 60 degrees.

[0032] 3 and 4 show an example in which the face recognition module 11 is installed on the right side of the user's travel direction, but the installation position of the face recognition module 11 is not limited to the right side. The face recognition module 11 may be installed on the left side of the user's travel direction. Although an example in which the face recognition module 11 is installed directly on the main body of the gate device 10 is shown in FIG. 3 and FIG. 4, the face recognition module 11 may be installed away from the main body of the gate device 10. Alternatively, if the gate device 10 has a ceiling portion (roof; not shown) or the like, the face recognition module 11 may be installed on the ceiling portion. The face recognition module 11 may be installed at a predetermined angle with respect to the user's travel direction.

[0033] The upper camera 12-1 is attached to the vertically upper side of the face recognition module 11. The lower camera 12-2 is attached to the vertically lower side of the face recognition module 11. That is, as shown in Fig. 3, the lower camera 12-2 is installed vertically lower than the upper camera 12-1.

[0034] On the surface of the face recognition module (the surface on which the cameras are installed), the vertical positions of the two cameras are different as described above, but they are installed at the same position in the horizontal direction (horizontal position). Similarly, in terms of the depth direction of the two cameras, the two cameras are installed on the same surface (the surface on which the cameras are installed). Note that "the horizontal positions of the two cameras are the same" does not mean that the horizontal positions of the two cameras exactly match, but rather allows for some deviation. In other words, it is sufficient that the horizontal positions of the two cameras are substantially the same.

[0035] The upper camera 12-1 and the lower camera 12-2 are cameras having substantially the same performance, and parameters such as the angle of view and focal length are substantially the same. Due to the above-mentioned method of installing the two cameras, the images obtained from the two cameras have different shooting ranges in the vertical direction, but the shooting ranges in the horizontal direction are substantially the same.

[0036] The two cameras (upper camera 12-1, lower camera 12-2) of the gate device 10 are provided to thoroughly authenticate people (subjects) ranging from short to tall. The upper camera 12-1 is installed to obtain facial images of tall subjects to be authenticated. The lower camera 12-2 is installed to obtain facial images of short subjects to be authenticated.

[0037] With these cameras, facial images of tall authentication subjects are captured by the upper camera 12-1, and facial images of short authentication subjects are captured by the lower camera 12-2. Facial images of authentication subjects of medium height are captured by both the upper camera 12-1 and the lower camera 12-2. Most users are classified as medium height, and it is rare for facial images to be captured by either the upper camera 12-1 or the lower camera 12-2.

[0038] The upper camera 12-1 and the lower camera 12-2 operate in conjunction with each other. Specifically, the two cameras capture images of the user at substantially the same time.

[0039] 3, the face recognition module 11 includes a dimming LED (Light Emitting Diode) (not shown) and a display 13. The gate device 10 uses the dimming LED to control the environment (light irradiated to the user) when acquiring a face image of the user. The gate device 10 uses the display 13 to notify the user of necessary messages and the like. For example, the gate device 10 displays the authentication result (authentication success, authentication failure) by the server device 20.

[0040] The gate device 10 includes a display 14. The gate device 10 uses the display 14 to notify the user of necessary messages and the like. For example, the gate device 10 displays the result of authentication by the server device 20 (authentication success, authentication failure). Alternatively, the gate device 10 may notify the user of necessary messages using a speaker (not shown).

[0041] The gate device 10 includes an area sensor 15. The area sensor 15 is a sensor for detecting whether or not a person is present in a predetermined area (an area surrounded by a dotted line in FIG. 3) set in front of the gate device 10. The area sensor 15 can be configured using a sensor that uses infrared rays (a so-called human presence sensor).

[0042] When the gate device 10 confirms that a person is present in the above-mentioned predetermined area based on a detection signal from the area sensor 15, it controls the two cameras (upper camera 12-1, lower camera 12-2) to obtain two images (image data). In the following description, the area in which the area sensor 15 detects the presence of a person is referred to as an "authentication area." When the gate device 10 detects that a user is present in the authentication area, it captures an image of the user in the authentication area.

[0043] The gate device 10 includes an in-gate sensor 16. The in-gate sensor 16 is a sensor for detecting that a person has entered the inside of the device (gate device 10). For example, a sensor composed of an optical transmitting device and an optical receiving device (so-called a passing sensor using light) can be used as the in-gate sensor 16. For example, the optical transmitting device and the optical receiving device are installed so as to face each other (the two devices are installed on the inner wall of the main body). The transmitting device constantly transmits light, and the receiving device receives the transmitted light. When the receiving device cannot receive light, the gate device 10 determines that a person has entered the inside of the gate device 10. Note that FIG. 3 illustrates one of the two devices constituting the in-gate sensor 16. A plurality of pairs of the in-gate sensors 16 may be installed at approximately equal intervals in the direction of travel of the person.

[0044] The gate device 10 includes a gate 17. When the gate device 10 confirms that a user is present in the authentication area, it acquires a facial image of the user. The gate device 10 generates features (biometric information) from the acquired image, and transmits an authentication request including the generated features to the server device 20. The gate device 10 opens the gate 17 when a person who has been successfully authenticated enters the internal area of ​​the gate device 10. The gate device 10 closes the gate 17 after the person who has been successfully authenticated has passed through the gate 17.

[0045] The type of gate 17 is not particularly limited, and may be, for example, a flapper gate in which a flapper installed on one or both sides of the passage opens and closes, or a turnstile gate in which three bars rotate.

[0046] Fig. 5 is a diagram showing an example of a processing configuration (processing module) of the gate device 10 according to the first embodiment. Referring to Fig. 5, the gate device 10 includes a communication control unit 201, a user detection unit 202, a biometric information acquisition unit 203, an authentication target setting unit 204, an authentication request unit 205, an intruder determination unit 206, a gate control unit 207, a message output unit 208, and a storage unit 209.

[0047] The communication control unit 201 is a means for controlling communication with other devices. Specifically, the communication control unit 201 receives data (packets) from the server device 20. The communication control unit 201 also transmits data to the server device 20. The communication control unit 201 passes the data received from the other devices to the other processing modules. The communication control unit 201 transmits the data acquired from the other processing modules to the other devices. In this way, the other processing modules transmit and receive data to and from the other devices via the communication control unit 201.

[0048] The user detection unit 202 is a means for detecting a user present in the authentication area. The user detection unit 202 detects a user present in the authentication area based on a detection signal from the area sensor 15. When the output of the area sensor 15 indicates that “a user is present in the authentication area”, the user detection unit 202 notifies the biometric information acquisition unit 203 of that effect.

[0049] The biometric information acquisition unit 203 is a means for acquiring biometric information of a user present in the authentication area. When the biometric information acquisition unit 203 acquires a notification related to "a user is present in the authentication area" from the user detection unit 202, it controls two cameras (upper camera 12-1 and lower camera 12-2) to capture an image of the user present in the authentication area.

[0050] The biometric information acquisition unit 203 acquires images from each of the two cameras. As described above, the upper camera 12-1 is attached to the upper side of the face recognition module 11. Therefore, the image acquired from the upper camera 12-1 (hereinafter referred to as the upper image) includes the upper side of the authentication area. For example, if a user of medium height is positioned in the authentication area, the upper image surrounded by the dashed line in FIG. 6(a) is acquired.

[0051] On the other hand, the lower camera 12-2 is attached to the lower side of the face recognition module 11. Therefore, the image obtained from the lower camera 12-2 (hereinafter referred to as the lower image) includes the lower side of the authentication area. For example, when a user of medium height is positioned in the authentication area, the lower image surrounded by the dashed line in FIG. 6(b) is obtained.

[0052] The upper and lower images include a common area. In the example of Fig. 6, the area indicated by the dashed line in Fig. 6(c) is the common area between the upper and lower images. In the following description, the common area between the upper and lower images is referred to as the "common area."

[0053] The biometric information acquisition unit 203 passes the two images (upper image and lower image) captured by the upper camera 12-1 and the lower camera 12-2 to the authentication target setting unit 204.

[0054] The authentication target setting unit 204 is a means for setting (determining) a user to be authenticated (an individual to be authenticated) from among users appearing in an image. When multiple users exist in an authentication area, the authentication target setting unit 204 selects one user from the multiple users and sets the user as the "an individual to be authenticated." The operation of the authentication target setting unit 204 will be described with reference to the flowchart shown in FIG. 7.

[0055] The authentication target setting unit 204 extracts a face image (step S101). The authentication target setting unit 204 extracts a face image from the upper image. The authentication target setting unit 204 extracts a face image from the lower image. The authentication target setting unit 204 extracts a face image from an image using various methods. For example, the authentication target setting unit 204 extracts a face image (face area) from image data using a learning model learned by a CNN (Convolutional Neural Network). Alternatively, the authentication target setting unit 204 may extract a face image using a method such as template matching.

[0056] The authentication target setting unit 204 judges the distribution state of face images in the two images (step S102). The authentication target setting unit 204 judges whether face images are included only in the upper image or the lower image, or whether face images are included in both the upper image and the lower image. In the former case, the processing from step S103 onwards is executed. In the latter case, the processing from step S110 onwards is executed.

[0057] When the above step S102 is executed and it is found that face images are distributed in the upper image or the lower image, the authentication target setting unit 204 determines whether or not there is one face image contained only in the upper image or the lower image (step S103).

[0058] If one face image is included in the upper image or the lower image (step S103, Yes branch), the authentication target setting unit 204 sets the face image as an authentication target (step S104).

[0059] For example, as shown in Fig. 8(a), the face image of a tall person to be authenticated is included only in the upper image. Also, as shown in Fig. 8(b), the face image of a short person to be authenticated is included only in the lower image. In a case such as that shown in Fig. 8, the authentication target setting unit 204 sets the face image included in the upper image and the face image included in the lower image as the authentication target (the face image of the person to be authenticated). Note that in the drawings including Fig. 8, the range of each image is shifted horizontally to make it easier to distinguish between the upper image and the lower image.

[0060] If two or more face images are included in the upper image or the lower image (step S103, No branch), the authentication target setting unit 204 sets an error (step S105).

[0061] For example, as shown in Fig. 9(a) or 9(b), an error is set when multiple face images are included only in the upper image or the lower image. This is because in a case such as that shown in Fig. 9, it is difficult to determine which of the multiple face images to set as the authentication target. However, the situation shown in Fig. 9 rarely occurs. When such a situation occurs, the gate device 10 takes action such as instructing the user to leave the authentication area.

[0062] As described above, when face images are distributed in both the upper and lower images, the process from step S110 onwards shown in FIG. 7 is executed.

[0063] The authentication target setting unit 204 determines whether or not one face image is included in each of the upper and lower images (step S110).

[0064] Specifically, the authentication target setting unit 204 judges whether a situation as shown in FIG. 10(a) or a situation as shown in FIG. 10(b) has occurred.

[0065] When one face image is included in each of the upper and lower images (step S110, Yes branch), the authentication target setting unit 204 sets the face image included in either the upper or lower image as the authentication target (step S111). In the example of Fig. 10(a), the face image extracted from either the upper or lower image is set as the authentication target.

[0066] If two or more face images are included in each of the upper and lower images (step S110, No branch), the authentication target setting unit 204 identifies the front-back relationship of each user (step S112). In the example of Fig. 10(b), it is identified (determined) which of user U1 and user U2 is walking in front. Details of the front-back relationship identification process will be described later.

[0067] The authentication target setting unit 204 sets the user in the front row (frontmost) of the multiple users (multiple users present in the authentication area) as the authentication target (step S113). In the example of Fig. 10(b), when user U1 is walking in front of user U2 (when user U1 is closer to the gate device 10 than user U2), the face image of user U1 is set as the authentication target.

[0068] In this way, when one user appears in both the upper image and the lower image, the authentication target setting unit 204 sets the one user as a person to be authenticated. Also, when one user appears in either the upper image or the lower image, the authentication target setting unit 204 sets the one user as a person to be authenticated.

[0069] It is preferable that the authentication target setting unit 204 checks whether the users appearing in the upper and lower images are the same person before setting the authentication target in step S111. Specifically, the authentication target setting unit 204 calculates feature amounts from the two face images. The authentication target setting unit 204 judges whether the two feature amounts substantially match. The authentication target setting unit 204 calculates a similarity between the two feature amounts (feature vectors). If the calculated similarity is equal to or greater than a predetermined value, the authentication target setting unit 204 judges that the two feature amounts substantially match. That is, the authentication target setting unit 204 judges whether the two face images are face images of the same person by one-to-one matching using feature amounts generated from the upper and lower face images, respectively. If the two feature amounts do not match, the authentication target setting unit 204 may set an error. That is, a situation in which one face image is included in each of the upper and lower images and the respective face images are not face images of the same person does not usually occur. Therefore, the status should be set to error.

[0070] Furthermore, if there is one tall and one short person to be authenticated in the authentication area, the process in step S112 is executed. However, in this case, since there is no common area between the upper image and the lower image, the context cannot be determined in step S112. Therefore, when the above situation occurs, the authentication target setting unit 204 sets an error.

[0071] Next, a description will be given of the process (processing for identifying an antecedent relationship) related to step S112 in Fig. 7. Fig. 11 is a flowchart showing an example of the antecedent relationship identifying process of the authentication target setting unit 204.

[0072] The authentication target setting unit 204 sets a "distance measurement zone" in the common area of ​​two images (upper image, lower image) (step S201). For example, when the upper image and lower image shown in FIG. 12 are obtained, the area colored gray is set as the "distance measurement zone". As shown in FIG. 12, it is preferable to set the distance measurement zone at the bottom of the upper image. The distance measurement zone is set in an area in the common area where two or more users are simultaneously captured. In an extreme example, the distance measurement zone may be set at the bottom of the upper image, or the entire common area may be set as the distance measurement zone. The authentication target setting unit 204 sets all or part of the upper image and the lower image as the distance measurement zone.

[0073] The authentication target setting unit 204 calculates the distance to the camera installation surface for the object captured in the distance measurement zone (all or part of the common area common to the upper image and the lower image) (step S202). Specifically, the authentication target setting unit 204 calculates the distance to the camera (upper camera 12-1, lower camera 12-2) installation surface for each small area included in the distance measurement zone. The authentication target setting unit 204 uses the parallax between the two images (deviation between the upper image and the lower image) to calculate the distance from each small area (object) to the camera installation surface.

[0074] The authentication target setting unit 204 calculates the amount of deviation (parallax) between the upper image and the lower image of the object included in the distance measurement band. The authentication target setting unit 204 calculates the distance between the object and the camera installation surface using the calculated parallax. That is, the authentication target setting unit 204 calculates the distance between the camera installation surface and the object (small area of ​​the distance measurement band) using a so-called stereo method. Although the calculation of distance using the stereo method is clear to those skilled in the art and a detailed explanation is omitted, the authentication target setting unit 204 calculates the distance between the camera installation surface and the object using the following method.

[0075] The authentication target setting unit 204 divides the distance measurement zone into a plurality of small regions. For example, the authentication target setting unit 204 divides the distance measurement zone so that the size of each region is A1×A2 pixels. The authentication target setting unit 204 calculates the distance D between the small region (object) and the camera installation surface by the following formula (1).

[0076] [Number 1] D=B×f / Z (1)

[0077] In formula (1), B is the distance (camera interval) between upper camera 12-1 and lower camera 12-2. f is the focal length of the two cameras. Z is the parallax (amount of displacement). The relationship between the parameters shown in formula (1) is illustrated in FIG. 13.

[0078] In formula (1), the camera interval B and the focal length f are values ​​that are determined in advance according to the installation and selection of the two cameras. Therefore, the authentication target setting unit 204 can calculate the distance D by obtaining the parallax Z.

[0079] The authentication target setting unit 204 searches for an area whose pattern matches that of the small areas constituting the common area zone of the reference camera (upper camera 12-1) for one small area in the image (distance measurement zone) of the base camera (lower camera 12-2 in FIG. 13). For example, the authentication target setting unit 204 calculates the city block distance for each small area, and selects the area with the shortest distance as the area whose pattern matches. The authentication target setting unit 204 calculates the deviation between the small area on the base camera side and the small area on the reference camera side selected as described above as the parallax Z.

[0080] For example, if you calculate the distance D between an object included in the distance measurement zone shown in Figure 12 and the camera installation surface, and set the position of the object (horizontal position) on the X-axis and the calculated distance D on the Y-axis, you will obtain a graph like the one shown in Figure 14.

[0081] It is preferable that the authentication target setting unit 204 performs a predetermined data processing or the like on the calculated distance D in order to obtain a waveform as shown in Fig. 14. For example, the authentication target setting unit 204 may apply a low pass filter (LPF) to the calculated distance D in order to remove high frequency components (high frequency components of the distance D) caused by noise contained in the image. Alternatively, when the variation of the distance D falls within a predetermined range, the authentication target setting unit 204 may execute a process of converging each value of the distance D to a predetermined value (for example, an average value of the variation values).

[0082] As shown in Fig. 14, when multiple users are captured by upper camera 12-1 and lower camera 12-2, a peak appears according to the number of people captured in the two images. In the examples of Fig. 12 and Fig. 14, two peaks appear.

[0083] The authentication target setting unit 204 identifies the position (X coordinate) where each peak appears (step S203). In the example of Fig. 14, coordinates X1 and X2 are identified. That is, the authentication target setting unit 204 identifies (searches for) a peak corresponding to the number of multiple users in the position of the object (the position of the object in the image) and the distance of the object to the camera installation surface (for example, the graph of Fig. 14 showing the relationship between the position and distance).

[0084] The authentication target setting unit 204 uses an arbitrary method to identify the peak of the distance D. For example, the authentication target setting unit 204 calculates the difference between data related to the distance D, and sets the point where the calculated difference changes from positive to negative as the peak. It is preferable that the authentication target setting unit 204 identifies the peak from the data after applying the above-mentioned filtering process and convergence process.

[0085] The authentication target setting unit 204 selects the X coordinate corresponding to the smallest peak value (step S204). The user (face image) located vertically above the selected X coordinate is identified as the person in the front row of the authentication area. In the examples of Figs. 12 and 14, user U1 located above coordinate X1 is identified as the user in the front row. That is, the authentication target setting unit 204 treats the user at the position corresponding to the smallest peak value among the identified multiple peaks as the user in the front row.

[0086] The authentication target setting unit 204 sets the face image of the user in the front row as the authentication target (step S113 in FIG. 7). In this way, when multiple users appear in both the upper image and the lower image, the authentication target setting unit 204 identifies the user in the front row from among the multiple users by calculating the distance between each of the multiple users and the camera installation surface. The authentication target setting unit 204 sets the identified user as the authentication target.

[0087] The authentication target setting unit 204 delivers a face image (biometric information) of the person to be authenticated to the authentication request unit 205 .

[0088] The authentication request unit 205 is a means for requesting the server device 20 to authenticate the person to be authenticated. The authentication request unit 205 generates a feature amount (a feature vector consisting of a plurality of feature amounts) from the face image acquired from the biometric information acquisition unit 203. The authentication request unit 205 transmits an authentication request including the generated feature amount (biometric information) to the server device 20.

[0089] Since existing technology can be used for the feature generation process by the authentication request unit 205, detailed description thereof will be omitted. For example, the authentication request unit 205 extracts the eyes, nose, mouth, etc. as feature points from the face image. After that, the authentication request unit 205 calculates the position of each feature point and the distance between each feature point as feature amounts, and generates a feature vector (vector information that characterizes the face image) consisting of multiple feature amounts.

[0090] The authentication request unit 205 generates an authentication request including an identifier of the own device (hereinafter, referred to as a gate identifier), features, etc. (see FIG. 15). Note that the gate identifier may be a MAC (Media Access Control) address or an IP (Internet Protocol) address of the gate device 10.

[0091] When transmitting an authentication request to the server device 20 , the authentication request unit 205 passes the feature amount included in the request to the intruder determination unit 206 .

[0092] The authentication request unit 205 receives a response to the authentication request from the server device 20. The authentication request unit 205 passes the response from the server device 20 (authentication success, authentication failure) to the gate control unit 207.

[0093] The intruder determination unit 206 is a means for detecting an intruder using the in-gate sensor 16 for detecting an intruder into its own device (gate device 10) and for determining whether the intruder and the person to be authenticated are the same person. The intruder determination unit 206 monitors (polls) the output of the in-gate sensor 16. When the output of the in-gate sensor 16 indicates that "there is an intruder inside," the intruder determination unit 206 determines whether the intruder and the person to be authenticated match. Specifically, the intruder determination unit 206 controls at least one camera to acquire a face image, similar to the biometric information acquisition unit 203. The intruder determination unit 206 calculates a feature amount (feature amount of the intruder) from the acquired face image.

[0094] The intruder determination unit 206 determines whether or not the calculated feature amount (the feature amount of the intruder) and the feature amount acquired from the authentication request unit 205 (the feature amount of the person to be authenticated) substantially match. Specifically, the intruder determination unit 206 calculates the similarity between the two feature amounts. The similarity can be calculated using a chi-square distance, Euclidean distance, or the like. Note that the greater the distance, the lower the similarity, and the closer the distance, the higher the similarity. When the similarity is equal to or greater than a predetermined value, the intruder determination unit 206 determines that the two feature amounts substantially match (the intruder and the person to be authenticated are the same person).

[0095] In this way, the intruder determination unit 206 detects an intruder based on a detection signal from a passage sensor (sensor 16 inside the gate) that uses light. After that, the intruder determination unit 206 performs one-to-one matching using two feature amounts (feature amount of the intruder and feature amount of the person to be authenticated). By the one-to-one matching (one-to-one matching using biometric information of the intruder and biometric information of the person to be authenticated), it is determined whether the intruder and the person to be authenticated are the same person.

[0096] If the intruder determination unit 206 determines that the intruder and the person to be authenticated are the same person, it notifies the gate control unit 207 to that effect.

[0097] The gate control unit 207 is a means for controlling the gate 17 provided in the gate device 10. The gate control unit 207 controls the gate 17 based on the authentication result of the person to be authenticated. The gate control unit 207 opens the gate 17 when the authentication result of the server device 20 is "authentication successful" and the entrant and the person to be authenticated are the same person. In other words, the gate control unit 207 will not open the gate 17 in principle unless the above conditions (authentication successful, the entrant and the person to be authenticated are the same person) are met. The gate control unit 207 closes the gate 17 after a user who has been permitted to pass using a distance sensor or the like has passed through the gate 17.

[0098] The message output unit 208 is a means for outputting a message to notify a user (a person to be authenticated, a person who has been successfully authenticated, etc.) The message output unit 208 notifies the user of a necessary message using the displays 13 and 14, a speaker, etc.

[0099] The storage unit 209 is a means for storing information necessary for the operation of the gate device 10.

[0100] FIG. 16 is a flowchart showing an example of the operation of the gate apparatus 10 according to the first embodiment.

[0101] The gate apparatus 10 detects whether or not a user is present in the authentication area (step S301). If a user is not detected in the authentication area (step S301, No branch), the gate apparatus 10 repeats the process of step S301.

[0102] When a user is detected (step S301, Yes branch), the gate apparatus 10 captures an image of the user located in the authentication area and obtains a face image (step S302).

[0103] The gate device 10 calculates the feature amount from the face image (step S303).

[0104] The gate device 10 transmits an authentication request including the calculated feature amount to the server device 20, and receives a response to the authentication request (authentication result) (step S304).

[0105] The gate device 10 judges whether or not an intruder is present in the internal area of ​​the gate device (step S305). If an intruder cannot be detected even after a predetermined time has elapsed since the intruder detection process was started (step S305, No branch), the gate device 10 executes the process of step S309.

[0106] When an intruder is detected during a predetermined period of time (step S305, Yes branch), the gate device 10 judges whether the intruder and the person to be authenticated are the same person (step S306). If the intruder and the person to be authenticated are not the same person (step S306, No branch), the gate device 10 executes the process related to step S309.

[0107] If the trespasser and the person to be authenticated are the same person (step S306, Yes branch), the gate device 10 judges whether the authentication result obtained from the server device 20 is "authentication successful" or not (step S307). If the authentication result is "authentication unsuccessful", the gate device 10 executes the process related to step S309. It is to be noted that the process related to step S307 may be executed before the process related to step S305.

[0108] If the authentication result is "authentication successful", the gate device 10 opens the gate 17 (step S308). When the gate opening conditions (authentication successful and the trespasser and the person to be authenticated are the same person) are satisfied, the gate device 10 controls the gate 17 to allow the person to be authenticated to pass through. At that time, the gate device 10 may display on the displays 13 and 14 that the authentication of the person to be authenticated has been successful.

[0109] If the above conditions are not met (No branch in any of steps S305, S306, and S307), the gate device 10 executes error processing (step S309). Specifically, if an intruder cannot be detected within a predetermined period of time (No branch in step S305), the gate device 10 discards the authentication result obtained from the server device 20. If the intruder and the person to be authenticated are different (No branch in step S306), the gate device 10 displays that fact on the displays 13 and 14. If the authentication by the server device 20 has failed (No branch in step S307), the gate device 10 displays that fact on the displays 13 and 14.

[0110] In this way, the gate device 10 acquires biometric information (facial images) of users ranging from short to tall using two cameras (upper camera 12-1, lower camera 12-2) installed above and below. Furthermore, the gate device 10 uses the parallax of objects commonly captured in the images obtained from the two cameras to determine the front-to-back relationship when multiple users are imaged, and identifies the correct person to be authenticated. In other words, the gate device 10 uses the two cameras to acquire biometric information without omission from users of significantly different heights, and prevents erroneous identification of the person to be authenticated.

[0111] [Server device] 17 is a diagram illustrating an example of a processing configuration (processing module) of the server device 20 according to the first embodiment. Referring to FIG. 17, the server device 20 includes a communication control unit 301, a user registration unit 302, an authentication unit 303, and a storage unit 304.

[0112] The communication control unit 301 is a means for controlling communication with other devices. Specifically, the communication control unit 301 receives data (packets) from the gate device 10. The communication control unit 301 also transmits data to the gate device 10. The communication control unit 301 passes the data received from other devices to other processing modules. The communication control unit 301 transmits data acquired from other processing modules to other devices. In this way, the other processing modules transmit and receive data to and from other devices via the communication control unit 301.

[0113] The user registration unit 302 is a means for system-registering a user (system user) who is allowed to pass through the gate device 10. The user registration unit 302 acquires biometric information (e.g., face image) and personal information (e.g., name) of a user who is allowed to pass through the gate device 10 by any means.

[0114] For example, if the gate device 10 is a device installed at an airport, the system user uses an airline's WEB page or a kiosk terminal installed at the airport to input biometric information and personal information (such as name and passport number) to the server device 20. Alternatively, if the gate device 10 is a device installed at a ticket gate at a station, the system user inputs biometric information, personal information, etc. to the server device 20 from a railroad company's WEB page or a terminal installed at the station.

[0115] When a facial image is acquired, the user registration unit 302 calculates features from the facial image. The user registration unit 302 registers the biometric information of the system user (e.g., features calculated from the facial image) in a "user database." If necessary, the user registration unit 302 registers a user ID (identifier) ​​for identifying the system user and personal information (e.g., name, nationality, sex, etc.) in the user database in association with the biometric information (see FIG. 18). The user database shown in FIG. 18 is an example, and other items may be stored in association with the biometric information (features). For example, a "facial image" may be registered in the user database.

[0116] The authentication unit 303 is a means for processing an authentication request received from the gate device 10. Specifically, the authentication unit 303 sets the biometric information (feature amount) included in the authentication request as a matching target, and performs a matching process with the biometric information registered in the user database.

[0117] More specifically, the authentication unit 303 sets the feature extracted from the authentication request as the target for matching, and performs 1:N (N is a positive integer, the same applies below) matching with multiple feature registered in the user database.

[0118] The authentication unit 303 calculates the similarity between the feature (feature vector) to be matched and each of the multiple feature on the registration side. If a feature with a similarity equal to or greater than a predetermined value is registered in the user database, the authentication unit 303 sets the result to "authentication successful." If a feature with a similarity equal to or greater than a predetermined value is not registered in the user database, the authentication unit 303 sets the result to "authentication failed."

[0119] Furthermore, the authentication unit 303 identifies a feature having a similarity to the feature to be compared that is equal to or greater than a predetermined value and has the highest similarity among a plurality of feature amounts registered in the user database. The authentication unit 303 transmits a user ID and personal information corresponding to the identified feature amount to other modules and the gate device 10 as necessary. For example, if the gate device 10 is a device installed at an airport, the progress of procedures (passing check-in, passing security check, etc.) is managed using the information (name, passport number, etc.) identified by the authentication unit 303. For example, if the gate device 10 is a device installed at a ticket gate at a station, the above-mentioned identified information is used to perform a fee settlement process, etc.

[0120] The storage unit 304 stores various information necessary for the operation of the server device 20. In the storage unit 304, a user database is constructed.

[0121] Fig. 19 is a sequence diagram showing an example of the operation of the authentication system according to the first embodiment. Note that it is assumed that the system users have been registered in advance prior to the operation of Fig. 19.

[0122] The gate device 10 detects a user present in the authentication area (step S01). When the gate device 10 detects a person to be authenticated, the gate device 10 may take measures to allow the user to recognize that authentication processing is being performed. For example, the gate device 10 may display a facial image of the person to be authenticated on the displays 13 and 14. Alternatively, the gate device 10 may display a message or a symbol (e.g., an arrow, etc.) on the displays 13 and 14 to encourage the person to enter the internal area of ​​the gate device 10 from the authentication area.

[0123] The gate device 10 acquires biometric information of an authentication target (a user present in an authentication area) and transmits an authentication request including the biometric information to the server device 20 (step S02).

[0124] The server device 20 executes an authentication process using the biometric information included in the authentication request and the biometric information registered in the user database (step S11).

[0125] The server device 20 transmits the authentication result (authentication success, authentication failure) to the gate device 10 (step S12).

[0126] The gate apparatus 10 judges whether or not a person to be authenticated has entered in parallel with the authentication process of the server apparatus 20 (detection of an intruder; step S03).

[0127] The gate device 10 checks whether the conditions for opening the gate 17 (gate opening conditions) are met (step S04). Specifically, the gate device 10 checks whether the authentication is successful and whether the trespasser and the person to be authenticated are the same person.

[0128] If the gate opening condition is satisfied, the gate device 10 opens the gate 17 (step S05).

[0129] As described above, the authentication system according to the first embodiment identifies the person to be authenticated when multiple people (system users, candidates for persons to be authenticated) appear in images obtained from two cameras (upper camera 12-1, lower camera 12-2). In this case, the gate device 10 calculates the distance between each user and the surface on which the cameras are installed, using the parallax of the objects (users who appear in both images) in the two images. The gate device 10 identifies the user who is closest to the device based on the calculated distance, and sets the identified user as the person to be authenticated. In this way, in the first embodiment, the user in the front row is identified regardless of the size of the face area, which varies greatly from person to person, so that the user to be authenticated can be accurately determined.

[0130] [Second embodiment] Next, the second embodiment will be described in detail with reference to the drawings.

[0131] In the first embodiment, the distance between the common area of ​​the upper and lower images and the camera installation surface is calculated, and the front-to-back relationship of the user is determined based on the calculated distance. In the second embodiment, a case will be described in which the front-to-back relationship is determined based on the positional relationship of the user captured in the image.

[0132] The configuration of the authentication system according to the second embodiment can be the same as that of the first embodiment, and therefore the explanation corresponding to Fig. 2 will be omitted. In addition, the processing configurations of the gate device 10 and the server device 20 according to the second embodiment can also be the same as those of the first embodiment, and therefore the explanation thereof will be omitted.

[0133] In the first embodiment, the case where two cameras are used to capture an image of a user has been described. However, in the second embodiment, it is sufficient that an image of a user is captured using at least one camera. For example, when a tall and short user can be captured using one camera, two cameras are not necessary. Alternatively, two cameras are not necessary when authentication of tall and short users is not required or when an error during authentication is acceptable. It goes without saying that the gate device 10 according to the second embodiment may include two cameras. In the following description, a case where the gate device 10 according to the second embodiment includes two cameras will be described.

[0134] The following description will focus on the differences between the first and second embodiments.

[0135] As described above, the face recognition module 11 is installed at a predetermined angle with respect to the moving direction of the user. For example, the face recognition module 11 is installed at an angle of 45 degrees (θ shown in FIG. 4 is 45 degrees).

[0136] Since the face recognition module 11 is installed at an angle to the direction in which the user is moving, the position in which the user appears in the image changes depending on the user's position (the distance between the user and the gate device 10). Specifically, when the user is far away, the user appears on the side of the gate device 10 where the face recognition module 11 is installed (to the left in the example of FIG. 4). When the user then approaches the gate device 10, the position of the user (the position of the user appearing in the image) moves to the right (to the right; the positive X coordinate when the bottom left of the image is the origin).

[0137] For example, consider a case where a user moves from position P1 to position P2 to position P3 as shown in FIG. 20(a). In this case, the position of the user in the captured image moves from left to right as shown in FIG. 20(b). This fact indicates that a user on the right side of the image is closer to the gate device 10. In other words, when multiple users are captured in the same image, this indicates that the user located on the right side of the image is closer to the gate device 10 than the user located on the left side. The gate device 10 according to the second embodiment uses this fact to identify the anteroposterior relationship between multiple users.

[0138] The authentication target setting unit 204 of the gate apparatus 10 according to the second embodiment differs from that of the first embodiment in the process of step S112 shown in Fig. 7. When multiple users are captured in an image, the authentication target setting unit 204 identifies the user in the front row among the multiple users according to the positions of each of the multiple users in the image. The authentication target setting unit 204 sets the identified user as an authentication target.

[0139] 21 is a flowchart showing an example of the operation of the authentication target setting unit 204 according to the second embodiment. A context specification process of the authentication target setting unit will be described with reference to FIG.

[0140] The authentication target setting unit 204 calculates the X coordinate (horizontal coordinate) of each face image included in the image (step S401). Note that the calculation of the X coordinate may be performed using either the upper image or the lower image.

[0141] The authentication target setting unit 204 selects the rightmost X coordinate (step S402). The face image corresponding to the selected X coordinate becomes the authentication target.

[0142] The authentication target setting unit 204 may impose a predetermined condition on the target for identifying the context of multiple users (face images) included in the image. The authentication target setting unit 204 may set a face image that satisfies a predetermined condition as the target for identifying the context. Specifically, the authentication target setting unit 204 may exclude users among the multiple users whose eye distance is smaller than a predetermined value (e.g., 60 pixels) from the determination regarding the users in the front row. By setting such a condition, it is possible to exclude users who are located far from the gate device 10 but appear on the right edge of the image.

[0143] 3 and 4, the face recognition module 11 is installed on the right side of the user's traveling direction, so the user appearing on the right edge of the image is set as the person to be authenticated. If the face recognition module 11 is installed on the left side of the user's traveling direction, the user appearing on the left edge of the image is set as the person to be authenticated.

[0144] As described above, the gate device 10 according to the second embodiment identifies the user located at the end of the side determined by the user's traveling direction and the installation position of the camera as the user in the front row. As a result, like the first embodiment, the gate device 10 can accurately determine the user to be authenticated.

[0145] Next, the hardware of each device constituting the authentication system will be described. Fig. 22 is a diagram showing an example of the hardware configuration of the gate device 10. Note that the face authentication module 11, the camera 12, etc. have already been explained, so these elements are not shown in Fig. 22.

[0146] The gate device 10 includes a processor 311, a memory 312, an input / output interface 313, and a communication interface 314. The components such as the processor 311 are connected by an internal bus or the like and are configured to be able to communicate with each other.

[0147] However, the configuration shown in Fig. 22 is not intended to limit the hardware configuration of the gate device 10. The gate device 10 may include hardware not shown, and may not include the input / output interface 313 as necessary. Furthermore, the number of processors 311 and the like included in the gate device 10 is not intended to be limited to the example shown in Fig. 22, and for example, the gate device 10 may include multiple processors 311.

[0148] The processor 311 is, for example, a programmable device such as a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP). Alternatively, the processor 311 may be a device such as a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC). The processor 311 executes various programs including an operating system (OS).

[0149] The memory 312 is a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), etc. The memory 312 stores an OS program, application programs, and various data.

[0150] The input / output interface 313 is an interface for a display device or an input device (not shown). The display device can be the display 14. The input device can be a touch panel integrated with the display 14.

[0151] The communication interface 314 is a circuit, a module, etc. that communicates with other devices. For example, the communication interface 314 includes a network interface card (NIC).

[0152] The functions of the gate device 10 are realized by various processing modules. The processing modules are realized, for example, by the processor 311 executing a program stored in the memory 312. The program can be recorded in a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. That is, the present invention can also be embodied as a computer program product. The program can be downloaded via a network, or updated using a storage medium storing the program. The processing modules may also be realized by a semiconductor chip.

[0153] The server device 20 can be configured in the same manner as the gate device 10, and the basic hardware configuration is the same as that of the gate device 10, so a description thereof will be omitted.

[0154] The gate device 10 is equipped with a computer, and the functions of the gate device 10 can be realized by causing the computer to execute a program. The gate device 10 also executes a "gate device control method" by the program.

[0155] [Variations] The configuration, operation, and the like of the authentication system described in the above embodiment are merely examples, and are not intended to limit the system configuration, and the like.

[0156] The application of the gate device 10 disclosed in the present application is not particularly limited. For example, the application (installation) of the gate device 10 disclosed in the present application is exemplified as an "airport". Specifically, the gate device 10 may be used as a part of an examination system using biometric authentication performed at an airport. In the examination system, identity verification (verification of whether a photographed face image matches a face image written on a passport) is performed at the time of check-in, and face data (face image or feature amount), flight information, and passport information are stored in association with each other. After identity verification is completed, a token is generated for examination using biometric authentication. The gate device 10 disclosed in the present application is installed in various examination areas (for example, security checkpoints) in the airport, and the gate device 10 authenticates a user by comparing a face image registered at the time of check-in with a face image captured by the gate device itself.

[0157] In the above embodiment, a case has been described in which an intruder is detected using the in-gate sensor 16. However, in cases where it is difficult for a user other than the person to be authenticated to enter the gate device 10, the detection of the intruder is not necessary. For example, by configuring the length of the gate device 10 body in the direction of travel of a person passing through the gate to be long and setting an authentication area inside the gate device 10, it becomes difficult for a user other than the person to be authenticated to enter the inside of the gate device 10.

[0158] In the above embodiment, the gate device 10 is described as having two cameras (upper camera 12-1, lower camera 12-2). However, the gate device 10 may have only one camera. In this case, the single camera is configured to be movable up and down, and the gate device 10 may obtain two images (upper image, lower image) by varying the position of the camera.

[0159] In the above embodiment, a case has been described in which a user is captured by two cameras (upper camera 12-1, lower camera 12-2) and the person to be authenticated (the user in the front row) is identified using the parallax obtained from the two images. However, the number of cameras is not limited to two, and the person to be authenticated may be identified using three or more cameras. For example, a middle camera is installed between the upper camera 12-1 and the lower camera 12-2. In this way, the gate device 10 can determine the front-to-back relationship of the users captured by the upper camera 12-1 and the middle camera, and the front-to-back relationship of the users captured by the middle camera and the lower camera 12-2.

[0160] In the above embodiment, the description has been given on the assumption that the performance of the upper camera 12-1 and the lower camera 12-2 is substantially the same. However, the performance of these cameras may be different. In this case, the gate device 10 may take measures to fill the performance difference between the two cameras. For example, the gate device 10 may convert the size, etc., of one of the two images obtained. In other words, the above-mentioned measures such as conversion processing can expand the selection of cameras that can be used, and the gate device disclosed in the present application can be realized even when there are restrictions on the installation location, etc. of the cameras in the face recognition module 11.

[0161] In the above embodiment, a case has been described in which biometric information related to feature amounts generated from a facial image is transmitted from the gate device 10 to the server device 20. However, the "facial image" itself may be transmitted from the gate device 10 to the server device 20 as biometric information. The server device 20 may generate feature amounts from the acquired facial image and execute authentication processing (1:N matching). Note that even when the facial image itself is transmitted to the server device 20, the gate device 10 determines whether the person to be authenticated and the entrant are the same person by using the feature amounts generated from the facial image.

[0162] In the above embodiment, the determination as to whether the person to be authenticated and the entrant are the same person is performed by the gate device 10, but the determination may be performed by the server device 20. In this case, the gate device 10 acquires biometric information of the person entrant into the internal area and transmits it to the server device 20. The server device 20 may compare the biometric information included in the authentication request with the biometric information of the entrant and determine whether the person to be authenticated and the entrant are the same person.

[0163] In the above embodiment, a so-called client-server system has been described in which the gate device 10 transmits an authentication request to the server device 20, and the server device 20 processes the request. However, the authentication process executed by the server device 20 may be executed by the gate device 10. In other words, a part or all of the functions of the server device 20 may be realized by the gate device 10.

[0164] In the above embodiment, it has been described that an error is set when a face image is distributed in either the upper image or the lower image and the image contains multiple face images (step S105 in FIG. 7). In this case, when the upper image contains multiple face images, the gate device 10 may determine the anteroposterior relationship using the parallax between the two images or the position of the user. For example, in the case of FIG. 9(a), the anteroposterior relationship may be determined using an object (such as the user's torso or legs) that appears in both the upper and lower images. Alternatively, the gate device 10 may determine the anteroposterior relationship according to the position of the face image included in the upper image, as described in the second embodiment.

[0165] 9(b), since there is no object (user) that appears in both the upper and lower images, it is not possible to determine the front-to-back relationship using parallax. Therefore, when a face image is present only in the lower image, the gate device 10 may determine the front-to-back relationship of the user according to the position of the user in the image (lower image).

[0166] Alternatively, the first and second embodiments may be combined, and in the case shown in FIG. 8(a) or FIG. 9(a), a person to be authenticated may be selected using parallax, and in the case shown in FIG. 9(b), a person to be authenticated may be selected according to the position of the image.

[0167] The gate device 10 may change the control of the dimming LED depending on the purpose of the image (face image; biometric information) to be acquired. For example, the gate device 10 may change the luminance when capturing an image of a user in an authentication area and when capturing an image of a person entering the gate device.

[0168] A plurality of gate devices 10 may operate in cooperation with each other. Specifically, the server device 20 may transmit a response to an authentication request acquired from one gate device 10 to the other gate devices 10. For example, in the example of FIG. 2, an authentication result in response to an authentication request acquired from the gate device 10-1 may be transmitted to each of the gate devices 10-1 to 10-3. In this case, even if the person to be authenticated detected by the gate device 10-1 heads toward (enters) the gate device 10-2, the person to be authenticated can pass through the gate device 10-2. In this case, the biometric information of the person to be authenticated generated by the gate device 10-1 is transmitted from the gate device 10-1 to the other gate devices 10-2 and 10-3. Alternatively, the biometric information of the person to be authenticated may be transmitted from the server device 20 to the other gate devices 10-2 and 10-3.

[0169] The form of data transmission and reception between the gate device 10 and the server device 20 is not particularly limited, but the data transmitted and received between these devices may be encrypted. Since the face image and the feature amount calculated from the face image are personal information, it is desirable to transmit and receive encrypted data in order to appropriately protect the personal information.

[0170] It is also assumed that the gate device 10 is installed in a limited space in an existing facility, for example. In this case, the length of the gate device 10 body in the moving direction of a person passing through the gate device 10 can be about 1.5 meters, which is shorter than the length of a typical face recognition gate, which is 2 to 3 meters, and the distance to the gate 17 can be, for example, 96 centimeters (equivalent to one and a half steps of an adult male). If the length of the gate device 10 body is about 1.5 meters, the length of the authentication area in the moving direction is preferably about 30 to 60 centimeters.

[0171] In the flow charts (flow charts, sequence diagrams) used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps executed in the embodiments is not limited to the order described. In the embodiments, the order of the steps shown in the figures can be changed to a degree that does not interfere with the content, such as executing each process in parallel.

[0172] The above-mentioned embodiments have been described in detail to facilitate understanding of the present disclosure, and it is not intended that all of the configurations described above are necessary. In addition, when a plurality of embodiments are described, each embodiment may be used alone or in combination. For example, it is possible to replace a part of the configuration of an embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of an embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of an embodiment with another configuration.

[0173] From the above explanation, it is clear that the present invention has industrial applicability, and the present invention can be suitably applied to authentication systems installed in airports, train stations, etc.

[0174] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. [Appendix 1] an acquisition unit that acquires an upper image and a lower image from an upper camera and a lower camera that is installed vertically below the upper camera, respectively; a setting unit that, when a plurality of users are shown in both the upper image and the lower image, identifies a user in a front row from among the plurality of users by calculating a distance between each of the plurality of users and a surface on which the camera is installed, and sets the identified user as a person to be authenticated; a gate control unit that controls a gate based on the authentication result of the person to be authenticated; A gate device comprising: [Appendix 2] The gate device according to claim 1, wherein the setting unit calculates a distance to the camera installation surface for an object appearing in a common area common to the upper image and the lower image, and identifies the user in the front row. [Appendix 3] The gate device described in Appendix 2, wherein the setting unit calculates the disparity between the upper image and the lower image of the object, and calculates the distance between the object and the camera installation surface using a stereo method that uses the calculated disparity. [Appendix 4] The gate device described in Appendix 2 or 3, wherein the setting unit identifies a peak corresponding to the number of the plurality of users in the relationship between the position of the object and the distance of the object to a camera installation surface, and identifies the user whose position corresponds to the smallest peak value among the plurality of identified peaks as the user in the front row. [Appendix 5] The gate device according to any one of claims 1 to 4, wherein the setting unit sets a single user as the person to be authenticated when the single user appears in both the upper image and the lower image. [Appendix 6] The gate device according to any one of claims 1 to 4, wherein the setting unit sets a user as the person to be authenticated when the user is shown in either the upper image or the lower image. [Appendix 7] 7. The gate device according to claim 1, further comprising an authentication request unit configured to transmit an authentication request including biometric information of the person to be authenticated to a server device. [Appendix 8] 8. The gate device according to claim 7, wherein the authentication request unit transmits the authentication request, which includes a feature amount generated from a face image of the person to be authenticated, to the server device. [Appendix 9] A gate device described in any one of appendix 1 to 8, wherein the upper camera and the lower camera are positioned the same in the horizontal direction on the camera installation surface. [Appendix 10] a server device that stores biometric information of each of a plurality of system users and executes authentication processing using the plurality of pieces of biometric information; a gate device connected to the server device; Including, The gate device includes: an acquisition unit that acquires an upper image and a lower image from an upper camera and a lower camera that is installed vertically below the upper camera, respectively; a setting unit that, when a plurality of users are shown in both the upper image and the lower image, identifies a user in a front row from among the plurality of users by calculating a distance between each of the plurality of users and a surface on which the camera is installed, and sets the identified user as a person to be authenticated; an authentication request unit that requests the server device to authenticate the person to be authenticated; a gate control unit that controls a gate based on the authentication result of the person to be authenticated; An authentication system comprising: [Appendix 11] the authentication request unit transmits an authentication request including biometric information of the person to be authenticated to the server device; 11. The authentication system according to claim 10, wherein the server device authenticates the person to be authenticated using biometric information acquired from the authentication request and biometric information of the multiple system users. [Appendix 12] In the gate device, acquiring an upper image and a lower image from an upper camera and a lower camera installed vertically below the upper camera, respectively; When multiple users are shown in both the upper image and the lower image, a user in the front row is identified from among the multiple users by calculating a distance between each of the multiple users and a surface on which the camera is installed, and the identified user is set as a person to be authenticated; A gate device control method for controlling a gate based on a result of authentication of the person to be authenticated. [Appendix 13] The computer installed in the gate device A process of acquiring an upper image and a lower image from an upper camera and a lower camera installed vertically below the upper camera, respectively; a process of, when a plurality of users are shown in both the upper image and the lower image, calculating a distance between each of the plurality of users and a surface on which the camera is installed, thereby identifying a user in the front row from among the plurality of users, and setting the identified user as a person to be authenticated; A process of controlling a gate based on a result of authentication of the person to be authenticated; A computer-readable storage medium that stores a program for executing the above.

[0175] The disclosures of the above cited prior art documents are incorporated herein by reference. Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. It will be understood by those skilled in the art that these embodiments are merely illustrative and that various modifications are possible without departing from the scope and spirit of the present invention. In other words, the present invention naturally includes various modifications and amendments that a person skilled in the art can make in accordance with the entire disclosure, including the scope of the claims, and the technical ideas. [Explanation of symbols]

[0176] 10, 10-1~10-3, 100 Gate device 11 Face Recognition Module 12-1 Upper camera 12-2 Lower camera 13. Display 14 Display 15 Area Sensor 16 Sensor inside the gate 17 Gate 20 Server device 101 Acquisition Department 102 Setting section 103, 207 Gate control section 201, 301 Communication control unit 202 User detection unit 203 Biometric information acquisition unit 204 Authentication target setting section 205 Authentication Request Section 206 Entrant Judgment Department 208 Message output section 209, 304 Storage section 302 User Registration Department 303 Authentication Department 311 Processor 312 Memory 313 Input / Output Interface 314 Communication Interface

Claims

1. an acquisition unit that acquires an upper image from an upper camera and acquires a lower image having a common area that is partially common to the upper image from a lower camera that is installed vertically below the upper camera; a setting unit that, when a plurality of users are captured in the common area of ​​the upper image and the lower image, identifies a user in a front row by calculating a distance between each of the plurality of users and a camera installation surface, and sets the user in the front row as a person to be authenticated; a gate control unit that controls a gate based on the authentication result of the person to be authenticated; Equipped with The setting unit sets an error when a plurality of user faces are shown in an area excluding the common area in the upper image or the lower image. Gate device.

2. The setting unit sets an error when a total of the number of user faces appearing in an area excluding the common area in the upper image and the number of user faces appearing in an area excluding the common area in the lower image is 2 or more.

2. The gate device according to claim 1.

3. The setting unit sets an error when a plurality of user faces are shown in an area of ​​the upper image excluding the common area.

2. The gate device according to claim 1.

4. The setting unit sets an error when a plurality of user faces are shown in an area of ​​the lower image excluding the common area.

2. The gate device according to claim 1.

5. When the faces of a plurality of users are captured in the common area, the setting unit calculates a distance between each of the plurality of users and the camera installation surface to identify the user in the front row and sets the user in the front row as a person to be authenticated.

2. The gate device according to claim 1.

6. The setting unit determines whether a user's face is captured in the common area, and when it is determined that the user's face is not captured in the common area, determines whether a plurality of user's faces are captured in an area excluding the common area in the upper image or the lower image.

6. The gate device according to claim 1, wherein the gate device is a gate unit.

7. When it is determined that the faces of multiple users are not shown in the area excluding the common area in the upper image or the lower image, the setting unit sets the user shown in the upper image or the lower image as an authentication target, and when it is determined that the faces of multiple users are shown in the area excluding the common area in the upper image or the lower image, sets an error.

7. The gate device according to claim 6.

8. When it is determined that a face of a user is captured in the common area, the setting unit determines the number of faces of users captured in the common area, and when it is determined that the faces of multiple users are captured in the common area, the setting unit calculates the distance between each of the multiple users and a camera installation surface to identify the user in the front row and sets the user in the front row as a person to be authenticated.

7. The gate device according to claim 6.

9. In the gate device, an upper image is acquired from an upper camera, and a lower image having a common area that is partially common to the upper image is acquired from a lower camera that is installed vertically below the upper camera; When a plurality of users are shown in the common area of ​​the upper image and the lower image, a user in the front row is identified by calculating a distance between each of the plurality of users and a surface on which the camera is installed, and the user in the front row is set as a person to be authenticated; If a plurality of user faces are shown in the area excluding the common area in the upper image or the lower image, an error is set; Controlling the gate based on the authentication result of the person to be authenticated. A method for controlling a gate device.

10. The computer installed in the gate device A process of acquiring an upper image from an upper camera, and acquiring a lower image having a common area that is partially common to the upper image from a lower camera installed vertically below the upper camera; a process of identifying a user in a front row by calculating a distance between each of the users and a surface on which the camera is installed, when a plurality of users are shown in the common area of ​​the upper image and the lower image, and setting the user in the front row as a person to be authenticated; a process of setting an error when a plurality of user faces are shown in an area excluding the common area in the upper image or the lower image; A process of controlling a gate based on a result of authentication of the person to be authenticated; A program for executing the above.

Images