Authentication method, authentication system, face authentication ticket gate, and face authentication server

Abstract

To manage user's entry / exit of a gate with high accuracy.SOLUTION: An authentication method includes steps of: creating a first face image obtained by extracting a face portion from a picked up image of a user passing through a face authentication ticket gate, the picked up image being picked up by a camera; checking the first face image against a second face image which is an image of the face of a registered person registered in advance; calculating a matching rate between the first face image and the second face image; acquiring terminal identification information which is identification information about a terminal possessed by a person located in the face authentication ticket gate; checking terminal identification information associated with the second face image having a high matching rate against the terminal identification information of the terminal possessed by the person; and determining whether the user is permitted to pass through the face authentication ticket gate based on user identification information associated with the checked terminal identification information and including information as to whether the user is permitted to pass through the face authentication ticket gate.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to an authentication method, an authentication system, a face recognition ticket gate, and a face recognition server. [Background technology]

[0002] Patent Document 1 discloses a station service system including an issuing device that issues passenger tickets, a server device, and a ticket gate that performs ticket gate processing. The issuing device issues passenger ticket information in response to an operational input by a user, acquires a first facial image of the user, and controls the server device to transmit the acquired first facial image to the server device. The server device controls the first facial image received from the issuing device to be delivered to the ticket gate. The ticket gate stores the first facial image delivered from the server device, acquires a second facial image of a user passing through the ticket gate, compares the second facial image with the first facial image, and controls the ticket gate to allow the user to pass if it is determined that the first facial image and the second facial image match.

[0003] Patent Document 2 discloses a gate device that does not require a ticket or the like and can improve the throughput of passing through the gate. The gate device includes a reading unit that receives one or more identifiers transmitted from wireless tags of one or more users who have entered a wireless area outside one longitudinal end of the gate, and an acquisition unit that acquires each facial feature registered corresponding to each identifier received from each wireless tag. The gate device further includes an imaging unit that images the user, an extraction unit that extracts facial features from image data generated by the imaging unit, and a face matching unit that receives the facial features extracted by the extraction unit and matches the features with the one or more facial features acquired by the acquisition unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5591650 [Patent Document 2] Patent No. 6816821 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, the entrance and exit of users is managed by comparing a facial image taken by a camera installed in the ticket gate with a registered facial image. However, in recent years, there has been a growing need to improve authentication accuracy by using multi-factor authentication that combines two or more factors for authentication.

[0006] Also, Patent Document 2 discloses two-factor authentication that combines authentication using an identifier transmitted from a wireless tag with authentication using facial features of the user. However, an authenticated user does not necessarily have the right to pass through a gate at the time of authentication. An authenticated user may not have the right to pass through a gate if, for example, the user's right to pass through the gate has expired or the user is unable to pay at the station ticket gate.

[0007] The present disclosure has been devised in consideration of the above-mentioned conventional situation, and aims to manage the entry and exit of users at gates with high accuracy. [Means for solving the problem]

[0008] The present disclosure provides an authentication method which creates a first facial image by extracting a facial portion from an image of a user passing through a facial recognition ticket gate captured by a camera, compares the first facial image with a second facial image which is an image of the face of a pre-registered person, calculates a matching rate between the first facial image and the second facial image, obtains terminal identification information which is identification information of a terminal carried by a person located inside the facial recognition ticket gate, compares the terminal identification information linked to the second facial image with the high matching rate with the terminal identification information of the terminal carried by the person, and determines whether the user is allowed to pass through the facial recognition ticket gate based on user identification information linked to the compared terminal identification information and including information on whether the user is allowed to pass through the facial recognition ticket gate.

[0009] The present disclosure also provides an authentication system including a facial recognition ticket gate through which a user passes, a facial recognition server that performs facial authentication of the user, and a right-of-passage determination server that determines whether the user is permitted to pass through the facial recognition ticket gate, wherein the facial recognition ticket gate creates a first facial image by extracting a facial portion from an image of the user captured by a camera, the facial recognition server compares the first facial image with a second facial image that is an image of the face of a pre-registered person, and calculates a matching rate between the first facial image and the second facial image, the facial recognition ticket gate acquires terminal identification information that is identification information of a terminal carried by a person located inside the facial recognition ticket gate, and compares the terminal identification information linked to the second facial image with the high matching rate with the terminal identification information of the terminal carried by the person, and the right-of-passage determination server determines whether the user is permitted to pass through the facial recognition ticket gate based on user identification information that is linked to the compared terminal identification information and includes information on whether the user is permitted to pass through the facial recognition ticket gate.

[0010] The present disclosure also provides a facial recognition ticket gate comprising a reader that acquires terminal identification information which is identification information of a terminal carried by a person passing through, and a processor, wherein the processor creates a first facial image by extracting a facial portion from an image of a user captured by a camera, compares the first facial image with a second facial image which is an image of the face of a pre-registered person, calculates a matching rate between the first facial image and the second facial image, acquires terminal identification information which is identification information of a terminal carried by the person, compares the terminal identification information linked to the second facial image with the high matching rate with the terminal identification information of the terminal carried by the person, and determines whether the user is allowed to pass through based on user identification information which is linked to the compared terminal identification information and includes information on whether the user is allowed to pass through.

[0011] The present disclosure also provides a facial recognition ticket gate through which a user passes, which creates a first facial image by extracting a facial portion from an image of the user captured by a camera, obtains a match rate between the first facial image and the second facial image calculated by comparing the first facial image with a second facial image which is an image of the face of a pre-registered person from a facial recognition server that performs facial authentication of the user, obtains terminal identification information which is identification information of a terminal carried by a person located inside the facial recognition ticket gate, compares the terminal identification information linked to the second facial image with the high match rate with the terminal identification information of the terminal carried by the person, and obtains a determination result as to whether or not the user is allowed to pass through the facial recognition ticket gate based on the user identification information including information as to whether or not the user is allowed to pass through the facial recognition ticket gate, which is linked to the compared terminal identification information from a right-of-passage determination server that determines whether or not the user is allowed to pass through the facial recognition ticket gate.

[0012] The present disclosure also provides a facial recognition server that performs facial recognition of a user, which acquires a first facial image in which a facial portion is extracted by a facial recognition ticket gate from an image of the user captured by a camera, compares the first facial image with a second facial image which is an image of the face of a pre-registered person, and calculates a matching rate between the first facial image and the second facial image which the facial recognition ticket gate uses when comparing terminal identification information which is identification information of a terminal held by a person located inside the facial recognition ticket gate with terminal identification information linked to the second facial image.

[0013] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0014] According to the present disclosure, it is possible to manage user entry and exit through gates with high accuracy. [Brief description of the drawings]

[0015] [Figure 1] An example of the appearance of a facial recognition ticket gate [Diagram 2] Block diagram showing the system configuration of the two-factor authentication system [Diagram 3] FIG. 1 is a block diagram showing a modified example of the system configuration of a two-factor authentication system. [Figure 4] Diagram showing the matching zone and pass-through determination zone inside a facial recognition ticket gate [Diagram 5] FIG. 1 is a diagram for explaining a face image and a body image according to the present embodiment. [Figure 6] A flowchart showing an example of a detection process performed by a face recognition ticket gate according to the present embodiment. [Figure 7] FIG. 1 is a diagram showing an example of a configuration of a person information table according to the present embodiment; [Figure 8] A flowchart showing an example of a process for managing a user's passage through a face authentication ticket gate using two-factor authentication. [Figure 9] Diagram explaining the narrowing down of candidates for face matching [Figure 10] FIG. 1 is a sequence diagram of an example of a detection process performed by the two-factor authentication system according to the present embodiment. [Figure 11] A diagram showing an example where one ID acquisition reader is installed. [Figure 12] A diagram showing an example in which four ID acquisition readers are installed [Figure 13] A diagram showing the detectable area when four ID acquisition readers are installed. [Figure 14] Diagram explaining the readable orientation of a wireless tag [Figure 15] A diagram showing an example in which eight ID acquisition readers are installed [Figure 16] FIG. 13 is a diagram showing an example of user management information data. [Figure 17] A diagram showing an example of payment and ticket information data [Figure 18] FIG. 13 is a diagram showing an example of passage history information data; [Figure 19] A flowchart showing a first modified example of the process of managing the passage of a user through a face authentication ticket gate using two-factor authentication. [Figure 20]A flowchart showing a second modified example of the process for managing the passage of a user through a face authentication ticket gate using two-factor authentication. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, with reference to the drawings as appropriate, an embodiment specifically disclosing an authentication method, an authentication system, a face authentication ticket gate, and a face authentication server according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0017] First, an example of the external appearance of the face authentication ticket gate 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the external appearance of the face authentication ticket gate 10.

[0018] The face recognition ticket gate 10 is, for example, a ticket gate installed at a station, which manages the entrance and exit of users to the station and the payment for each user. The face recognition ticket gate 10 is not limited to a ticket gate at a station, and may be installed at an event venue, a theme park, a live performance, or the like. For example, when the face recognition ticket gate 10 is installed at an event venue, it determines the entry qualification of each user to the event venue, manages the entrance and exit, and settles the entrance fee or the payment within the event venue. In the following, for convenience of explanation, the face recognition ticket gate 10 according to the present embodiment is assumed to be a ticket gate installed at a station, but is not limited to a ticket gate installed at a station.

[0019] The face authentication ticket gate 10 authenticates the user using registration information that the user has registered in advance in an ID management server 40 (see FIG. 2) in order to use the face authentication ticket gate 10, and performs pass management and payment management for the user.

[0020] As shown in FIG. 1, the facial recognition ticket gate 10 is designed so that a user entering a station and a user exiting a station can pass each other at the same time. The facial recognition ticket gate 10 is not limited to a design that allows passage from both directions, and may be designed to allow passage from only one direction. The one direction is the direction of entering the station or the direction of exiting the station. In this embodiment, the first direction is described as the direction of entering the station, and the second direction is the direction of exiting the station. However, the first direction may be the direction of exiting the station, and the second direction may be the direction of entering.

[0021] Also, in this embodiment, for convenience of explanation, the axis along the first direction or the second direction will be described as the X-axis, the axis perpendicular to the X-axis and extending in the height direction will be described as the Z-axis, and the axis perpendicular to the X-axis and Z-axis will be described as the Y-axis.

[0022] When the facial recognition ticket gate 10 allows passage from both directions, the facial recognition ticket gate 10 is provided with a camera (not shown) that photographs the face of a user entering the station and a camera (not shown) that photographs the face of a user leaving the station. The facial recognition ticket gate 10 also has at least one ID acquisition reader (not shown) that detects radio waves emitted from a wireless tag held by the user. Here, ID is an abbreviation for identification. The wireless tag is mounted on a terminal device such as an integrated circuit (hereinafter referred to as "IC") card, smartphone, or smartwatch that allows the user to make a payment. The wireless tag emits radio waves including identification information of each terminal device. The facial recognition ticket gate 10 uses the camera and the ID acquisition reader to perform two-factor authentication of face information and identification information of the wireless tag held by the user, and automatically manages the entrance and exit of the user and the payment. The details of the two-factor authentication performed by the facial recognition ticket gate 10 will be described in detail later in this embodiment. That is, the payment process is properly performed by the user simply passing through the facial recognition ticket gate 10 without holding a terminal device such as an IC card over the ticket gate. In other words, by smoothing the user's entry and exit into the station, the facial recognition ticket gate 10 can ease congestion near the ticket gate and improve user convenience.

[0023] The facial recognition ticket gate 10 is equipped with a display device 14 that can be seen by users entering the station and a display device 14 that can be seen by users leaving the station. The display device 14 is provided, for example, at the top of the facial recognition ticket gate 10 and oriented in a direction facing the direction in which users pass through. The position of the display device 14 is not limited, and it may be provided on the inner side surface of the facial recognition ticket gate 10.

[0024] 1, the facial recognition ticket gate 10 does not have an opening and closing bar, so a user can smoothly pass through the facial recognition ticket gate 10 without stopping inside the facial recognition ticket gate 10. Note that the facial recognition ticket gate 10 may be provided with an opening and closing bar.

[0025] The appearance of the face authentication ticket gate 10 is not limited to the example shown in FIG. 1, and the design may be changed as appropriate as long as it is equipped with a camera and an ID acquisition reader required for authenticating users.

[0026] The following describes in detail a two-factor authentication method for managing users' entrance and exit to and from a station using the face recognition ticket gate 10.

[0027] Next, the system configuration of the two-factor authentication system 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the system configuration of the two-factor authentication system 1. In Fig. 2, the dotted lines indicate that the connection is not limited to real-time connection, and may be connection only during times set by the operator, such as at night, or connection at predetermined time intervals.

[0028] The two-factor authentication system 1 includes a face recognition ticket gate 10, a face recognition server 20, a two-factor authentication server 30, an ID management server 40, a right-to-pass determination server 50, a passage history server 60, a billing management server 70 and a face information registration terminal 80.

[0029] The face recognition ticket gate 10 includes a camera 11 , an ID acquisition reader 12 , a communication I / F 13 , a display device 14 , a processor 15 , a personal information database 18 and a memory 19 .

[0030] The camera 11 is a camera that photographs a user passing through the face authentication ticket gate 10. A plurality of cameras 11 are installed at the face authentication ticket gate 10. The camera 11 is configured to have at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the angle of view of the area photographed by the camera 11, and forms an optical image of the object on the light receiving surface of the image sensor (in other words, the photographing surface). The image sensor is a solid-state photographing element such as a Charged Coupled Device (hereinafter referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter referred to as "CMOS"). The image sensor converts the optical image formed on the photographing surface through the lens into an electrical signal every predetermined time (for example, 1 / 30 (second)). For example, when the predetermined time is 1 / 30 (second), the frame rate of the camera 11 is 30 fps. Furthermore, camera 11 may generate image data (video data) by performing a predetermined signal processing on the electrical signal at each predetermined time. The image data is a still image, and the video data is a video. Camera 11 outputs the image data (video data) to processor 15. Hereinafter, image data and video data will be referred to as captured images. Camera 11 may be omitted from face recognition ticket gate 10. In this case, the captured images are acquired from camera 11 provided outside face recognition ticket gate 10 via communication I / F 13.

[0031] The ID acquisition reader 12 is a reader that acquires radio waves emitted from a wireless tag in a terminal such as an IC card or smartphone owned by a user. The terminal owned by a user is a terminal device that performs payment processing based on information on the entry station and exit station when using a station. The ID acquisition reader 12 acquires ID information (hereinafter referred to as tag ID) emitted by a wireless tag. The ID acquisition reader 12 outputs the acquired tag ID to the processor 15. The tag ID can be interpreted as terminal identification information. The ID acquisition reader 12 is an ultra-high frequency band reader or a beacon using Bluetooth (registered trademark), etc.

[0032] The communication I / F 13 is an interface circuit that performs wireless or wired communication between the face recognition ticket gate 10 and the face recognition server 20, between the face recognition ticket gate 10 and the two-factor authentication server 30, between the face recognition ticket gate 10 and the right-to-pass determination server 50, and between the face recognition ticket gate 10 and the passage history server 60. Here, I / F refers to an interface. Communications between the face recognition ticket gate 10 and the face recognition server 20, between the face recognition ticket gate 10 and the two-factor authentication server 30, between the face recognition ticket gate 10 and the right-to-pass determination server 50, and between the face recognition ticket gate 10 and the passage history server 60 may be via a network. The communication method by the communication I / F 13 is, for example, a Wide Area Network (hereinafter referred to as "WAN"), a Local Area Network (hereinafter referred to as "LAN"), Long Term Evolution (hereinafter referred to as "LTE"), mobile communications such as 4G and 5G, power line communication, short-distance wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones. In addition, since the communication I / F 21, communication I / F 31, communication I / F 41, communication I / F 51, communication I / F 61 and communication I / F 71 use the same communication methods as the communication I / F 13, the explanation of the communication methods will be omitted.

[0033] The display device 14 displays the authentication result of the user, etc. The display device 14 is, for example, a display.

[0034] The processor 15 is, for example, a central processing unit (hereinafter referred to as "CPU"), a digital signal processor (hereinafter referred to as "DSP"), a graphical processing unit (hereinafter referred to as "GPU"), or a field programmable gate array (hereinafter referred to as "FPGA"). The processor 15 functions as a controller that manages the overall operation of the face recognition ticket gate 10. The processor 15 performs control processing for managing the operation of each part of the face recognition ticket gate 10, input / output processing of data between each part of the face recognition ticket gate 10, calculation processing of data, and storage processing of data. The processor 15 operates according to a program stored in the memory 19. The processor 15 uses the memory 19 during operation, and temporarily stores data generated or acquired by the processor 15 in the memory 19. The processor 15 realizes the functions of the ID matching unit 16 and the tracking unit 17 by using the programs and data stored in the memory 19. The processor 15 detects a face image 220 (see FIG. 5) showing the user's face and a body image 200 (see FIG. 5) showing the user's body from the captured image acquired from the camera 11. The detection of the user's face image 220 may be realized using any known technology, such as pattern matching or artificial intelligence (hereinafter referred to as "AI"). The AI ​​may be an image recognition technology using deep learning such as a Convolutional Neural Network. The face image 220 may be read as a first face image.

[0035] The ID matching unit 16 performs the above-mentioned matching by matching the result of the face authentication with the tag ID received from the ID acquisition reader 12. The ID matching unit 16 performs the above-mentioned matching using a User-ID linked to registration information registered in advance by the face information registration terminal 80. Here, the User-ID is identification information linked to the registration information of the user registered by the face information registration terminal 80. Note that "matching" may be read as "authentication". The User-ID may be read as user identification information.

[0036] The tracking unit 17 tracks and detects the position of a user passing through the face authentication ticket gate 10. The tracking unit 17 includes an update unit 17A, a coordinate calculation unit 17B, a same person determination unit 17C, a zone determination unit 17D, and a direction determination unit 17E.

[0037] The update unit 17A updates, at set time intervals, the face image and the body image detected from the captured image acquired by the camera 11. The set time can be arbitrarily set by the business operator of the two-factor authentication system 1, etc.

[0038] The coordinate calculation unit 17B detects the coordinates of the face (hereinafter referred to as "face coordinates") from the face image updated by the update unit 17A. The coordinate calculation unit 17B detects the coordinates of the body (hereinafter referred to as "body coordinates") from the body image updated by the update unit 17A.

[0039] The same person determination unit 17C calculates the coordinate difference (hereinafter referred to as the "position difference") from the face coordinates and body coordinates before and after the captured image is updated by the update unit 17A, and determines from the position difference whether the person appearing in the captured images before and after the update is the same person.

[0040] The zone determination unit 17D determines the area in which the user is located within the face authentication ticket gate 10 based on the face coordinates and body coordinates. The areas within the face authentication ticket gate 10 will be described with reference to FIG.

[0041] The direction determining unit 17E determines the direction in which the user is moving from the position of the camera 11 that photographed the user's face.

[0042] The personal information database 18 stores a personal information table 400 (see FIG. 7). The personal information database 18 is a non-volatile storage device, such as a hard disk drive (hereinafter referred to as "HDD"), a solid state drive (hereinafter referred to as "SSD"), a universal serial bus (hereinafter referred to as "USB") memory, an optical disk, or a magnetic tape. The personal information database 18 may be omitted from the face recognition ticket gate 10. In this case, the personal information table 400 may be temporarily stored in the memory 19 and deleted after the authentication of the target person is completed. In addition, when the personal information database 18 is omitted from the face recognition ticket gate 10, the personal information table 400 may be transmitted to the passage history server 60 and stored in the passage history database 64 in association with the person's passage history.

[0043] Memory 19 is configured using, for example, Random Access Memory (hereinafter referred to as "RAM") and Read Only Memory (hereinafter referred to as "ROM"), and temporarily stores programs necessary for the operation of facial recognition ticket gate 10, as well as data generated during operation. RAM is, for example, a work memory used during operation of facial recognition ticket gate 10. ROM stores and holds, for example, programs for controlling facial recognition ticket gate 10 in advance. Note that descriptions of memory 22, memory 33, memory 43, memory 54, memory 63, and memory 73 will be omitted because the facial recognition ticket gate 10 in the description of memory 19 can be replaced with the name of the server on which each memory is installed.

[0044] The face authentication server 20 is a server device that performs face authentication of a user using a photographed image acquired from the face authentication ticket gate 10. The face authentication server 20 is, for example, a server computer. The face authentication server 20 has a communication I / F 21, a memory 22, a processor 23, and a face feature database 27.

[0045] The communication I / F 21 is an interface circuit that performs wireless or wired communication between the face recognition ticket gate 10 and the face recognition server 20, and between the face recognition server 20 and the ID management server 40. The communication between the face recognition ticket gate 10 and the face recognition server 20, and between the face recognition server 20 and the ID management server 40 may be performed via a network.

[0046] The processor 23 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 23 functions as a controller that manages the overall operation of the face authentication server 20. The processor 23 performs control processing for managing the operation of each part of the face authentication server 20, data input / output processing between each part of the face authentication server 20, data arithmetic processing, and data storage processing. The processor 23 operates according to a program stored in the memory 22. The processor 23 uses the memory 22 during operation, and temporarily stores data generated or acquired by the processor 23 in the memory 22. The processor 23 realizes the functions of the body detection unit 24, the face feature extraction unit 25, and the face matching unit 26 by using the programs and data stored in the memory 22.

[0047] The facial feature amount extraction unit 25 extracts facial feature amounts from a facial image. The extraction of the facial feature amounts may be realized using a known technique such as AI.

[0048] The face matching unit 26 matches the face image linked to the face ID acquired from the face feature database 27 with the face image acquired from the face authentication ticket gate 10, using the features extracted by the face feature extraction unit 25. The face ID is an ID linked to the face image of the user registered in advance in the ID management server 40 by the face information registration terminal 80. Therefore, the face matching unit 26 may acquire the face ID from the ID management server 40. The face image of the user registered in advance in the ID management server 40 by the face information registration terminal 80 may be read as the second face image.

[0049] The facial feature database 27 stores information in which a facial ID and a User-ID are linked. The facial feature database 27 is a non-volatile storage device, such as a HDD, SSD, USB memory, optical disk, or magnetic tape. The facial feature database 27 may be omitted, and the facial authentication server 20 may acquire the facial ID and the User-ID from the ID management server 40.

[0050] The two-factor authentication server 30 is a server device that manages tag IDs and User-IDs. The two-factor authentication server 30 is, for example, a server computer. The two-factor authentication server 30 includes a communication I / F 31, a processor 32, a memory 33, and a User-ID database .

[0051] The communication I / F 31 is an interface circuit that performs wireless or wired communication between the face recognition ticket gate 10 and the two-factor authentication server 30, and between the two-factor authentication server 30 and the ID management server 40. The communication between the face recognition ticket gate 10 and the two-factor authentication server 30, and between the two-factor authentication server 30 and the ID management server 40 may be performed via a network.

[0052] The processor 32 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 32 functions as a controller that manages the overall operation of the two-factor authentication server 30. The processor 32 performs control processing for managing the operation of each part of the two-factor authentication server 30, input / output processing of data between each part of the two-factor authentication server 30, data arithmetic processing, and data storage processing. The processor 32 operates according to a program stored in the memory 33. The processor 32 uses the memory 33 during operation, and temporarily stores data generated or acquired by the processor 32 in the memory 33.

[0053] Information linking the tag ID and the User ID is stored in the User-ID database 34. The two-factor authentication server 30 acquires information linking the tag ID and the User ID from the ID management server 40 and stores the information in the User-ID database 34. The User-ID database 34 is a non-volatile storage device, such as an HDD, an SSD, a USB memory, an optical disk, or a magnetic tape.

[0054] In addition, since information such as a tag ID is registered in the ID management server 40 in association with a User-ID, the ID management server 40 may be used instead of the two-factor authentication server 30. However, the face recognition ticket gate 10 can perform authentication of a user by using the two-factor authentication server 30 instead of the ID management server 40, and can perform authentication while considering privacy without using personal information of the user more than necessary. Similarly, the face recognition server 20, the right-to-pass determination server 50, the passage history server 60, and the charge management server 70 are preferably provided separately from the viewpoint of user privacy and not substituted by the ID management server 40. However, the face recognition server 20, the right-to-pass determination server 50, the passage history server 60, and the charge management server 70 can be substituted by the ID management server 40, and the configuration of the server device that communicates with the face recognition ticket gate 10 can be appropriately changed by the operator.

[0055] The ID management server 40 is a server device that manages the registration information of the user acquired from the face information registration terminal 80. The registration information of the user will be described later. The ID management server 40 is, for example, a server computer. The ID management server 40 includes a communication I / F 41, a processor 42, a memory 43, and a user management database 44.

[0056] The communication I / F 41 is an interface circuit that performs wireless or wired communication between the face authentication server 20 and the ID management server 40, between the two-factor authentication server 30 and the ID management server 40, between the ID management server 40 and the right of passage judgment server 50, and between the ID management server 40 and the right of passage judgment server 50. The communication between the face authentication server 20 and the ID management server 40, between the two-factor authentication server 30 and the ID management server 40, between the ID management server 40 and the right of passage judgment server 50, and between the ID management server 40 and the right of passage judgment server 50 may be performed via a network.

[0057] The processor 42 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 42 functions as a controller that manages the overall operation of the ID management server 40. The processor 42 performs control processing for managing the operation of each part of the ID management server 40, input / output processing of data between each part of the ID management server 40, calculation processing of data, and storage processing of data. The processor 42 operates according to a program stored in the memory 43. The processor 42 uses the memory 43 during operation, and temporarily stores data generated or acquired by the processor 42 in the memory 43.

[0058] The user management database 44 stores user registration information acquired from the face information registration terminal 80. The user management database 44 stores user management information 500. The user registration information acquired from the face information registration terminal 80 is, for example, a name, face image data, a tag ID read from a terminal such as an IC card, credit card information, a telephone number, and an email address. The user management database 44 is a non-volatile storage device, for example, an HDD, an SSD, a USB memory, an optical disk, or a magnetic tape.

[0059] The right-to-passage determination server 50 is a server device that determines whether or not a user is permitted to pass through the face recognition ticket gate 10. The right-to-passage determination server 50 is, for example, a server computer. The right-to-passage determination server 50 includes a communication I / F 51, a processor 52, a memory 54, and a payment and ticket information database 55.

[0060] The communication I / F 51 is an interface circuit that performs wireless or wired communication between the face recognition ticket gate 10 and the right to pass determination server 50, between the ID management server 40 and the right to pass determination server 50, and between the right to pass determination server 50 and the passage history server 60. The communication between the face recognition ticket gate 10 and the right to pass determination server 50, between the ID management server 40 and the right to pass determination server 50, and between the right to pass determination server 50 and the passage history server 60 may be performed via a network.

[0061] The processor 52 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 52 functions as a controller that manages the overall operation of the right-of-passage determination server 50. The processor 52 performs control processing for managing the operation of each part of the right-of-passage determination server 50, input / output processing of data between each part of the right-of-passage determination server 50, calculation processing of data, and storage processing of data. The processor 52 operates according to a program stored in the memory 54. The processor 52 uses the memory 54 during operation, and temporarily stores data generated or acquired by the processor 52 in the memory 54. The processor 52 realizes the function of the right-of-passage determination unit 53 by using the program and data stored in the memory 54.

[0062] The right-to-passage determination unit 53 uses payment / passenger ticket information 600 (see FIG. 15) stored in the payment / passenger ticket information database to determine whether or not the user is permitted to pass through the face authentication ticket gate 10. The right-to-passage determination unit 53 uses, for example, the entry / exit status and payment information in the payment / passenger ticket information 600 to determine whether or not the user has a sufficient balance or whether or not the commuter pass has expired, to determine whether or not the user is permitted to pass through.

[0063] The payment / passenger ticket information database 55 stores information linking the face ID and User-ID obtained from the ID management server 40. The payment / passenger ticket information database 55 stores payment / passenger ticket information 600. The payment / passenger ticket information database 55 is a non-volatile storage device, such as an HDD, SSD, USB memory, optical disk, or magnetic tape.

[0064] The passage history server 60 is a server device that determines whether or not a user has passed through the face recognition ticket gate 10. The passage history server 60 is, for example, a server computer. The passage history server 60 has a communication I / F 61, a processor 62, a memory 63, and a passage history database 64.

[0065] The communication I / F 61 is an interface circuit that performs wireless or wired communication between the face recognition ticket gate 10 and the passage history server 60, between the right-of-passage determination server 50 and the passage history server 60, and between the passage history server 60 and the billing management server 70. Communications between the face recognition ticket gate 10 and the passage history server 60, between the right-of-passage determination server 50 and the passage history server 60, and between the passage history server 60 and the billing management server 70 may be performed via a network.

[0066] The processor 62 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 62 functions as a controller that manages the overall operation of the passage history server 60. The processor 62 performs control processing for managing the operation of each part of the passage history server 60, input / output processing of data between each part of the passage history server 60, calculation processing of data, and storage processing of data. The processor 62 operates according to a program stored in the memory 63. The processor 62 uses the memory 63 during operation, and temporarily stores data generated or acquired by the processor 62 in the memory 63.

[0067] The passage history database 64 stores information on the station entry / exit status of the user and the charged amount. The passage history database 64 is a non-volatile storage device, for example, an HDD, an SSD, a USB memory, an optical disk, or a magnetic tape.

[0068] The charge management server 70 is a server device that manages charges to users. For example, the charge management server 70 determines a charge amount from information on the station where the user enters and exits, and performs payment processing based on the payment information linked by the user at the time of registration. The charge management server 70 is, for example, a server computer. The charge management server 70 has a communication I / F 71, a processor 72, a memory 73, and a charge management database 74.

[0069] The communication I / F 71 is an interface circuit that performs wireless or wired communication between the ID management server 40 and the billing management server 70, and between the passage history server 60 and the billing management server 70. The communication between the ID management server 40 and the billing management server 70, and between the passage history server 60 and the billing management server 70 may be performed via a network.

[0070] The processor 72 is, for example, a CPU, a DSP, a GPU, or an FPGA. The processor 72 functions as a controller that manages the overall operation of the billing management server 70. The processor 72 performs control processing for managing the operation of each part of the billing management server 70, input / output processing of data between each part of the billing management server 70, calculation processing of data, and storage processing of data. The processor 72 operates according to a program stored in the memory 73. The processor 72 uses the memory 73 during operation, and temporarily stores data generated or acquired by the processor 72 in the memory 73.

[0071] The charge management database 74 stores the charge amount for the user. The charge management database 74 is a non-volatile storage device, such as an HDD, SSD, USB memory, optical disk, or magnetic tape.

[0072] The billing management server 70 may be integrated with the passage history server 60 and omitted from the two-factor authentication system 1.

[0073] The face information registration terminal 80 is a terminal device used by a user to input registration information. The face information registration terminal 80 may be, for example, a terminal device such as a smartphone, a personal computer, or a tablet, or may be a terminal device installed at a station or the like for registration. A user inputs registration information into the face information registration terminal 80 to register information in which a face image and an IC tag are linked in advance in the ID management server 40. The face information registration terminal 80 acquires information on the IC tag using a reader mounted on the face information registration terminal 80 itself or an external card reader. The face information registration terminal 80 transmits the acquired registration information of the user to the ID management server 40.

[0074] 3, the facial authentication ticket gate 10 may be configured to include the functions and databases of a facial authentication server 20 and a right-to-pass determination server 50. FIG. 3 is a block diagram showing a modified example of the system configuration of the two-factor authentication system 1.

[0075] Next, the verification zone and the passage determination zone in the face authentication ticket gate 10 will be described with reference to Fig. 4. Fig. 4 is a diagram showing the verification zone and the passage determination zone in the face authentication ticket gate 10. Note that, although the following description will be given taking the first direction as an example, the same applies to the second direction.

[0076] The image captured by camera 11A is used for face matching of a user moving in the first direction. In detection area 300, which is an area for detecting users within face authentication ticket gate 10, an area close to camera 11A, ID acquisition reader 12A, and display device 14A is set as passage determination zone 301. In detection area 300, an area close to camera 11B, ID acquisition reader 12B, and display device 14B is set as matching zone 302.

[0077] The proportion of the area of ​​passage determination zone 301 and verification zone 302 in detection area 300 can be set appropriately by the business operator. For example, the area in which the user is captured by camera 11A may be set as verification zone 302, and the area in which the user is not captured by camera 11A or is too close to capture the entire user's body may be set as passage determination zone 301.

[0078] The passage determination zone 301 is an area where it is determined whether or not a user has passed through the face recognition ticket gate 10. When the camera 11 detects that the user has entered the passage determination zone 301, the passage history server 60 determines that the user has passed through the face recognition ticket gate 10.

[0079] The verification zone 302 is an area where facial authentication of a user is performed. That is, the facial authentication of a user is performed by the facial authentication server 20 based on an image captured by the camera 11 when the user is located in the verification zone 302. The facial authentication ticket gate 10 collates the result of facial authentication of a user located in the verification zone 302 with a User-ID linked to a tag ID. In addition, the right-to-pass determination server 50 determines whether or not a user located in the verification zone 302 has the right to pass through the facial authentication ticket gate 10. That is, in the verification zone 302, facial authentication of a user, authentication of a tag ID, and determination of the right to pass are performed.

[0080] Next, a face image and a body image according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram for explaining a face image and a body image according to this embodiment.

[0081] As shown in Figures 5(a) and 5(b), the processor 15 detects the entire body of the user 7 from the captured image received from the camera 11. In this embodiment, the image of the entire body of the user 7 detected from the captured image is referred to as a body image 200.

[0082] 5(a), the processor 15 detects the face of the user 7 from the captured image received from the camera 11. In this embodiment, the image of the face of the user 7 detected from the captured image is referred to as a face image 220.

[0083] The processor 15 generates a personal information table 400 (see FIG. 7) corresponding to the user 7 located in the detection area 300 based on the body image 200 or the face image 220. The processor 15 may generate a personal information table 400 for each user 7 detected in the detection area 300. Details of the personal information table 400 will be described later (see FIG. 7). The processor 15 stores the generated personal information table 400 in the personal information database 18.

[0084] Next, an example of detection processing performed by the face authentication ticket gate 10 according to the present embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of detection processing performed by the face authentication ticket gate 10 according to the present embodiment.

[0085] The processor 15 receives a captured image from the camera 11 (step S101).

[0086] The processor 15 receives a distance image from the camera 11 (step S102). In this way, the camera 11 may generate a distance image indicating the distance to the user 7 or the like present in the detection area 300. The generation of the distance image may be realized using any known technology, such as a stereo camera or a distance measuring sensor (e.g., Light Detection and Ranging). Note that, in this embodiment, the process of step S102 is performed after the process of step S101, but this is not limited thereto, and the order may be reversed.

[0087] As shown in FIG. 5(a), the processor 15 detects a face image 220 of the user 7 from the captured image (S103).

[0088] 5(a), the processor 15 determines whether or not a face image 220 of the user 7 is detected from the captured image (S104). If the face image 220 is not detected (S104, NO), the processor 15 advances the process to step S108.

[0089] When the processor 15 detects the face image 220 (S104, YES), the coordinate calculation unit 17B uses the distance image to calculate three-dimensional coordinates (hereinafter referred to as face coordinates 221 (see FIG. 5)) related to the detected face image 220 (S105). For example, the coordinate calculation unit 17B calculates the center point of the face image 220 as the face coordinates 221, as shown in FIG. 5(a).

[0090] The processor 15 generates face detection information (S106). For example, the processor 15 generates face detection information including an ID (hereinafter referred to as a photographing device ID) for identifying which of multiple cameras the camera 11 that is the transmission source of the captured image is, a face image 220, a size of the face image 220 (hereinafter referred to as a face image size), face coordinates 221, and a capture time of the captured image.

[0091] The processor 15 transmits the generated face detection information to the face authentication server 20 (S107), and proceeds to the next step S108.

[0092] The processor 15 detects a body image 200 of the user 7 from the captured image, as shown in FIG. 5(a) or FIG. 5(b) (S108).

[0093] The processor 15 determines whether or not the body image 200 has been detected from the photographed image in step S108 (S109). If the body image 200 has not been detected (S109, NO), the processor 15 returns the process to step S101.

[0094] When the processor 15 detects the body image 200 (S109, YES), the coordinate calculation unit 17B uses the distance image to calculate three-dimensional coordinates (hereinafter, referred to as body coordinates 201 (see FIG. 5)) for the detected body image 200 (S110). For example, the coordinate calculation unit 17B calculates the center point of the body image 200 as the body coordinates 201, as shown in FIG. 5(a) and FIG. 5(b).

[0095] The processor 15 generates body detection information (S111). For example, the processor 15 generates body detection information including an imaging device ID, a body ID for identifying the body image 200, the body image 200, the body coordinates 201, and the imaging time of the captured image.

[0096] The processor 15 transmits the generated body detection information to the face authentication server 20 (S112), and returns the process to step S101. Note that in the present embodiment, the process of detecting a face image from a captured image (S103 to S107) is followed by the process of detecting a body image from a captured image (S108 to S112), but is not limited thereto, and the steps may be reversed.

[0097] Next, a configuration example of the personal information table 400 according to the present embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing a configuration example of the personal information table 400 according to the present embodiment.

[0098] The person information table 400 is generated for each user detected in the detection area 300. The person information table 400 includes, as information or data, a User-ID, a first image capture time, an image update time, a body image, a body image update time, body coordinates, a face image, a face image update time, face coordinates, zone information, and a movement direction.

[0099] The first image capturing time is set to the time when a body image or face image included in the personal information table 400 was first captured. The first image capturing time can also be said to be the time when the personal information table 400 was first generated.

[0100] The image update time is set to the time when the body image or face image included in the personal information table 400 was last updated. The image update time can also be said to be the time when the personal information table 400 was last updated.

[0101] The body image is set to the body image 200 detected from a photographed image of the user indicated in the personal information table 400. The body image may be updated at any time to the body image 200 detected from the latest photographed image.

[0102] The time when the body image 200 included in the person information table 400 was last updated is set as the body image update time.

[0103] The body coordinates 201 calculated based on the body image are set as the body coordinates. The body coordinates 201 may be expressed as three-dimensional coordinates (X coordinate, Y coordinate, Z coordinate) in the detection area 300.

[0104] The face image is set to the face image 220 detected from the photographed image of the user indicated in the personal information table 400. The face image may be updated as needed to the face image 220 detected from the latest photographed image.

[0105] The time when the face image included in the personal information table 400 was last updated is set as the face image update time.

[0106] The face coordinates are set to face coordinates 221 calculated based on a face image included in the person information table 400. The face coordinates may be expressed as three-dimensional coordinates in the detection area 300 (X coordinate, Y coordinate, Z coordinate).

[0107] In the zone information, the zone in which the user is located at the time the image in the person information table 400 is updated (passage determination zone 301 or verification zone 302 (see FIG. 4)) is set.

[0108] The movement direction is set to the movement direction of the user indicated in the person information table 400. For example, the movement direction is set to either the first direction or the second direction.

[0109] Next, an example of the process of managing the passage of a user through the face authentication ticket gate 10 using two-factor authentication will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the process of managing the passage of a user through the face authentication ticket gate 10 using two-factor authentication. Each process in the flowchart of Fig. 8 is executed by the face authentication ticket gate 10.

[0110] When a user enters the matching zone 302 of the face authentication ticket gate 10, the processor 15 receives a photographed image of the user from the camera 11. The processor 15 determines whether or not the face and body of the user can be detected from the received photographed image (step St200).

[0111] When the processor 15 determines that the face and body of the user cannot be detected from the received captured image (step St200, NO), it executes the process of step St200 again and repeats the process of step St200 until the face and body are detected.

[0112] When the processor 15 determines that the face and body of the user have been detected from the received photographed image (step St200, YES), it generates face detection information and body detection information (step St201). The method of creating the face detection information and body detection information has been described with reference to FIG. 6, and therefore will not be described here.

[0113] The processor 15 transmits the face detection information and the body detection information to the face authentication server 20 (step St202). The face authentication server 20 calculates the facial feature amount from the face detection information received from the processor 15. The calculation of the facial feature amount may use a known technology such as AI. The face authentication server 20 performs face matching of the user using the face image linked to the face ID stored in the face feature amount database 27 and the calculated facial feature amount. That is, the face authentication server 20 performs one-to-N matching using the face IDs of N (N: an integer equal to or greater than 1) registered persons stored in the face feature amount database 27. The face authentication server 20 performs the above-mentioned one-to-N matching and calculates the matching rate between the face detection information and the face image of the person stored in the face feature amount database 27. The face authentication server 20 narrows down the persons with high matching rates in descending order of matching rate. The face authentication server 20 transmits the User-IDs of the multiple persons narrowed down from the persons stored in the face feature amount database 27 as a result of face matching to the processor 15. Hereinafter, a plurality of people narrowed down from among the people stored in the face feature amount database 27 will be referred to as narrowed down candidates.

[0114] The processor 15 acquires a User-ID based on the narrowed-down candidates as a result of face matching from the face authentication server 20 (step St203). After the process of step St203, the processor 15 executes the process of step St206.

[0115] The processor 15 starts the process of step St200 and the process of step St204 at the same time. Note that the process of step St200 and the process of step St204 do not need to be performed strictly at the same time, and there may be a time lag of several milliseconds to several hundred milliseconds.

[0116] The processor 15 determines whether or not a tag ID has been received from a wireless tag carried by a person located in the verification zone 302 (step St204). The process of step St204 is performed based on whether or not a tag ID has been received from the ID acquisition reader 12.

[0117] When the processor 15 determines that the tag ID has not been received from the wireless tag present in the detection area 300 (step St204, NO), the processor 15 executes the process of step St204 again and repeatedly executes the process of step St204 until the tag ID is received.

[0118] If the processor 15 determines that it has received a tag ID from a wireless tag present within the detection area 300 (step St204, YES), it obtains the User-ID linked to the received tag ID from the User-ID database 34 of the two-factor authentication server 30 (step St205).

[0119] The ID matching unit 16 matches the User-ID acquired in the process of step St203 with the User-ID acquired in the process of step St205. That is, the ID matching unit 16 determines whether or not the User-ID acquired in the process of step St205 is present among the multiple User-IDs based on the narrowed-down candidates acquired in the process of step St203 (step St206).

[0120] When the ID matching unit 16 determines that the User-ID acquired in the process of step St205 does not exist among the multiple User-IDs based on the narrowed-down candidates acquired in the process of step St203 (step St206, NO), it displays an alert on the display device 14 or notifies the user (step St207). The alert displayed on the display device 14 is, for example, a content indicating that authentication has failed, a content urging the user to perform authentication again, or a content indicating that entry and exit are not possible. As the alert, the alert content may be displayed on the display device 14, an alarm may be sounded, or a content indicating that passage is not possible may be notified by voice. In addition, as a notification to the user, for example, the face authentication ticket gate 10 transmits a notification that authentication has failed to a terminal device such as a smartphone owned by the user. In addition, the face authentication ticket gate 10 may notify the user's terminal device when the user passes through the face authentication ticket gate 10 even though the right of passage determination is NG, or when the user turns back because the right of passage determination is NG.

[0121] When the ID matching unit 16 determines that the User-ID acquired in the process of step St205 exists among the multiple User-IDs based on the narrowed-down candidates acquired in the process of step St203, it picks up the corresponding User-ID. The ID matching unit 16 transmits the picked-up User-ID to the passage right determination server 50. The passage right determination unit 53 of the passage right determination server 50 acquires information corresponding to the User-ID received from the ID matching unit 16 from the information stored in the payment / passenger ticket information database 55. The passage right determination unit 53 determines whether or not the user is allowed to pass through the face authentication ticket gate 10 using the information acquired from the payment / passenger ticket information database 55. The passage right determination unit 53 transmits the determination result of whether or not the user is allowed to pass through the face authentication ticket gate 10 to the processor 15.

[0122] The processor 15 determines whether or not the user is permitted to pass through the face authentication ticket gate 10 based on the result of the determination of the user's right of passage received from the right-of-passage determination unit 53 (step St208). When the processor 15 determines that the user is not permitted to pass through the face authentication ticket gate 10 (step St208, NO), it executes the process of step St207.

[0123] When the processor 15 determines that the user is allowed to pass through the face authentication ticket gate 10 (step St208, YES), the processor 15 creates a personal information table 400 (see FIG. 7) (step St209). The processor 15 stores the created personal information table 400 in the personal information database 18. The processor 15 may temporarily store the created personal information table 400 in the memory 19 instead of in the personal information database 18.

[0124] The same person determination unit 17C starts tracking the user who has been determined to be able to pass (step St210). The same person determination unit 17C calculates a position difference from the face coordinates and body coordinates before and after the update of the captured image, and determines whether the user to be tracked is the same person before and after the update of the captured image.

[0125] The processor 15 displays on the display device a message indicating that the user is permitted to pass through the face authentication ticket gate 10 (step St211). Note that the process of step St211 is not limited to be performed after the process of step St210, and may be performed after the process of step St208, YES.

[0126] The same person determination unit 17C determines whether or not tracking of the user can be continued (step St212). For example, the same person determination unit 17C may be unable to continue tracking the user because the user turns back and exits the face authentication ticket gate 10 and is no longer photographed by the camera 11, or the user overlaps with another person within the angle of view of the camera 11 and cannot be photographed for a certain period of time.

[0127] When the same person determination unit 17C determines that tracking of the user cannot be continued (step St212, NO), the processor 15 starts the process again from the process of step St200.

[0128] When it is determined that tracking of the user can be continued (step St212, YES), the same person determination unit 17C causes the zone determination unit 17D to detect the zone in which the user is located.

[0129] The zone determination unit 17D detects the zone in which the user is located, and detects that the user has moved to the passage determination zone 301 (step St213). When the zone determination unit 17D detects that the user has moved to the passage determination zone 301, that is, that the user has entered the passage determination zone 301, the zone determination unit 17D determines that the user has passed through the face authentication ticket gate 10.

[0130] The processor 15 executes the passage management process (step St214). As part of the passage management process performed by the processor 15, the processor 15 transmits to the passage history server 60 the User-ID of the user who has passed through, the entrance / exit information, the date and time of entrance / exit, and the entry station and exit station information. The passage history server 60 determines the charge amount based on the entry / exit station information received from the processor 15. The charge amount may be determined by the charge management server 70, which obtains the entry / exit station information from the passage history server 60. When the charge management server 70 determines the charge amount, the processor 15 may perform payment processing using payment information previously registered by the user.

[0131] Furthermore, as the passage management process, the processor 15 may issue a notification indicating that the user is permitted to pass through the facial recognition ticket gate 10. As the notification, the processor 15 may display on the display device 14 a message indicating that the user is permitted to pass through the facial recognition ticket gate 10, or may issue a voice message indicating that the user is permitted to pass through the facial recognition ticket gate 10. As the notification, the processor 15 may issue a message to the user's terminal device indicating that the user is permitted to pass through the facial recognition ticket gate 10. The processor 15 may also cause the display device to display an image of the user's face together with the authentication result.

[0132] As a result, the facial recognition ticket gate 10 performs two-factor authentication by matching the face with the User-ID linked to the tag ID detected in the facial recognition ticket gate 10, and further determines whether or not a user who has succeeded in the two-factor authentication has the right to pass through the facial recognition ticket gate 10. The facial recognition ticket gate 10 can perform two-factor authentication of the face image and the tag ID by using information that links the user's own face image registered in advance in the ID management server 40 with the tag ID of the terminal to be used. The facial recognition ticket gate 10 can authenticate users who pass through the facial recognition ticket gate 10 with high accuracy by using the two-factor authentication. In addition, the facial recognition ticket gate 10 determines whether or not the user is allowed to pass through the facial recognition ticket gate 10 after authenticating the user, thereby preventing users who do not have the right to pass through the facial recognition ticket gate 10 from passing through the facial recognition ticket gate 10, and can accurately manage the entrance and exit of users to the station. The facial recognition ticket gate 10 can prevent a user from passing if, for example, the amount of money charged to the terminal is insufficient or if the commuter pass has expired. As described above, the facial recognition ticket gate 10 can simultaneously perform user authentication, passage management, and payment management, thereby enabling users to smoothly enter and exit the station. Since users can enter and exit the station without taking out an IC card or the like and holding it over a designated place, the facial recognition ticket gate 10 can improve user convenience. In addition, the facial recognition ticket gate 10 can reduce congestion near the ticket gates of stations.

[0133] Next, narrowed-down candidates for face matching will be described with reference to Fig. 9. Fig. 9 is a diagram for explaining narrowed-down candidates for face matching.

[0134] Data DA1 represents an example of data of narrowed-down candidates narrowed down as a result of face matching. In data DA1, information linking the face IDs and User-IDs of the narrowed-down candidates is arranged in order of the degree of match in face matching.

[0135] The data DA2 includes information on multiple tag IDs acquired by the ID acquisition reader 12. The information included in the data DA2 is linked to a tag ID, a Received Signal Strength Indicator (hereinafter referred to as "RSSI"), and a User-ID. The RSSI indicates the received signal strength of a signal from a wireless tag acquired by the ID acquisition reader 12. The larger the RSSI value (i.e., the closer to 0), the stronger the received signal strength. The data DA2 is created by the processor 15 by linking the tag ID detected in the detection area 300 (see FIG. 3) of the face recognition ticket gate 10 with the User-ID acquired from the two-factor authentication server 30. When creating the data DA2, the processor 15 may create the data DA2 only for tag IDs whose RSSI values ​​are equal to or greater than a threshold, rather than creating the data for all tag IDs detected in the detection area 300 (see FIG. 3) of the face recognition ticket gate 10.

[0136] The ID matching unit 16 selects a User-ID that matches from the data DA1 and the data DA2, and performs matching of the user. For example, in the example shown in FIG. 9, the User-ID with the fifth highest degree of matching in face matching in the data DA1 matches the data with the tag ID of "0001" in the data DA2. In this way, data with a lower rank, rather than the data with the highest degree of matching in face matching, may match the tag ID detected in the detection area 300 (see FIG. 3) of the face recognition ticket gate 10. In this way, the two-factor authentication system 1 performs matching of the User-ID using multiple candidates narrowed down in order of the degree of matching in face matching, thereby preventing the user associated with the detected tag ID from being overlooked and enabling highly accurate matching.

[0137] Next, an example of the detection process performed by the two-factor authentication system 1 according to the present embodiment will be described with reference to Fig. 10. Fig. 10 is a sequence diagram of an example of the detection process performed by the two-factor authentication system 1 according to the present embodiment. Note that the detailed description of each process related to the sequence diagram of Fig. 10 has been given in Fig. 7, and therefore will not be repeated here.

[0138] The ID acquisition reader 12 receives the tag ID of a user who is located in the verification zone 302 (see FIG. 4) of the face authentication ticket gate 10 (step St300).

[0139] The ID acquisition reader 12 transmits the tag ID received in the process of step St300 to the two-factor authentication server 30, and at the same time requests the User-ID (step St301).

[0140] The two-factor authentication server 30 transmits the User-ID to the processor 15 (step St302).

[0141] The processor 15 receives the User-ID from the two-factor authentication server 30 (step St303).

[0142] In parallel with the process of step St300 by the ID acquisition reader 12, the camera 11 captures an image of the user who enters the detection area 300 (see FIG. 3) of the face authentication ticket gate 10 (step St304).

[0143] The camera 11 transmits the captured image to the processor 15 (step St305). Note that the camera 11 transmits the captured image to the processor 15 at any time after tracking of the user is started in the process of step St317. For example, the camera 11 transmits the captured image to the processor 15 at a time interval set by the business operator.

[0144] The processor 15 detects the face and body of the user from the captured image (step St306) and generates a face image and a body image.

[0145] The processor 15 transmits face image data to the face authentication server 20 and requests face matching (step St307).

[0146] The face authentication server 20 extracts facial features from the face image (step St308).

[0147] The face authentication server 20 performs one-to-N face matching using the face feature amounts extracted in the process of step St308 and the data stored in the face feature amount database 27 (step St309).

[0148] The face authentication server 20 narrows down the faces with the highest degree of match (step St310).

[0149] The face authentication server 20 transmits the narrowed-down candidates narrowed down in the process of step St310 to the processor 15 (step St311).

[0150] The processor 15 receives the result of face matching from the face authentication server 20 (step St312).

[0151] The processor 15 uses the face matching result received in step St312 to match the User-ID associated with the tag ID detected in the detection area 300 received in step St303 (step St313).

[0152] The processor 15 transmits the User-ID checked in the process of step St313 to the right-of-passage determination server 50, and requests the server 50 to determine the right-of-passage of the user (step St314).

[0153] The right-of-passage determination server 50 determines the right of passage of the user (step St315).

[0154] The passage right determination server 50 transmits the determination result of the process of step St315 to the processor 15 (step St316).

[0155] If the determination result of the process in step St316 is that the user has the right of passage, the processor 15 starts tracking the user within the detection area 300 (see FIG. 4) (step St317).

[0156] The processor 15 determines the message or screen data to be displayed on the display device 14 based on the result of the determination in step St316 (step St318).

[0157] The processor 15 transmits the message or screen data determined in the process of step St318 to the display device 14 (step St319).

[0158] The display device 14 displays the message or screen data received in the process of step St319 (step St320).

[0159] The processor 15 detects that the user has entered the passage determination zone 301 (see FIG. 4) by using the captured image transmitted from the camera 11 at any time, and detects that the user has passed through the face authentication ticket gate 10 (step St321).

[0160] The processor 15 transmits the detection result of the process of step St321 to the passage history server 60 (step St322).

[0161] The passage history server 60 stores the passage history of the user in the passage history database 64 (step St323).

[0162] The passage history server 60 transmits the entry / exit information of the user to the passage right determination server 50 (step St324).

[0163] Next, the installation position of the ID acquisition reader 12 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example in which one ID acquisition reader 12 is installed.

[0164] As shown in Fig. 11, for example, one ID acquisition reader 12 is provided near the center of the detection area 300 (see Fig. 3) of the face recognition ticket gate 10. For example, the ID acquisition reader 12 is installed under the floor of the face recognition ticket gate 10. Note that the ID acquisition reader 12 is not limited to being installed under the floor, and may be installed on the ceiling of the face recognition ticket gate 10.

[0165] When the facial recognition ticket gate 10 is provided with only one ID acquisition reader 12, it is necessary to use an ID acquisition reader 12 that can cover the detection area 300 depending on the size of the facial recognition ticket gate 10. Furthermore, when there is only one ID acquisition reader 12, there is a possibility that radio waves from the wireless tag of a user cannot be detected depending on the position or orientation of the wireless tag held by the user. For this reason, it is preferable to have multiple ID acquisition readers 12.

[0166] Next, an example in which four ID acquisition readers 12 are installed will be described with reference to Fig. 12 and Fig. 13. Fig. 12 is a diagram showing an example in which four ID acquisition readers 12 are installed. Fig. 13 is a diagram showing a detectable area when four ID acquisition readers 12 are installed.

[0167] 12, the ID acquisition readers 12 are provided at the four corners of the detection area 300. The locations at which the four ID acquisition readers 12 are provided are not strictly limited to the four corners, and the four ID acquisition readers 12 may be provided to surround the detection area 300. The height at which the ID acquisition readers 12 are provided at the face recognition ticket gate 10 is preferably near the center of the face recognition ticket gate 10, but the ID acquisition readers 12 may be provided at any height, such as below the floor or above the ceiling of the face recognition ticket gate 10. The four ID acquisition readers 12 are each provided so as to face into the detection area 300.

[0168] Area AC represents an area where the ID acquisition reader 12C can detect radio waves emitted from a wireless tag (hereinafter referred to as detectable area). Area AD represents an area where the ID acquisition reader 12C can detect the ID acquisition reader 12D. Area AE represents an area where the ID acquisition reader 12E can detect the ID acquisition reader 12F. Area AF represents an area where the ID acquisition reader 12F can detect the ID acquisition reader 12C.

[0169] As shown in FIG. 13, by providing four ID acquisition readers 12 at the face recognition ticket gate 10, the entire area of ​​the detection area 300 can be covered as a detectable area. The number of ID acquisition readers 12 may be two or three instead of four depending on the size of the detectable area of ​​the ID acquisition readers 12 (i.e., depending on the performance), but it is preferable to have four or more ID acquisition readers 12 in order to cover the detection area 300 as a detectable area. In this way, the face recognition ticket gate 10 can make most of the area of ​​the detection area 300 a detectable area by installing at least one ID acquisition reader 12 at the entrance and one at the exit. The most of the area of ​​the detection area 300 is, for example, 80% or more of the area of ​​the detection area 300.

[0170] Also, the ID acquisition reader 12 may be arranged so that the user always passes through the detectable area while passing through the face recognition ticket gate 10, even if the entire area of ​​the detection area 300 is not necessarily covered as the detectable area. For example, by providing one ID acquisition reader 12 near the center of the detection area 300 as shown in Fig. 10, the user can pass through the detection area while passing through the face recognition ticket gate 10, and the ID acquisition reader 12 can detect the tag ID at any position in the detection area 300.

[0171] Next, the readable orientation of the wireless tag will be described with reference to Fig. 14. Fig. 14 is a diagram for explaining the readable orientation of the wireless tag.

[0172] 12 and 13, it has been explained that the entire detection area 300 can be covered as a detectable area by providing four ID acquisition readers 12. However, there are cases where the ID acquisition reader 12 cannot read an IC tag depending on the orientation of a terminal such as an IC card or a smartphone held by a user.

[0173] The terminal CA 1 is placed perpendicular to the detection plane P of the ID acquisition reader 12 .

[0174] The terminal CA 2 is placed parallel to the detection plane P of the ID acquisition reader 12 .

[0175] The ID acquisition reader 12 cannot read radio waves from a terminal such as terminal CA1 that is oriented perpendicular to the detection surface P. On the other hand, the ID acquisition reader 12 can read radio waves from a terminal such as terminal CA2 that is oriented parallel to the detection surface P. Note that the orientation of the terminal does not need to be strictly parallel, and the ID acquisition reader 12 can read radio waves as long as the terminal is not perpendicular, such as terminal CA1.

[0176] In this way, even if the entire detection area 300 is covered as a detectable area using four ID acquisition readers 12, there are cases where the IC tag cannot be read because radio waves from the wireless tag cannot be received depending on the orientation of the terminal, depending on the type of ID acquisition reader 12. To solve this situation, an example in which eight ID acquisition readers 12 are provided at the face recognition ticket gate 10 is described in FIG.

[0177] 15 is a diagram showing an example in which eight ID acquisition readers 12 are installed. The ID acquisition readers 12 are installed two at each of the four corners of the detection area 300 at different heights. Note that the locations at which the eight ID acquisition readers 12 are installed are not strictly limited to the four corners, and the eight ID acquisition readers 12 may be installed so as to surround the inside of the detection area 300.

[0178] When the positive direction of the Z axis is taken as the up direction, the four ID acquisition readers 12H provided at the upper side of the two ID acquisition readers 12 at different heights are provided so as to face the negative direction of the Z axis. On the other hand, the four ID acquisition readers 12L provided at the lower side are provided so as to face the positive direction of the Z axis. In this way, by using eight ID acquisition readers 12, the face recognition ticket gate 10 can cover the three-dimensional space within the detection region 300 as a detectable area. Note that the number of ID acquisition readers 12 is not limited to eight.

[0179] Next, an example of data of the user management information 500 will be described with reference to Fig. 16. Fig. 16 is a diagram showing an example of data of the user management information 500.

[0180] The user management information 500 is an example of data created based on data input by a user to the face information registration terminal 80 and stored in the user management database 44. In the user management information 500, various information is linked to a User-ID.

[0181] Information that can be linked to a User-ID includes, for example, name, phone number, email address, payment information, commuter pass information, tag ID, and face ID.

[0182] The payment information is, for example, the type of terminal on which the billing process is performed, the card number, whether or not the card can be used, etc. Whether or not the card can be used is, for example, information such as whether or not the card can be used because the balance of the terminal on which the payment process is performed is insufficient or the credit card limit has been exceeded and the card cannot be used.

[0183] The commuter pass information includes the departure station, the arrival station, and the expiration date and time.

[0184] Next, an example of data of the payment / ticket information 600 will be described with reference to Fig. 17. Fig. 17 is a diagram showing an example of data of the payment / ticket information 600.

[0185] The payment / ticket information 600 is stored in the payment / ticket information database 55 of the passage right determination server 50. In the payment / ticket information 600, the User-ID is linked to the entry / exit status, payment information, and commuter pass information. The availability of the payment information indicates whether the user can enter or exit the station, depending on the payment availability status of the terminal where the payment process is performed.

[0186] Next, an example of data of the passage history information 700 will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of data of the passage history information 700.

[0187] The passage history information 700 is stored in the passage history database 64 of the passage history server 60. The passage history information 700 is registered by linking the user's entry / exit status, the date and time of the user's entry / exit, the entry station, the exit station, and the charged amount to the User-ID. If the charged amount is 0 yen, it indicates that the entry station and the exit station are included in the commuter pass section, and no charge has been made.

[0188] (Variation 1) Next, a first modified example of the process of managing the passage of a user through the face authentication ticket gate 10 using two-factor authentication will be described with reference to Fig. 19. Fig. 19 is a flowchart showing a first modified example of the process of managing the passage of a user through the face authentication ticket gate 10 using two-factor authentication. Note that the same processes as those in the flowchart shown in Fig. 8 are denoted by the same reference numerals and will not be described.

[0189] The face recognition ticket gate 10 acquires a User-ID and a Tag ID associated with a narrowed-down candidate for face matching from the face recognition server (step St401).

[0190] The face authentication ticket gate 10 compares the tag ID received in the process of step St204 with the User-ID and the tag ID linked to the narrowed-down candidate acquired in the process of step St401, and determines whether they match (step St402). In other words, the face authentication ticket gate 10 extracts narrowed-down candidates that match the tag ID received in the process of step St204 from the tag IDs linked to the narrowed-down candidates.

[0191] The facial recognition ticket gate 10 compares the tag ID received in the processing of step St204 with the User-ID and tag ID linked to the narrowed-down candidate obtained in the processing of step St401, and if it determines that they do not match (step St402, NO), it executes the processing of step St207.

[0192] The facial recognition ticket gate 10 compares the tag ID received in the processing of step St204 with the User-ID and tag ID linked to the narrowed-down candidate obtained in the processing of step St401, and if it determines that they match (step St402, YES), it transmits the User-ID of the extracted narrowed-down candidate to the passage right determination server 50.

[0193] As a result of the above, the face authentication ticket gate 10 can verify the result of face authentication with the tag ID received from the ID acquisition reader 12, thereby authenticating the user.

[0194] (Variation 2) Next, a second modified example of the process of managing the passage of a user through the face authentication ticket gate 10 using two-factor authentication will be described with reference to Fig. 20. Fig. 20 is a flowchart showing a second modified example of the process of managing the passage of a user through the face authentication ticket gate 10 using two-factor authentication. Note that the same processes as those in the flowchart shown in Fig. 8 are denoted by the same reference numerals and will not be described.

[0195] The face recognition ticket barrier 10 calculates the amount of facial features from the face detection information (step St501).

[0196] The face recognition ticket gate 10 performs face matching of the user using a face image associated with a face ID stored in the face feature amount database 27 and the face feature amount calculated in the process of step St501 (step St502).

[0197] The facial recognition ticket gate 10 performs one-to-N matching using the face IDs of N (N: an integer equal to or greater than 1) registered persons stored in the facial feature database 27. The facial recognition ticket gate 10 calculates the matching rate between the face detection information and the person's face image stored in the facial feature database 27. The facial recognition ticket gate 10 narrows down multiple candidates with high matching rates from among the person's face images stored in the facial feature database 27 in order of high matching rate (step St503).

[0198] The face authentication ticket gate 10 collates the tag ID received in the process of step St204 with the User-ID and the tag ID associated with the narrowed-down candidate acquired in the process of step St503, and determines whether they match (step St504).

[0199] The facial recognition ticket gate 10 compares the tag ID received in the processing of step St204 with the User-ID and tag ID linked to the narrowed-down candidates obtained in the processing of step St503, and if it determines that they do not match (step St504, NO), it executes the processing of step St207.

[0200] The facial recognition ticket gate 10 compares the tag ID received in the processing of step St204 with the User-ID and tag ID linked to the narrowed-down candidate obtained in the processing of step St503, and if it determines that they match (step St504, YES), it uses information from the payment / ticket information database 55 to determine whether the user has the right to pass through the facial recognition ticket gate 10 (step St505).

[0201] When the face authentication ticket gate 10 determines that the user does not have the right to pass through the face authentication ticket gate 10 (step St505, NO), it executes the process of step St207.

[0202] When the face authentication ticket gate 10 determines that the user has the right to pass through the face authentication ticket gate 10 (step St505, YES), it executes the process of step St209.

[0203] (Summary of this embodiment) The above description of the present embodiment discloses the following techniques.

[0204] <Technology 1> The authentication method of this embodiment creates a first facial image by extracting a facial portion from an image of a user passing through a facial recognition ticket gate captured by a camera, compares the first facial image with a second facial image which is an image of the face of a pre-registered person, calculates a match rate between the first facial image and the second facial image, obtains terminal identification information which is identification information of a terminal carried by a person located inside the facial recognition ticket gate, compares the terminal identification information linked to the second facial image with the high match rate with the terminal identification information of the terminal carried by the person, and determines whether or not the user is allowed to pass through the facial recognition ticket gate based on user identification information linked to the compared terminal identification information and including information on whether the user is allowed to pass through the facial recognition ticket gate.

[0205] As a result, the authentication method according to the present embodiment performs face authentication and authentication using terminal identification information, and can authenticate that a user passing through a face authentication ticket gate is a pre-registered registrant. After authenticating a user, the authentication method can determine whether the user is allowed to pass through the face authentication ticket gate, that is, whether the user has the right to pass through the face authentication ticket gate. As a result, the authentication method can prevent a user who does not have the right to enter or exit the face authentication ticket gate from entering or exiting the face authentication ticket gate, and can manage the entry and exit of users through the face authentication ticket gate with high accuracy.

[0206] <Technology 2> In the authentication method described in Technology 1, the second face images of multiple registrants are narrowed down to those with the highest matching rate, and device identification information linked to the narrowed down second face images is extracted that matches the device identification information of a device held by the person.

[0207] Depending on the shooting conditions, the person with the highest matching rate in the face authentication results may be a person other than the user. The authentication method according to the present embodiment narrows down multiple candidates in descending order of face authentication matching rate, and there is a high probability that the user himself / herself is included in the narrowed down candidates, so that the user's face can be authenticated with high accuracy.

[0208] <Technology 3> In the authentication method described in Technology 1 or Technology 2, the facial recognition ticket gate has a matching zone where the user is authenticated using the registration information of the registrant, and a passage determination zone where it is determined whether the user has passed through the facial recognition ticket gate. When it is detected that the user has entered the passage determination zone, it is determined that the user has passed through the facial recognition ticket gate.

[0209] As a result, the authentication method according to the present embodiment can prevent a user who entered a facial recognition ticket gate but then turned back and did not pass through from being mistakenly determined to have entered or exited the facial recognition ticket gate. In other words, the authentication method can manage the entry and exit of users through the facial recognition ticket gate with high accuracy.

[0210] <Technology 4> In the authentication method according to any one of the techniques 1 to 3, the determination of whether or not a user is permitted to pass through the face recognition ticket gate is performed in a matching zone.

[0211] As a result, the authentication method according to the present embodiment can perform all of the following while the user is located in the verification zone: face authentication of the user, verification of the terminal identification information, and determination of whether or not the user has the right to pass. As a result, the authentication method performs the above-mentioned determination after the user passes through the face authentication ticket gate, and can prevent a person who does not have the right to pass from mistakenly passing through the face authentication ticket gate.

[0212] <Technology 5> In the authentication method according to any one of the first to fourth techniques, the result of the determination as to whether or not the user is permitted to pass through the face recognition ticket gate is displayed on a display device.

[0213] As a result, the authentication method according to the present embodiment can assist the user in recognizing whether or not he or she is permitted to pass through the face recognition ticket gate.

[0214] <Technology 6> In the authentication method according to any one of Techniques 1 to 5, the user identification information is information that links the second face image, the terminal identification information, and information on whether or not payment is possible at the face recognition ticket gate.

[0215] As a result, the authentication method of this embodiment can perform two-factor authentication using facial authentication using user identification information and authentication using terminal identification information, and can determine whether or not the authenticated user has the right to pass through a facial authentication gate.

[0216] <Technology 7> In the authentication method described in any one of Technology 1 to Technology 6, when it is determined that a user is allowed to pass through the facial recognition ticket gate, the user's entry / exit status through the facial recognition ticket gate is further linked to user identification information and stored.

[0217] As a result, the authentication method according to this embodiment can prevent users who have no record of entering (i.e., users who entered despite not being qualified to enter, or users who were not captured on camera and therefore not properly authenticated, etc.) from passing through the facial recognition ticket gate to exit.

[0218] <Technology 8> In the authentication method described in any one of Technology 1 to Technology 7, if it is determined that a user is allowed to pass through the facial recognition ticket gate, information about the station where the user entered and exited is further linked to the user identification information and stored.

[0219] As a result, the authentication method according to the present embodiment can determine the charge amount using information on the station where the user enters and exits.

[0220] <Technology 9> In the authentication method according to any one of Techniques 1 to 8, terminal identification information of a terminal carried by a person located inside a face authentication ticket gate is acquired by a plurality of readers.

[0221] As a result, the authentication method according to the present embodiment can obtain terminal identification information regardless of the orientation and position of the terminal held by the user.

[0222] <Technology 10> In the authentication method according to any one of Techniques 1 to 9, at least one reader is installed at each of the entrance and exit of the face recognition ticket gate.

[0223] As a result, the authentication method according to the present embodiment can cover the matching zone of the face recognition ticket gate as an area in which terminal identification information can be detected.

[0224] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0225] The technology disclosed herein is useful as an authentication method, an authentication system, a face recognition ticket gate, and a face recognition server that manages user entrance and exit through a gate with high accuracy. [Explanation of symbols]

[0226] 1. Two-factor authentication system 7 User 10 Facial recognition ticket gate 11,11A,11B Camera 12, 12A, 12B, 12C, 12D, 12E, 12F, 12H, 12L ID acquisition reader 13, 21, 31, 41, 51, 61, 71 Communication I / F 14, 14A, 14B Display device 15,23,32,42,52,62,72 processors 16 ID matching unit 17 Tracking Department 17A Update section 17B Coordinate calculation section 17C Same person determination section 17D Zone Judgment Section 17E Direction determination section 18 Biographical Information Database 19,22,33,43,54,63,73 Memory 20 Face Recognition Server 25 Facial Feature Extraction Unit 26 Face matching unit 27 Facial Feature Database 30 Two-Factor Authentication Server 34 User-ID Database 40 ID Management Server 44 User Management Database 50 Right of way determination server 53 Right of Way Determination Division 55 Payment and ticket information database 60 Passing History Server 64 Passing History Database 70 Billing Management Server 74 Billing Management Database 80 Face information registration terminal 200 Body Images 201 Body coordinates 220 Face Images 221 Face Coordinates 300 detection area 301 Passing Judgment Zone 302 Matching Zone

Claims

1. creating a first face image by extracting a facial portion from an image of a user passing through a face recognition ticket gate captured by a camera; comparing the first face image with a second face image, which is an image of a face of a pre-registered person, and calculating a matching rate between the first face image and the second face image; Acquire terminal identification information that is identification information of a terminal carried by a person located inside the face recognition ticket gate; comparing terminal identification information associated with the second face image having the high matching rate with terminal identification information of a terminal carried by the person; determining whether the user is permitted to pass through the facial recognition ticket gate based on user identification information linked to the collated terminal identification information and including information on whether the user is permitted to pass through the facial recognition ticket gate; Authentication method.

2. Narrowing down the second face images of the plurality of registrants in order of the matching rate; extracting terminal identification information that matches the terminal identification information of a terminal carried by the person from the terminal identification information associated with the plurality of second face images that have been narrowed down; The authentication method according to claim 1 .

3. the face recognition ticket gate has a matching zone in which authentication of the user is performed using registration information of the registrant, and a passage determination zone in which it is determined whether or not the user has passed through the face recognition ticket gate, When it is detected that the user has entered the passage determination zone, it is determined that the user has passed through the face recognition ticket gate. The authentication method according to claim 1 .

4. The determination of whether or not the user is allowed to pass through the face recognition ticket gate is performed in the matching zone. The authentication method according to claim 3.

5. displaying on a display device a determination result as to whether or not the user is permitted to pass through the face authentication ticket gate; The authentication method according to claim 1 .

6. The user identification information is information in which the second face image, the terminal identification information, and information on whether or not payment is possible at the face recognition ticket gate are linked together. The authentication method according to claim 1 .

7. When it is determined that the user is allowed to pass through the face recognition ticket gate, the entry / exit state of the user through the face recognition ticket gate is further linked to the user identification information and stored. The authentication method according to claim 6.

8. If it is determined that the user is allowed to pass through the face recognition ticket gate, information on the station where the user entered and exited is further linked to the user identification information and stored. The authentication method according to claim 7.

9. The terminal identification information of the terminal carried by the person located inside the face authentication ticket gate is acquired by a plurality of readers. The authentication method according to claim 1 .

10. At least one reader is installed at each of the entrance and exit of the face recognition ticket gate. The authentication method according to claim 9.

11. A facial recognition ticket gate through which users pass, A face authentication server that performs face authentication of the user; a right-of-passage determination server that determines whether the user is permitted to pass through the face authentication ticket gate; the face recognition ticket barrier creates a first face image by extracting a face portion from an image of the user captured by a camera; the face authentication server compares the first face image with a second face image, which is an image of a face of a pre-registered person, and calculates a matching rate between the first face image and the second face image; The face recognition ticket gate is Acquire terminal identification information that is identification information of a terminal carried by a person located inside the face recognition ticket gate; comparing terminal identification information associated with the second face image having the high matching rate with terminal identification information of a terminal carried by the person; the right-to-pass determination server determines whether the user is permitted to pass through the facial recognition ticket gate based on user identification information linked to the collated terminal identification information and including information on whether the user is permitted to pass through the facial recognition ticket gate; Authentication system.

12. A reader that acquires terminal identification information that is identification information of a terminal carried by a person passing by; a processor; The processor, creating a first face image by extracting a facial portion from an image of the user captured by a camera; comparing the first face image with a second face image, which is an image of a face of a pre-registered person, and calculating a matching rate between the first face image and the second face image; Acquire terminal identification information that is identification information of a terminal possessed by the person; comparing terminal identification information associated with the second face image having the high matching rate with terminal identification information of a terminal carried by the person; determining whether the user is permitted to pass through based on user identification information linked to the collated terminal identification information and including information on whether the user is permitted to pass through; Facial recognition ticket gate.

13. A facial recognition ticket gate through which a user passes, creating a first face image by extracting a facial portion from an image of the user captured by a camera; obtaining a matching rate between the first face image and the second face image, the matching rate being calculated by comparing the first face image with a second face image, which is an image of the face of a pre-registered person, from a face authentication server that performs face authentication of the user; Acquire terminal identification information that is identification information of a terminal carried by a person located inside the face recognition ticket gate; comparing terminal identification information associated with the second face image having the high matching rate with terminal identification information of a terminal carried by the person; acquiring a determination result as to whether the user is permitted to pass through the facial recognition ticket gate based on user identification information including information as to whether the user is permitted to pass through the facial recognition ticket gate, the determination result being linked to the terminal identification information collated from a right-of-passage determination server that determines whether the user is permitted to pass through the facial recognition ticket gate; Facial recognition ticket gate.

14. A face authentication server that performs face authentication of a user, acquiring a first face image in which a face portion is extracted by a face recognition ticket gate from a photographed image of the user photographed by a camera; Matching the first face image with a second face image which is an image of a face of a pre-registered person; calculating a matching rate between the first face image and the second face image, which is used when the face recognition ticket gate compares terminal identification information, which is identification information of a terminal carried by a person located inside the face recognition ticket gate, with terminal identification information linked to the second face image; Face recognition server.

Images