Authentication method, authentication system, terminal, and server
By acquiring and matching facial information during authentication operations, the system efficiently verifies user identities, addressing the inefficiencies of existing systems by reducing verification time.
Patent Information
- Application Number
- JP2023086479
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-25
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2043-05-25
AI Technical Summary
The existing authentication systems require a significant amount of time for user verification due to the need for capturing images before instructing operations, leading to inefficiencies.
An authentication method and system that acquires facial information from identity verification documents, instructs users to perform authentication operations, and matches facial information during the operation, including specific poses to ensure the face is facing forward for efficient verification.
This approach significantly reduces the time required for user verification by capturing facial images during the authentication process, ensuring quick and accurate identity confirmation.
Smart Images

Figure 2024169208000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an authentication method, an authentication system, a terminal, and a server, and in particular to an authentication method, an authentication system, a terminal, and a server for identity verification. [Background technology]
[0002] 2. Description of the Related Art Conventionally, there has been known an authentication system for user authentication that utilizes a captured image acquired by having a user capture an image of his or her face (Patent Document 1).
[0003] The authentication system in Patent Document 1 mentioned above prevents "impersonation" by using the facial photograph on an identification card such as a driver's license as the captured image by determining whether the person in the captured image of the user and the photograph on the identification card are the same person and determining whether the captured image of the user is a misappropriation of the image on the identification card.
[0004] In the authentication system of Patent Document 1, one or more actions are randomly selected, the user is instructed to perform the action, and a video of the user performing the action is captured. When the video is captured, the authentication system determines whether the user performed the instructed action based on the captured video, and determines whether the user who appears in the captured images before and after the action is instructed is the same person. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent No. 6541140 Summary of the Invention [Problem to be solved by the invention]
[0006] The inventors of the present application came up with the idea of confirming that the provided face information belongs to the user who is to be registered, that is, performing identity verification, when registering face information for face authentication. Therefore, the inventors of the present application devised a method of performing user authentication based on the method described in the authentication system described in the above Patent Document 1, to verify that the provided face information belongs to the user who is to be registered, that is, to perform identity verification.
[0007] However, the inventors of the present application noticed that the authentication system of Patent Document 1 requires a time period for capturing an image before issuing an instruction for an operation, which results in a longer time required to complete authentication.
[0008] Therefore, the main objective of the present disclosure is to provide an authentication method, authentication system, terminal, and server for performing identity verification, which shortens the time required to complete the procedure. [Means for solving the problem]
[0009] The authentication method disclosed herein is an authentication method for identity verification, which acquires first facial information from a user's identification document, instructs the user to perform an authentication operation, determines whether the instructed authentication operation has been performed, compares the first facial information with second facial information acquired by imaging the user to determine whether they match, and authenticates that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and is configured such that the second facial information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0010] The authentication system disclosed herein is an authentication system for identity verification, and includes a first facial information acquisition unit that acquires first facial information from a user's identification document, an operation authentication unit that instructs the user to perform an authentication operation and determines whether the instructed authentication operation has been performed, a face matching unit that compares the first facial information with second facial information acquired by capturing an image of the user and determines whether they match, and an authentication unit that certifies that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and the second facial information is acquired when the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0011] The terminal disclosed herein is a terminal for performing authentication for identity verification, which acquires first facial information from a user's identification document, instructs the user to perform an authentication operation, determines whether the instructed authentication operation has been performed, compares the first facial information with second facial information acquired by photographing the user to determine whether they match, and authenticates that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and is configured such that the second facial information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0012] The server of the present disclosure is a server for performing authentication for identity confirmation, which acquires first facial information from a user's identification document, instructs the user to perform an authentication operation, determines whether the instructed authentication operation has been performed, compares the first facial information with second facial information acquired by photographing the user to determine whether they match, and authenticates that identity confirmation has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and is configured such that the second facial information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward. Effect of the Invention
[0013] According to the present disclosure, an authentication method, authentication system, terminal, and server for performing identity verification can reduce the time required to complete the procedure. [Brief description of the drawings]
[0014] [Figure 1] Functional block diagram of an authentication system and a registration system according to an embodiment. [Diagram 2] Block diagram of the authentication system and registration system [Diagram 3] Example of a pose determination database [Figure 4] Example of an instruction database [Diagram 5] Example of a user database [Figure 6] A sequence diagram showing a process flow up to the start of a service according to the first embodiment. [Figure 7] FIG. 1 shows an example of a document selection screen (A), an example of a PIN input screen (B), and an example of a read execution screen (C). [Figure 8] (A) Example of a registration notification screen, (B) Example of a first pause instruction screen, and (C) Example of a pause authentication error screen. [Figure 9] (A) An example of a second pause instruction screen, (B) an example of a matching execution screen, and (C) an example of an authentication completion screen. [Figure 10] (A) An example of an authentication completion screen, (B) an example of a registration completion screen, and (C) an example of a notification screen notifying the user that user registration was not successful. [Figure 11] 1 is a timing chart showing a flow of user authentication operations and instructions when a sequence with a sequence ID of 1 is selected, and a timing at which a captured image is taken. [Figure 12] 1 is a timing chart showing a flow of user authentication operations and instructions when a sequence with a sequence ID of 3 is selected, and a timing at which a captured image is taken. [Figure 13] A sequence diagram showing a process flow up to the start of a service according to the second embodiment. [Figure 14] A sequence diagram showing a process flow up to the start of a service according to the third embodiment. [Figure 15] Modifications of the authentication and registration systems DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] A first invention made to solve the above problem is an authentication method for identity verification, which acquires first facial information from a user's identification document, instructs the user to perform an authentication operation, determines whether the instructed authentication operation has been performed, compares the first facial information with second facial information acquired by imaging the user to determine whether they match, and authenticates that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and is configured such that the second facial information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0016] According to this, it is possible to perform an authentication operation and identity confirmation using the first face information and the second face information. Also, since the second face information that is easy to match is acquired while the authentication operation is being performed, it is possible to provide an authentication method for identity confirmation that can shorten the time required to complete the procedure.
[0017] In the second invention, when it is authenticated that the identity of the person has been confirmed, at least one of the first face information and the second face information is recorded as face information for face authentication.
[0018] According to this, at least one of the first face information and the second face information can be used as face information for face authentication.
[0019] In the third invention, the authentication operation includes taking two or more poses with different facial orientations.
[0020] This makes it possible to obtain second face information that is easy to match during authentication operations.
[0021] In a fourth aspect of the present invention, the authentication action includes successively assuming two of the above-mentioned poses with the face facing in symmetrical directions with the front of the body as the center.
[0022] This makes it possible to obtain second face information that is easy to match during authentication operations.
[0023] A fifth invention made to solve the above problem is an authentication system for identity verification, comprising: a first face information acquisition unit that acquires first face information from a user's identification document; an operation authentication unit that instructs the user to perform an authentication operation and determines whether the instructed authentication operation has been performed; a face matching unit that compares the first face information with second face information acquired by imaging the user to determine whether they match; and an authentication unit that certifies that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first face information and the second face information match, wherein the second face information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0024] This makes it possible to perform an authentication operation and authentication for identity verification using the first face information and the second face information, and since the second face information is acquired while the authentication operation is being performed, an authentication system can be provided that can shorten the time required to complete the procedure.
[0025] A sixth invention made to solve the above problem is a terminal for performing authentication for identity verification, which acquires first facial information from a user's identification document, instructs the user to perform an authentication operation, determines whether the instructed authentication operation has been performed, compares the first facial information with second facial information acquired by photographing the user to determine whether they match, and authenticates that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and is configured such that the second facial information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0026] According to this, an authentication operation and authentication for identity verification using the first face information and the second face information are performed, and the second face information is acquired while the authentication operation is being performed, so a terminal can be provided that can shorten the time required to complete the procedure.
[0027] In addition, in a seventh invention, a server for performing authentication for identity verification acquires first facial information from a user's identification document, instructs the user to perform an authentication operation, determines whether the instructed authentication operation has been performed, compares the first facial information with second facial information acquired by imaging the user to determine whether they match, and authenticates that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first facial information and the second facial information match, and is configured such that the second facial information is acquired while the authentication operation is being performed, and the authentication operation includes a timing when the face faces forward.
[0028] According to this, an authentication operation and authentication for identity verification using the first face information and the second face information are performed, and the second face information is acquired while the authentication operation is being performed, so a server can be provided that can shorten the time required to complete the procedure.
[0029] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[0030] The authentication method according to the present invention is an authentication method for identity verification, and is implemented in authentication system 1 that authenticates users when providing services, during identity verification at the time of registration.
[0031] (First embodiment) The authentication system 1 acquires a face image from a user, verifies the identity of the user by the authentication method according to the present invention, and performs face authentication by comparing the face image with registered face information. The authentication system 1 permits the user who has been face-authenticated to receive a predetermined service.
[0032] Services that users can receive include, for example, services related to the rental of buildings and parts of buildings (e.g., rooms for holding events, meetings, etc.) and access management, use of tools and equipment, rental of offices and vehicles, etc., reservation of train cars and seats, sales of goods and tickets, use of various facilities including transportation (e.g., stadiums), examinations and treatment at hospitals, logging in to devices such as computers, use of train stations and parcel lockers, etc. The services referred to here may also include authentication services that determine whether or not a service provided by another person (including an individual, corporation, organization, etc.) can be provided by that other person.
[0033] Other services that users can receive could include services that provide online testing, office and car sharing services, and matching services for human resources, skills, and space.
[0034] Businesses that provide services to users, i.e., service businesses, may include individuals, corporations, and organizations.
[0035] The authentication system 1 has a function as a registration system 101 for registering face information of a user who wishes to newly use a service (i.e., a new user). It is desirable that the authentication system 1 has a function for performing identity verification when registering face information, and preventing a third party from registering the face information by impersonating the user.
[0036] Therefore, in the authentication system 1, when a user registers, it is confirmed that the person who is registering is the person himself / herself (i.e., identity verification). Furthermore, the authentication system 1 confirms that the person has not already been registered, i.e., that double registration has not occurred.
[0037] Confirmation of identity (identification) is based on matching the face photo on a photo identification document with the captured face image of the person making the registration, and on confirmation that the person making the registration is not misappropriating another person's image. When the authentication system 1 determines that the two are the same person and confirms that the person making the registration is not misappropriating another person's image, it performs authentication and assumes that the person has been identified. In other words, the authentication system 1 also functions as an authentication system that performs authentication for identity confirmation.
[0038] The identity document may be an ID such as a driver's license, a My Number card, a passport, a residence card, or a student ID card. The identity document preferably has an integrated circuit chip (IC chip) built in, and personal information is recorded on the IC chip. The information recorded on the IC chip can be read by short-range wireless communication. For example, personal information stored on the IC chip of a driver's license may include the name, face photo, date of birth, address, license issue date, validity period, license number, etc. This personal information may be provided to a service provider as necessary.
[0039] To ensure that someone else's image is not being used, a pose check is used in which the person making the registration is instructed to assume a specific pose (posture) and it is checked whether they are doing it correctly.
[0040] The configuration of the authentication system 1 will now be described in detail.
[0041] As shown in FIG. 1, an authentication system 1 includes a user terminal 2 carried by a user, and an authentication server 3 operated and managed by a business that carries out authentication business (authentication service business).
[0042] The user terminal 2 and the authentication server 3 are connected to each other so as to be able to communicate with each other via a communication network 5. The communication network 5 may be the so-called Internet, or may be a closed network. The user terminal 2 and the authentication server 3 may each be connected to the communication network 5 by a wire, or may be connected to the communication network 5 by wireless.
[0043] The authentication server 3 and the user terminal 2 are connected to a provider server 7 managed by a service provider. The provider server 7 executes various processes for providing various services provided by the service provider. The processes provided by the provider server 7 may include processes required for various online services such as online shopping, online reservations, and online tests. The authentication system 1 performs face authentication before a service is provided by the provider server 7, and the provider server 7 provides the service to the authenticated user.
[0044] The user terminal 2 may be an information device having a communication function, such as a smartphone, a tablet terminal, or a PC. The user uses the user terminal 2 carried by the user to perform various processes for face authentication and user registration by communicating with the authentication server 3.
[0045] Additionally, the authentication system 1 may include an authentication machine 2A as the user terminal 2. The authentication machine 2A is installed, for example, in various stores owned by a service provider, and performs various processes such as unlocking doors and gates, and face authentication for electronic payments and the like, and user registration.
[0046] 2, the user terminal 2 includes a processor 11 (CPU, MPU, etc.), a memory 12 such as a volatile memory (RAM) or a non-volatile memory (ROM), a touch panel display 13, a camera 14, a reading device 15, a communication interface 16, and a storage 17. In addition, the user terminal 2 may include a speaker 18.
[0047] The reader 15 reads data stored in an IC chip mounted on the personal identification document by near field communication (hereinafter, NFC). The reader 15 may be a known NFC reader (or reader / writer).
[0048] As shown in FIG. 1, the user terminal 2 includes, as functional units, a communication unit 21, a storage unit 22, a reading unit 23, an imaging unit 24, a notification unit 25, an operation authentication unit 26, and a face image acquisition unit 27.
[0049] The communication unit 21 includes a communication interface 16. The communication unit 21 performs wireless or wired communication with other devices (such as the authentication server 3) via the communication network 5 in accordance with a known communication protocol.
[0050] The storage unit 22 includes the memory 12 and the storage 17 of the user terminal 2. The storage unit 22 stores information required for various processes performed in the user terminal 2.
[0051] The reading unit 23 includes at least one of the reading device 15 and the camera 14. The reading unit 23 reads the face photo (first face information) on the identification document and transmits it to the authentication server 3. In this embodiment, the reading unit 23 reads data corresponding to the face photo from an IC chip mounted on the identification document via the reading device 15 and transmits it to the authentication server 3. The reading unit 23 may also display a notification on the touch panel display 13 instructing the user to capture an image of the identification document, read the face photo on the identification document (e.g., a driver's license), and transmit the face photo and personal information to the authentication server 3. In this embodiment, the reading unit 23 obtains the face photo and name from the IC chip on the identification document and transmits them to the authentication server 3.
[0052] The imaging unit 24 includes a single or multiple cameras 14. The camera 14 is provided near the touch panel display 13 and at a position where it can capture an image of the face of a user operating the user terminal 2. The imaging unit 24 captures an image of the user operating the user terminal 2 to obtain a captured image (still image) or a video.
[0053] The notification unit 25 includes at least one of the touch panel display 13 and the speaker 18, and provides various notifications to the user by displaying the touch panel or by sound emitted from the speaker 18.
[0054] The action authentication unit 26 performs action authentication by instructing the person who has registered to take a predetermined pose (posture) and performing a pose judgment to determine whether the instructed pose (posture) is taken correctly. The action authentication unit 26 is configured by the processor 11 of the user terminal 2 performing processing according to a predetermined program.
[0055] To carry out such processing, the action authentication unit 26 includes a pose instruction unit 31, a pose verification unit 32, and an action verification unit 33.
[0056] The pause instruction unit 31 can be configured by executing a predetermined program by the processor 11 of the user terminal 2. The pause instruction unit 31 obtains the content to be notified from the pause determination database 22A (pause determination DB) stored in the storage unit 22, and issues a notification from the notification unit 25, thereby instructing the user to pause.
[0057] In the pose determination database 22A, an identification number (hereinafter, pose ID) corresponding to a pose (posture) and a corresponding instruction text are recorded. When the pose instruction unit 31 receives a pose ID to be instructed from the action comparison unit 33, the pose instruction unit 31 refers to the pose determination database 22A to acquire the instruction text corresponding to the pose ID, and notifies the notification unit 25 of the acquired instruction text.
[0058] The pose matching unit 32 can be configured by executing a predetermined program by the processor 11 of the user terminal 2. After a pose instruction is given by the pose instruction unit 31, the pose matching unit 32 acquires a captured image from the imaging unit 24 in real time, extracts the user's face direction, hand position, and the like using a known method, and determines whether the user has taken a pose (posture) that matches the pose instruction.
[0059] The pose matching unit 32 can determine the direction of the face (specifically, how many degrees to the left or right, how many degrees up or down, etc.) using, for example, a learning model constructed by machine learning. In addition, the pose matching unit 32 can extract the positions of the right hand, left hand, left shoulder, and right shoulder using, for example, a learning model constructed by machine learning, and determine whether the right hand is located above the right shoulder, whether the left hand is located above the left shoulder, etc.
[0060] In this embodiment, the pose determination database 22A records conditions for determining that a pose instruction has been satisfied (hereinafter, pose determination conditions) in association with the pose ID. When the pose comparison unit 32 acquires a pose ID to be determined from the action comparison unit 33, the pose comparison unit 32 refers to the pose determination database 22A to acquire the pose determination conditions corresponding to the pose ID, and determines whether the acquired pose determination is satisfied. When it is determined that the pose determination conditions are satisfied, the pose comparison unit 32 notifies the action comparison unit 33 that the pose determination conditions are satisfied.
[0061] 3 shows an example of the pose determination database 22A in which pose IDs and the contents of corresponding pose instructions are recorded. In the pose determination database 22A, the pose IDs, information related to the corresponding poses, text indicating the corresponding pose instructions, and conditions for determining that a pose (posture) corresponding to the pose instruction has been taken are recorded in association with each other.
[0062] The action matching unit 33 can be configured by the processor 11 of the user terminal 2 executing processing according to a predetermined program. When the action matching unit 33 receives an instruction to start action authentication from the authentication server 3, it acquires one piece of sequence data from the action instruction database 22B held in the storage unit 22. The sequence data includes information related to one or more pose instructions (posture instructions) and their order.
[0063] 4 shows the action instruction database 22B according to this embodiment. The action instruction database 22B records a plurality of sequence data including a sequence identification number (sequence ID), a first pose identification number (first pose ID), and a second pose identification number (second pose ID). However, the sequence data recorded in the action instruction database 22B always includes the first pose ID. That is, the action instruction database 22B may also record sequence data that does not include the second pose ID (for example, the second pose ID is Null).
[0064] When receiving an instruction to start motion authentication, the motion matching unit 33 randomly selects a sequence from the motion instruction database 22B. Then, the motion matching unit 33 obtains sequence data of the selected sequence from the motion instruction database 22B.
[0065] When the second pose ID is recorded in the sequence data, the action matching unit 33 first (initial) issues a pose instruction corresponding to the first pose ID to the pose instructing unit 31. When the pose matching unit 32 determines that the pose determination condition corresponding to the first pose ID is satisfied, the action matching unit 33 issues a pose instruction corresponding to the second pose ID. When the pose matching unit 32 determines that the pose determination condition corresponding to the first pose ID is satisfied, it determines that a series of actions corresponding to the selected sequence have been performed (i.e., action authentication has been performed), and notifies the authentication server 3 of the determination result. Furthermore, when the action authentication is completed, the pose matching unit 32 notifies the face image acquiring unit 27 that action authentication has been completed.
[0066] When the second pose ID is not recorded in the sequence data, the action comparison unit 33 first (initial) issues a pose instruction corresponding to the first pose ID to the pose instruction unit 31. When the pose comparison unit 32 determines that the pose determination condition corresponding to the first pose ID is satisfied, it determines that a series of actions corresponding to the selected sequence have been performed (i.e., action authentication has been performed), and notifies the face image acquisition unit 27 and the authentication server 3 of the determination result.
[0067] When both the first pose ID and the second pose ID are recorded, the pose corresponding to the first pose ID (hereinafter, the first pose) and the pose corresponding to the second pose ID (the second pose) are set so that the facial orientations are different.
[0068] 3 and 4, in a sequence with a sequence ID of 1, the first pose is a pose (posture) with the face turned to the left, and in the motion authentication, the pose is first instructed to turn to the left. The second pose is a pose (posture) with the face turned to the right, and in a sequence with a sequence ID of 1, after the pose with the face turned to the left is confirmed, the instruction is given to turn to the right.
[0069] In a sequence with a sequence ID of 2, the first pose is a pose (posture) with the face turned to the right, and the second pose is a pose (posture) with the face turned to the left. In a sequence with a sequence ID of 3, the first pose is a pose (posture) with the face turned down, and the second pose is a pose (posture) with the face turned up. In a sequence with a sequence ID of 4, the first pose is a pose (posture) with the face turned up, and the second pose is a pose (posture) with the face turned down.
[0070] In this way, the first pose (first posture) and the second pose (second posture) are preferably set so that the user's face faces the imaging unit 24 (camera 14) when the user moves from the first pose to the second pose. The first pose (first posture) and the second pose (second posture) are preferably set so that the face directions are symmetrical with respect to the front of the body.
[0071] When the second pose ID is not recorded, the pose (posture) corresponding to the first pose ID is set so that the user faces forward with respect to the touch panel of the user terminal 2 (i.e., the user faces forward with respect to the imaging unit 24).
[0072] Specifically, as shown in Figures 3 and 4, in a sequence with a sequence ID of 5, the first pose is set to a pose (posture) of raising the right hand, and in a sequence with a sequence ID of 6, the first pose is set to a pose (posture) of raising the left hand.
[0073] The facial image acquisition unit 27 can be configured by the processor 11 of the user terminal 2 executing a predetermined program and performing processing. The facial image acquisition unit 27 acquires a facial image of the user when the user faces the front of the imaging unit 24 (camera 14) from an image acquired by the imaging unit 24 during the period from when the action authentication starts to when the action authentication is completed, that is, while the authentication action is being performed by the user.
[0074] In this embodiment, after receiving a signal from the authentication server 3 instructing the start of the motion authentication and instructing the user to pose in the first pose (i.e., starting the motion authentication), the face image acquisition unit 27 acquires a captured image from the photographing unit at predetermined intervals (or in real time) until the motion verification unit 33 (motion authentication unit 26) notifies the user that the motion authentication is completed, and determines the direction of the user's face. The face image acquisition unit 27 may determine the direction of the user's face using a learning model constructed by machine learning. When the face image acquisition unit 27 determines that the user's face is facing forward, the face image acquisition unit 27 may store the captured image acquired by the photographing unit as a face image in the storage unit 22. When the direction of the user's face is within 5 degrees left or right of the front of the body, the face image acquisition unit 27 may determine that the user's face is facing forward.
[0075] During the period from when the behavior authentication is started until the result of the behavior authentication is obtained, if the facial image acquisition unit 27 is able to acquire an image (second facial information) captured by the imaging unit 24 when the user's face is facing forward, the acquired captured image is transmitted to the authentication server 3.
[0076] When the facial image acquiring unit 27 is unable to acquire an image of the user's face facing forward by the imaging unit 24 during the period from when the action authentication is started until the result of the action authentication is acquired, the facial image acquiring unit 27 instructs the action authentication unit 26 to once again perform the action authentication by instructing the action authentication unit 26 to perform a pose corresponding to the first pose.
[0077] 2, the authentication server 3 includes a processor 41 (CPU, MPU, etc.), a memory 42 such as a volatile memory (RAM) or a non-volatile memory (ROM), a storage 43, and a communication interface 44. In addition, the authentication server 3 may further include a display (not shown) and an input device (not shown) such as a mouse or a keyboard.
[0078] As shown in FIG. 1, the authentication server 3 includes, as functional units, a communication unit 51, a memory unit 52, a facial feature acquisition unit 53, a facial matching unit 54, a facial information registration unit 55, and a service start processing unit 56.
[0079] The communication unit 51 includes a communication interface 44. The communication unit 51 performs wireless or wired communication with other devices (such as the user terminal 2) via the communication network 5 in accordance with a known communication protocol.
[0080] The storage unit 52 includes the memory 42 and the storage 43. The storage unit 52 stores information required for various processes performed in the authentication server 3.
[0081] The storage unit 52 holds a user database 52A.
[0082] An example of the user database 52A is shown in Fig. 5. In the user database 52A, the identification number (user ID) of a registered user, the name of the corresponding user, and the corresponding face information are stored in association with each other.
[0083] The face information may include information related to at least one of a face feature amount and a face image. In this embodiment, the face information includes an identification number indicating a face feature amount (a face feature amount ID) and an identification number indicating a face image (a face image ID).
[0084] In addition, the user database 52A stores the type of image (either an identification document or a captured photograph) corresponding to the face image ID and the date and time when the image was registered in the user database 52A in association with the user ID. In the example shown in Fig. 5, the type of identification document such as a driver's license or a passport is recorded as the type of image.
[0085] When the facial feature amount acquiring unit 53 acquires the facial image of the identification document and the captured image acquired by the imaging unit 24 from the user terminal 2, it acquires facial feature amounts from each image and outputs them to the face matching unit 54. The facial feature amount is data for determining whether the person appearing in the image is the same person, and may be an amount acquired based on feature points such as the eyes, nose, mouth, and facial contour in the image.
[0086] In addition, when only a face image on an identification document is acquired from the user terminal 2, the face feature amount acquiring unit 53 acquires face feature amounts from each image and outputs them to the face matching unit .
[0087] Similarly, when the facial feature acquisition unit 53 acquires only a facial image acquired by the authentication machine 2A or a captured image acquired by the user terminal 2, it acquires facial features from each image and outputs them to the facial matching unit 54.
[0088] When the face matching unit 54 acquires the face feature amount of the face image of the identification document and the face feature amount of the captured image from the face feature amount acquisition unit 53, it judges whether the person appearing in the face image and the captured image is the same person based on the two face feature amounts. At this time, the face matching unit 54 may compare the two face feature amounts and calculate a matching score. The matching score is a numerical value indicating the degree of agreement (matching degree) between the two face feature amounts, and the higher the numerical value, the more the match is. The face matching unit 54 may judge that the face image of the identification document and the captured image are the same person when the matching score is equal to or greater than a predetermined threshold value. The face matching unit 54 outputs the judgment result of whether the person appearing in the face image of the identification document and the person appearing in the captured image are the same person and the respective face feature amounts to the face information registration unit 55.
[0089] When the face matching unit 54 acquires only the facial feature amount corresponding to the facial image on the identification document from the facial feature amount acquisition unit 53, it refers to the user database 52A stored in the storage unit 52 and extracts the user ID of the user corresponding to the acquired facial feature amount. When the face matching unit 54 cannot acquire the corresponding user ID, it notifies the user terminal 2 that it will start operation authentication. When the face matching unit 54 acquires the corresponding user ID, it determines that the user has already been registered, notifies the user terminal 2 of that effect, and ends the user registration process.
[0090] When the face matching unit 54 acquires only the face feature corresponding to the face image acquired by the authentication machine 2A or the captured image acquired by the user terminal 2 from the face feature acquisition unit 53, the face matching unit 54 refers to the user database 52A stored in the storage unit 52, and extracts the user ID of the user who has the face feature recorded therein that matches the acquired face feature (i.e., the matching score is equal to or greater than a predetermined threshold). When the face matching unit 54 is able to extract the user ID of the corresponding user, it transmits information such as the name of the user to the authentication machine 2A or the user terminal 2, and when it is unable to extract the user ID, it transmits a notice to the effect that matching was not possible to the authentication machine 2A or the user terminal 2.
[0091] The face information registration unit 55 performs authentication that the identity is confirmed when it is determined that the identity is confirmed and that the face photo on the identification document matches the captured image based on the judgment result of the action verification unit 33 (action authentication unit 26) as to whether the action authentication is confirmed or not and the judgment result of the face verification unit 54 as to whether the face photo and the captured image match (they are photos of the same person). That is, the face information registration unit 55 functions as an authentication unit that performs authentication for identity confirmation. When the authentication for identity confirmation is completed, the face information registration unit 55 adds data to the user database 52A held in the storage unit 52 and performs user registration.
[0092] Specifically, when the action comparison unit 33 determines that a series of actions have been performed (a series of poses have been taken) and the face comparison unit 54 determines that the user is the same person, the face information registration unit 55 acquires a new user ID that is not recorded in the user database 52A. Thereafter, the face information registration unit 55 stores the face photo or captured image of the identification document in the storage unit 52 as a face image corresponding to the user ID, and also stores the facial feature amount acquired from the face photo or captured image in the storage unit 52. At this time, the face information registration unit 55 records the identification information (file name, etc.) of the face image stored in the storage unit 52, the identification information (file name, etc.) of the facial feature amount acquired from the face photo or captured image, and personal information such as the name, in the user database 52A in association with the newly acquired user ID, and adds the data.
[0093] When adding data to the user database 52A, the facial information registration unit 55 may record in the user database 52A the type of image to be registered in the user database 52A (captured image, type of identification document, etc.) and the date and time of registration, together with identification information for the facial image and facial features, and personal information such as name, in association with the newly acquired user ID.
[0094] When recording in the user database 52A is completed and data addition is completed (i.e., user registration is completed), the face information registration unit 55 instructs the service start processing unit 56 to start the processing required to provide the service of the service provider.
[0095] However, when there is an input indicating a desire to receive a service from a service provider without registering the face information registration unit 55 in the user terminal 2, the face information registration unit 55 may instruct the service start processing unit 56 to start processing required to provide the service from the service provider, without registering the data in the user database 52A.
[0096] The service start processing unit 56 performs processing required before the service provider starts providing a service to a registered user (hereinafter, service start pre-processing).
[0097] When the authentication server 3 has acquired only the face photograph from the personal identification document (i.e., when the acquisition of the face image by the imaging unit 24 has not been completed), the service start processing unit 56 may notify the user terminal 2 in the pre-start processing to instruct the execution of the processing required for face authentication and motion authentication again. When the pre-start processing is completed, the service start processing unit 56 notifies the server that performs the service provided by the service provider of the completion of the pre-start processing together with the user ID of the user corresponding to the acquired face photograph. Note that, when the service start processing unit 56 performs the service pre-start processing without registering data in the user database 52A, the service provider may provide personal information acquired from the personal identification document instead of the user ID. This allows the user to use the service provided by the service provider after the authentication of the personal identification is completed.
[0098] Next, the operation of the authentication system 1 will be described, firstly, the flow of registration when a user uses a service provided by a service provider for the first time.
[0099] The authentication system 1 executes a predetermined program (authentication program) and performs authentication processing to implement an authentication method for performing authentication for identity confirmation at the time of registration. Figure 6 shows a time chart showing the flow including the authentication processing until the user registration is completed.
[0100] When the registration process is started, the processor 11 of the user terminal 2 starts the personal identification document reading process (S101).
[0101] When the processor 11 of the user terminal 2 starts the identification document reading process, it displays a document selection screen 61 for the identification document on the touch panel display 13. An example of the document selection screen 61 is shown in Fig. 7(A). The document selection screen 61 is provided with one or more buttons 61A for selecting an identification document. In this embodiment, the document selection screen 61 is provided with two buttons 61A: a button 61A for selecting a driver's license as the identification document, and a button 61A for selecting a My Number card.
[0102] When the user touches any of the buttons 61A, the processor 11 of the user terminal 2 displays a PIN input screen 62 including a PIN input field 62A on the touch panel display 13. Fig. 7(B) shows the PIN input screen 62 when the user selects a driver's license as the identification document.
[0103] When the PIN number is entered, the processor 11 of the user terminal 2 displays a reading execution screen 63 on the touch panel display 13. An example of the reading execution screen 63 is shown in Fig. 7(C). In the example shown in Fig. 7(C), a message 63A is displayed on the touch panel display 13 instructing the user to hold the identification document (driver's license) over the reading unit 23. When the reading unit 23 (reader 15, first face information acquisition unit) acquires a facial photograph and personal information such as a name from the IC chip of the identification document, the processor 11 of the user terminal 2 transmits data indicating the read facial photograph, personal information, and type of identification document to the authentication server 3 (S102).
[0104] However, the document selection screen 61 may be configured to allow the user to select a document that does not have an IC chip (e.g., a student ID card, etc.). In this case, the processor 11 of the user terminal 2 may issue a notification instructing the user to take a photograph of the user's face with the imaging unit 24 (camera 14) instead of the facial image on the identification document. The processor 11 of the user terminal 2 may transmit the facial image acquired by the imaging unit 24 to the authentication server 3 instead of the facial photograph on the identification document. In addition, the processor 11 of the user terminal 2 may accept input of personal information such as a name on the touch panel display 13 and transmit the acquired facial image and personal information to the authentication server 3 before transmitting the acquired facial image instead of the facial photograph on the identification document to the authentication server 3.
[0105] When the processor 41 of the authentication server 3 acquires data indicating the facial photograph and personal information on the identification document from the user terminal 2, it determines whether the person appearing in the facial photograph on the identification document is already registered and performs a double registration check (S104).
[0106] Specifically, the processor 41 of the authentication server 3 searches the user database 52A using facial features acquired from the facial photograph on the identification document, and determines whether facial features matching the facial features acquired from the facial photograph are recorded.
[0107] When facial features matching those obtained from a face photograph are recorded in the user database 52A, the processor 41 of the authentication server 3 displays a registration notification screen 65 indicating that the person in the face photograph on the personal identification document has already been registered (corresponding to double registration) on the terminal. This prevents the double registration of a face image or personal information of a user who has already been registered.
[0108] 8(A) shows an example of a registered notification screen 65. In this embodiment, the registered notification screen 65 displays a pop-up 65A including a message indicating that the user is registered and information related to the read user (in this embodiment, the user's name and face photo).
[0109] If the facial feature amount corresponding to the face photograph is not registered in the user database 52A (i.e., if there is no double registration), the processor 41 of the authentication server 3 and the processor 11 of the user terminal 2 transmit an instruction to start the behavior authentication (S104). As a result, the processor 11 (behavior authentication unit 26) of the user terminal 2 starts the behavior authentication process.
[0110] In the action authentication process, first, the processor 11 of the user terminal 2 selects a sequence for performing action authentication from the action instruction database 22B (S105). The sequence may be selected randomly.
[0111] Next, the processor 11 of the user terminal 2 refers to the action instruction database 22B and extracts the first pose ID and the second pose ID associated with the sequence ID determined by the authentication server 3 (S106).
[0112] In the following, an example will be described in which the first pose and the second pose are set (the field for the second pose in the sequence ID is not null).
[0113] Next, the processor 11 of the user terminal 2 refers to the pause determination database 22A based on the extracted first pause ID, and extracts text indicating a pause instruction corresponding to the first pause ID. After that, the processor 11 of the user terminal 2 issues a pause instruction corresponding to the first pause to the user by, for example, displaying the extracted text on the touch panel display 13 or generating a sound corresponding to the extracted text from the speaker 18 (S107).
[0114] 8(B) shows an example of a first pose instruction screen 66 displayed on the touch panel display 13 of the user terminal 2 when a sequence with a sequence ID of 1 is selected and the first pose is a pose facing left. The illustrated first pose instruction screen 66 may display a real-time captured image 66A acquired by the imaging device (camera 14) together with text 66B instructing the user to take a second pose (i.e., a pose (posture) facing right).
[0115] Next, when the pose instruction corresponding to the first pose is completed, processor 11 of user terminal 2 refers to pose determination database 22A based on the extracted first pose ID to acquire a pose determination condition corresponding to the first pose. After that, processor 11 of user terminal 2 determines whether or not the user has taken the first pose (i.e., whether or not the pose determination condition corresponding to the first pose is satisfied) for a predetermined time (e.g., 5 to 10 seconds) based on an image captured by an imaging device (camera 14) (S108).
[0116] 8(C) shows a pose authentication error screen 67 that is displayed on the touch panel display 13 of the user terminal 2 when the user does not take the first pose (No in S108). As shown in FIG 8(C), the pose authentication error screen 67 may display text 67A instructing the user to take the first pose (a pose (posture) facing left).
[0117] 8(C), a resume button 67B is displayed on the pause authentication error screen 67. When the user inputs the resume button 67B, the processor 11 of the user terminal 2 again issues a pause instruction corresponding to the first pause by displaying on the touch panel display 13 or by sound output from the speaker 18 (S107).
[0118] When the processor 11 of the user terminal 2 determines that the user has taken the first pose, that is, that the pose determination condition corresponding to the first pose has been satisfied (cleared) (Yes in S108), the processor 11 of the user terminal 2 refers to the pose determination database 22A based on the extracted second pose ID, and extracts text indicating a pose instruction corresponding to the second pose ID. After that, the processor 11 of the user terminal 2 issues a pose instruction corresponding to the second pose to the user by, for example, displaying the extracted text on the touch panel display 13 or generating a sound corresponding to the extracted text from the speaker 18 (S109).
[0119] 9(A) shows an example of a second pose instruction screen 71 displayed on the touch panel display 13 of the user terminal 2 after a sequence with a sequence ID of 1 is selected and the user takes a first pose. The illustrated second pose instruction screen 71 may display a real-time captured image 71A acquired by the imaging device (camera 14) as well as text 71B instructing the user to take a second pose (i.e., a pose facing to the right).
[0120] Next, when the pose instruction corresponding to the second pose is completed, the processor 11 of the user terminal 2 refers to the pose determination database 22A based on the extracted second pose ID to acquire the pose determination condition corresponding to the second pose. After that, the processor 11 of the user terminal 2 determines whether or not the user has taken the first pose (i.e., whether or not the pose determination condition corresponding to the first pose is satisfied) for a predetermined time (e.g., 5 to 10 seconds) based on the image captured by the imaging device (camera 14) (S110).
[0121] When the processor 11 of the user terminal 2 determines that the user has not taken the second pose (No in S110), the processor 11 of the user terminal 2 again issues a pose instruction corresponding to the first pose by displaying on the touch panel display 13 or by issuing a sound from the speaker 18. At this time, the screen displayed on the user terminal 2 may be similar to the pose authentication error screen 67 shown in FIG. 8(C).
[0122] After issuing a pose instruction corresponding to the first pose (i.e., after starting motion authentication), processor 11 of user terminal 2 attempts to capture an image of the user facing forward using camera 14 (imaging device) until a determination result as to whether or not the user has taken the second pose is obtained (until a result of motion authentication is obtained) (S111).
[0123] At this time, the processor 11 of the user terminal 2 may obtain a video by the camera 14 after the start of the motion authentication until the completion of the motion authentication, extract from the video the timing at which the user faces forward, and obtain an image of the user facing forward. Alternatively, the processor 11 of the user terminal 2 may obtain the user's posture by analyzing real-time captured images by the camera 14 after the start of the motion authentication until the completion of the motion authentication, and when it is possible to obtain the timing at which the user faces forward, it may take an image by the camera 14 and obtain the captured image. The processor 11 of the user terminal 2 may determine that the user faces forward when the direction of the user's face is within 5 degrees left or right of the shooting direction (optical axis) of the camera 14.
[0124] When the processor 11 of the user terminal 2 determines that the user has taken the second pose (Yes in S110), it determines whether or not a captured image in which the user faces forward (i.e., a captured image that can be used for face matching) has been acquired by the camera 14 after instructing the user to take the first pose and before acquiring the determination result of whether or not the user has taken the second pose (S112). At this time, the processor 11 of the user terminal 2 may also determine whether or not the captured image is suitable for face matching (for example, it is out of focus, etc.).
[0125] When the processor 11 of the user terminal 2 determines that an image of the user facing forward has been acquired (an image suitable for face matching has been acquired) (Yes in S112), it transmits the image acquired by the camera 14 and a notification that the operation has been authenticated to the authentication server 3 (S113).
[0126] If the processor 11 of the user terminal 2 has instructed the user to assume the first pose and has not obtained an image of the user facing forward before obtaining a determination result as to whether the user has assumed the second pose (No in S112), the processor 11 of the user terminal 2 re-executes the action authentication process from the pose instruction for the first pose (S107).
[0127] At this time, the processor 11 of the user terminal 2 also determines whether the image at the moment the user faces forward is suitable for face matching (for example, it is out of focus, etc.), and if it is determined to be unsuitable, it may execute the motion authentication process again from the pose instruction for the first pose.
[0128] When the processor 41 (face matching unit 54) of the authentication server 3 receives from the user terminal 2 a captured image acquired by an imaging device and a notification that the motion authentication has been performed, the processor 41 (face matching unit 54) compares the captured image with the face photograph on the identification document (S114). Specifically, the processor 41 of the authentication server 3 extracts facial features from the captured image and the face photograph on the identification document. The authentication server 3 then compares the two acquired facial features to determine whether the person appearing in the captured image and the person appearing in the face photograph are the same person. At this time, the processor 41 of the authentication server 3 may obtain a matching score for the two facial features, and when the matching score is equal to or greater than a predetermined threshold, determine that the person appearing in the captured image and the person appearing in the face photograph on the identification document are the same person.
[0129] 9(B) shows an example of a matching execution screen 72 that is displayed on the touch panel display 13 until face matching is completed after the user terminal 2 transmits a captured image and a notification that the operation authentication has been completed to the authentication server 3. As shown in FIG. 9(B), the matching execution screen 72 may display a captured image and a loading animation 72A superimposed thereon.
[0130] The processor 41 (face information registration unit 55, authentication unit) of the authentication server 3 determines that the authentication for identity verification has been performed when the person in the captured image and the person in the face photograph are the same person after the motion authentication (pose confirmation) (S110) is performed. Thereafter, the processor 41 of the authentication server 3 performs user registration by adding data to the user database 52A held in the storage unit 52 (S114).
[0131] Specifically, the processor 41 of the authentication server 3 acquires a new user ID, and adds either one of the facial feature amounts acquired from the captured image or the facial feature amounts acquired from the face photograph on the identification document, and the name read from the identification document, to the user database 52A in association with the user ID. At the same time, the processor 41 of the authentication server 3 also adds either one of the captured image or the face photograph on the identification document, the registration date and time, and the type of image to be added to the user database 52A in association with the newly acquired user ID.
[0132] In another embodiment, the processor 41 of the authentication server 3 transmits a notification to the user terminal 2 indicating that authentication has been successful. When the user terminal 2 receives the notification indicating that authentication has been successful, it displays an authentication completion screen 73 on the touch panel display 13. On the authentication completion screen 73, text 73A indicating that authentication has been successful and a registration button 73B are displayed. When an input to the registration button 73B is detected, the user terminal 2 notifies that registration has been permitted. When the authentication server 3 receives the notification indicating that registration has been permitted, it associates either one of the captured image and the facial photograph on the identification document, and the registration date and time with the newly acquired user ID, and adds them to the user database 52A, thereby performing user registration.
[0133] Furthermore, the authentication completion screen 73 displayed by the user terminal 2 may display a start use button with the words "Start using the service without registering" along with (or instead of) the registration button 73B. When the user touches the start use button, an instruction is given to the service start processing unit 56 to start processing required for providing the service of the service provider, without registering data in the user database 52A. This allows the user to receive the service without registering as a user.
[0134] Fig. 9(C) shows an example of an authentication completion screen 73. The authentication completion screen 73 shown in Fig. 9(C) displays text 73A indicating that authentication has been successful, a registration button 73B, a face image 73C read from an identification document, and a captured image 73D acquired by the camera 14 (imaging device).
[0135] Furthermore, in another embodiment, the processor 11 of the user terminal 2 displays, on the touch panel, a selection screen for an image to be registered (or recorded) in the user database 52A, and accepts the user's selection of an image to be registered.
[0136] FIG. 10(A) shows an example of an authentication completion screen 75 in the case of accepting a user selection of an image to be registered. On the authentication completion screen 75, similar to the example shown in FIG. 9(C), text 75A indicating that authentication has been successful, a registration button 75B, a facial photograph 75C on the identification document, and a captured image 75D acquired by the camera 14 (imaging device) are displayed. In the example of FIG. 10(A), when the user touches either the facial photograph 75C or the captured image 75D, the frame (circle) indicating the boundary of the touched image becomes thicker and the touched image is selected. In the example of the authentication completion screen 75 shown in FIG. 10(A), the facial photograph 75C on the identification document is selected. After that, when the user touches the registration button 75B, information indicating the selected image is transmitted to the user terminal 2, and the processor 41 of the authentication server 3 adds either one of the selected captured image and the facial photograph on the identification document to the user database 52A, and performs user registration.
[0137] Alternatively, the processor 41 of the authentication server 3 may compare the captured image with the facial photograph on the identification document based on various elements such as image quality, facial expression, background, reflection of objects other than a person, etc., and store the one suitable for matching in the user database 52A. However, the storage destination of these images is not limited to the authentication server 3, and these images may be stored in the user terminal 2, the provider server 7, etc.
[0138] When the processor 41 of the authentication server 3 finishes adding the user to the user database 52A and the user registration is complete, the processor 41 transmits a notification that the registration is complete to the user terminal 2 (S115). As a result, a registration completion screen 76 indicating that the user registration is complete is displayed on the touch panel display 13 of the user terminal 2 (S116).
[0139] An example of a registration completion screen 76 is shown in Fig. 10(B). The registration completion screen 76 displays a facial photograph 76A on the identification document and a captured image 76B acquired by the camera 14 (imaging device), and may also display personal information read from the identification document. In the example of the registration completion screen 76 shown in Fig. 10(B), text 76C and an image 76D are displayed that indicate information (name, registration date, registered image, and type of image) recorded in the user database 52A, including a portion of the read personal information.
[0140] When the processor 41 of the authentication server 3 is unable to determine that the person in the captured image and the person in the facial photograph are the same person, the processor 41 notifies the user terminal 2 that the identity verification was unsuccessful. As a result, a notification screen 77 is displayed on the touch panel display 13 of the user terminal 2 to notify that the user registration was unsuccessful.
[0141] Fig. 10(C) shows an example of a notification screen 77 notifying that user registration has failed. The notification screen 77 shown in Fig. 10(C) displays a face photo 77A obtained from an identification document, a captured image 77B, and a notification 77C notifying that face matching has failed. In addition, the notification screen 77 shown in Fig. 10(C) displays back and resume buttons 77D for resuming the registration process, and an end button 77E for ending the registration process.
[0142] In this way, in the authentication system 1, the reading unit 23 that acquires a face photograph (first face information) from the user's identification document, the operation authentication unit 26 that performs operation authentication, the face matching unit 54 that matches the face photograph with a captured image captured during the authentication operation, and the face information registration unit 55 as an authentication unit that performs authentication for identity verification constitute a system that performs authentication for identity verification. Furthermore, after identity verification, the authentication system 1 also functions as a registration system 101 that executes a program for implementing a registration method for registering face information, registers face information for face authentication, and performs user registration.
[0143] After completing the registration process, the processor 41 of the authentication server 3 executes a pre-service start process as necessary (S117). The pre-service start process is a process required for the provider server 7 to provide a service to a registered user after the face information is registered, and is determined appropriately according to the content of the provider's service, etc.
[0144] For example, in the pre-service start processing, the authentication server 3 may transmit registered personal information (such as name) and information related to a facial image to the provider server 7. Also, in the pre-service start processing, the authentication server 3 may cause the user terminal 2 to display a notification urging the user to capture an image of his or her face, and perform facial authentication by comparing the captured image again with facial feature amounts stored in the user database 52A.
[0145] When the pre-service start process is completed, the authentication server 3 transmits a completion notification of the pre-service start process to the provider server 7 (S118). Upon receiving the completion notification of the pre-service start process, the provider server 7 provides a service (e.g., online shopping, rental car service reservation, etc.) to the user who has completed registration (S119).
[0146] The authentication system 1 also performs processing related to face authentication (hereinafter, face authentication processing) for a registered user to receive a service from a service provider.
[0147] The face authentication process is performed by cooperation between the storage unit 52 and face matching unit 54 of the authentication server 3 and the imaging unit 24 of the user terminal 2. That is, the storage unit 52, the face matching unit 54, and the imaging unit 24 configure an authentication system 1 for performing face authentication of a user.
[0148] In this embodiment, the authentication system 1 performs face authentication based on face features. Therefore, the face authentication process is performed by the memory unit 52, face matching unit 54, imaging unit 24, and face feature acquisition unit 53 working together. However, face authentication may be performed based on any known method. Therefore, the face feature acquisition unit 53 is not essential for the authentication system 1, and in cases where face authentication is not based on face features, the authentication system 1 may be configured by the memory unit 52, face matching unit 54, and imaging unit 24.
[0149] During face authentication, a notification urging the user to capture an image of his or her face is displayed on the touch panel display 13 of the user terminal 2. When the user captures an image of his or her own face with the imaging unit 24 and performs a predetermined operation input on the touch panel display 13, the captured image acquired by the imaging unit 24 is transmitted to the authentication server 3.
[0150] When the captured image is transmitted to the authentication server 3, the face matching unit 54 compares the captured image with the face information stored in the user database 52A to determine whether or not there is a person among the registered users who is identical to the person appearing in the captured image.
[0151] In this embodiment, when a captured image is transmitted to the authentication server 3, the facial feature amount acquiring unit 53 acquires facial feature amounts of a face appearing in the captured image from the captured image. Thereafter, the face matching unit 54 uses the facial feature amounts acquired by the facial feature amount acquiring unit to determine whether or not a person identical to the person appearing in the captured image is present among registered users.
[0152] When the face matching unit 54 determines that a person identical to the person appearing in the captured image is present among the registered users, it transmits a notification to the user terminal 2 that the user has been authenticated, and notifies the provider server 7 of the user ID of the registered user corresponding to the user. As a result, a notification that face authentication has been performed is displayed on the user terminal 2, and the user can use the services provided by the provider server 7.
[0153] When the face matching unit 54 determines that the same person as the registered user does not exist in the captured image, it transmits a notification that the user has been authenticated to the user terminal 2. As a result, a notification that face authentication has not been performed is displayed on the user terminal 2.
[0154] The face authentication process is also performed in the authentication device 2A as the user terminal 2. As in the case where the user terminal 2 is a smartphone, when the imaging unit 24 of the authentication device 2A acquires a captured image including the user's face, the image is transmitted to the authentication server 3, and the face matching unit 54 determines whether or not a person identical to the person appearing in the captured image is present among the registered users. If the same person is present, various displays and processes are performed, such as a notification that face authentication has been performed being displayed on the authentication device 2A.
[0155] Next, the effects of the authentication system 1 configured as above will be described.
[0156] In the authentication system 1, when it is determined that the face photograph (first face information) read from the identification document matches the captured image (second face image) and that the user has taken a pose (posture) in accordance with the pose instruction, the face information is added to the user database 52A and the user is registered. As shown in Fig. 6, the captured image is acquired by the imaging device (imaging unit 24) while the user is performing an action (authentication action) in accordance with the pose instruction.
[0157] As shown in Fig. 6, while the user is performing an action (authentication action) in accordance with a pause instruction, a captured image is acquired so as to include the user's face, and facial authentication is performed using the captured image. Therefore, when registering face information to be used for facial authentication, the user does not need to capture an image of his or her own face separately, and the time required to complete the registration procedure can be shortened.
[0158] FIG. 11 shows the flow of actions that the user may perform when the sequence with sequence ID 1 (see FIG. 4) is selected and an action instruction is given.
[0159] In a sequence with a sequence ID of 1, the user is first instructed to assume a pose facing left (first pose). Once it is confirmed that the user has turned left, the user is instructed to assume a pose facing right (second pose).
[0160] As shown in Fig. 11, after the user takes a pose facing left (first pose) and before taking a pose facing right (second pose), there is a timing when the user's face faces forward (see the box marked "shooting execution" in Fig. 11). This allows the camera 14 to take a photograph at that timing, and obtain a captured image of the user's face facing forward, making it possible to appropriately match the face photograph read from the identification document at the time of registration.
[0161] FIG. 12 shows the flow of actions that the user is likely to perform when the sequence with sequence ID 3 (see FIG. 4) is selected and an action instruction is given.
[0162] In a sequence with a sequence ID of 3, the user is first instructed to assume a pose looking down (first pose). When it is confirmed that the user is looking down, the user is instructed to assume a pose looking up (second pose).
[0163] As shown in Fig. 12, after the user takes a pose looking down (first pose) and before taking a pose looking up (second pose), there is a timing when the user's face faces forward (see the box marked "shooting execution" in Fig. 12). This makes it possible to obtain a captured image in which the user's face faces forward, enabling appropriate matching with the facial photo read from the identification document at the time of registration.
[0164] As shown in Fig. 3 and Fig. 4, in sequences with sequence IDs 1 to 4, the authentication operation includes successively taking two poses (postures) with different facial orientations. Furthermore, these two poses (postures) are configured so that the face faces in symmetrical directions with the front of the body as the center. Therefore, as can be seen from Fig. 11 and Fig. 12, there is a timing when the user's face faces forward during the authentication operation. This makes it possible to easily obtain a captured image in which the user's face faces forward, enabling appropriate matching with the facial photo read from the identification document at the time of registration.
[0165] As shown in Fig. 3 and Fig. 4, in sequences with sequence IDs 5 and 6, a pose instruction is issued to raise the left or right hand. In the case of raising the left or right hand, it is predicted that the user will take the corresponding pose (posture) while facing the user terminal 2. Therefore, the authentication operation includes a timing when the user's face faces forward. Therefore, it is possible to obtain a captured image in which the user's face faces forward, making it possible to appropriately match the face with the facial photograph read from the identification document at the time of registration.
[0166] Second embodiment The authentication system 1 according to the second embodiment is different from the authentication system 1 according to the first embodiment in the sequence of the registration process, but other configurations are similar, so that the description will be omitted.
[0167] 13 shows a sequence of the registration process according to the second embodiment, in which the selected sequence includes two pauses. When a sequence is selected in the user terminal 2, a pause instruction is issued.
[0168] As shown in Fig. 6, in the first embodiment, a pause instruction for a first pose is issued (S017), and when the corresponding first pose is taken (Yes in S108), a pause instruction for a second pose is issued (S109). On the other hand, as shown in Fig. 13, in the second embodiment, the processor 11 of the user terminal 2 issues a pause instruction for the first pose (S017) and a pause instruction for the second pose (S109) in that order, and then, after a predetermined time (for example, 5 to 10 seconds) has elapsed, determines whether the pause condition is satisfied (S201).
[0169] However, the pose instruction for the first pose and the pose instruction for the second pose may be issued in order or simultaneously.
[0170] The processor 11 of the user terminal 2 issues multiple pause instructions, and comprehensively determines whether the pause condition is satisfied based on the determination result of whether the corresponding pauses have been taken. For example, the processor 11 of the user terminal 2 may determine that the pause condition is satisfied when all the pauses corresponding to the respective pause instructions have been taken, or the processor 11 of the user terminal 2 may determine that the pause condition is satisfied when a rate at which the pauses corresponding to the pause instructions have been taken is equal to or greater than a certain rate. Additionally, the processor 11 of the user terminal 2 may determine that the pause condition is satisfied when at least a portion of the pauses corresponding to the pause instructions have been taken.
[0171] In this embodiment, the processor 11 of the user terminal 2 determines that the pause condition is met when either the first pose is taken in response to a first pose instruction (hereinafter, the first pause condition) or the second pose is taken in response to a second pose instruction (hereinafter, the second pause condition) is met. Alternatively, the processor 11 of the user terminal 2 may determine that the pause condition is met when both the first pause condition and the second pause condition are met.
[0172] After the first pose instruction (S107) is given, until it is determined that the pose condition is satisfied (S112), the processor 11 of the user terminal 2 uses the camera 14 to acquire a captured image at the time when the user faces forward (S111).
[0173] When it is determined that the pose condition is satisfied, the processor 11 of the user terminal 2 determines whether the captured image acquired by the camera 14 is an image in which the user faces forward and face matching is possible (S112). When it is determined that the image is not face matching possible (No in S112), the processor 11 of the user terminal 2 issues a pose instruction (first pose instruction and second pose instruction) again (S107). When it is determined that the image is face matching possible (Yes in S112), the processor 11 of the user terminal 2 transmits a notification that the action authentication has been performed and the captured image to the authentication server 3, as in the first embodiment (S113).
[0174] When the processor 41 of the authentication server 3 receives the notification of the successful operation authentication and the captured image, it extracts facial features from the face photo (first face information) on the identification document and the captured image (second face information) and performs face matching to determine whether or not they are the same person (S114). If it is determined that they are the same person, the processor 41 of the authentication server 3 performs user registration by adding at least one of the face photo on the identification document or the captured image, its type, at least one of the extracted facial features, and personal information such as the name to the user database 52A (S114).
[0175] When the processor 41 of the authentication server 3 finishes adding the user to the user database 52A and the user registration is completed, similarly to the first embodiment, it transmits a notification that the registration is completed to the user terminal 2 (S115). As a result, a registration completion screen 76 indicating that the user registration is completed is displayed on the touch panel display 13 of the user terminal 2 (S116).
[0176] In the authentication system 1 according to the second embodiment, too, a captured image is acquired by the imaging device (imaging unit 24) while the user is performing an action (authentication action) in accordance with a pause instruction. Therefore, when registering face information to be used for face authentication, the user does not need to capture an image of his or her own face separately, and the time required to complete the registration procedure can be shortened.
[0177] Third embodiment The authentication system 1 according to the third embodiment is different from the authentication system 1 according to the first embodiment in the sequence of the registration process, but other configurations are similar, so that the description will be omitted.
[0178] 13 shows a sequence of the registration process according to the second embodiment, in which the selected sequence includes two pauses. When a sequence is selected in the user terminal 2, a pause instruction is issued.
[0179] 14 shows a sequence of the registration process according to the third embodiment, in which the selected sequence includes two pauses. When a sequence is selected in the user terminal 2, a pause instruction is issued.
[0180] 14, in the third embodiment, the processor 11 of the user terminal 2 sequentially issues a pose instruction for a first pose (S105) and a pose instruction for a second pose (S106). After issuing the pose instruction for the first pose (S105) and before a predetermined time (e.g., 5 to 10 seconds) has elapsed after the pose instruction for the second pose (S106), the processor 11 of the user terminal 2 acquires a captured image at the timing when the user faces forward by the camera 14 (S111).
[0181] Next, the processor 11 of the user terminal 2 judges whether the captured image acquired by the camera 14 is an image in which the user is facing forward and face matching is possible (S112). If it is judged that the image is not face matching possible (No in S112), the processor 11 of the user terminal 2 issues a pose instruction (first pose instruction and second pose instruction) again (S107, S109). If it is judged that the image is face matching possible, the processor 11 of the user terminal 2 transmits the captured image to the authentication server 3 (S301).
[0182] When the processor 41 of the authentication server 3 acquires the captured image from the processor 11 of the user terminal 2, the processor 41 extracts facial features from the facial photograph on the identification document and the captured image, respectively, and performs face matching to determine whether or not they are the same person (S302). The processor 41 of the authentication server 3 transmits the result of the face matching to the user terminal 2 (S303).
[0183] When the processor 11 of the user terminal 2 receives the results of the face matching from the authentication server 3, it determines whether the first pose (first posture) and the second pose (second posture) have been taken and whether the facial photograph (first facial information) on the identification document and the captured image (second facial information) match, i.e., whether the conditions required for registration (hereinafter, registration conditions) are met (cleared) (S304).
[0184] When the processor 11 of the user terminal 2 determines that the conditions required for registration (hereinafter, the registration conditions) are satisfied (cleared) (Yes in S304), it transmits a registration permission notification to the authentication server 3 (S305). When the processor 11 of the user terminal 2 determines that the registration conditions are not satisfied (No in S304), it again issues a pause instruction (first pause instruction and second pause instruction) (S107).
[0185] When the processor 41 of the authentication server 3 receives the registration permission notification, it adds at least one of the facial photograph or the captured photograph on the identification document, its type, at least one of the extracted facial features, and personal information such as the name to the user database 52A, and registers the user (S306).
[0186] When the processor 41 of the authentication server 3 finishes adding the user to the user database 52A and the user registration is completed, similarly to the first embodiment, it transmits a notification that the registration is completed to the user terminal 2 (S115). As a result, a registration completion screen 76 indicating that the user registration is completed is displayed on the touch panel display 13 of the user terminal 2 (S116).
[0187] In the authentication system 1 according to the third embodiment, too, a captured image is acquired by the camera 14 (imaging device, imaging unit 24) while the user is performing an action (authentication action) in accordance with a pause instruction. Therefore, when registering face information to be used for face authentication, the user does not need to capture an image of his or her own face separately, and the time required to complete the registration procedure can be shortened.
[0188] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments.
[0189] In the above first to third embodiments, an example has been described in which the action instruction database 22B and the pose determination database 22A are stored in the storage unit 22 of the user terminal 2, but as shown in Fig. 15, they may be stored in the storage unit 52 of the authentication server 3. In addition, in the above first to third embodiments, the action authentication unit 26 is provided in the user terminal 2, but the action authentication unit 26 may be provided in the authentication server 3.
[0190] 4 shows an example in which only sequences instructing one or two poses are recorded in the action instruction database 22B, but the action instruction database 22B may contain sequences for two or more poses. However, in order to obtain a captured image suitable for face authentication, it is preferable that the action instruction database 22B records a sequence in which the face faces forward when a series of poses (postures) are taken (i.e., when an authentication action is performed).
[0191] In the above embodiment, the processor 11 of the user terminal 2 is configured to acquire a captured image at the timing when the user faces forward, but if the processor 11 of the user terminal 2 cannot acquire a captured image at the timing when the user faces forward, the processor 11 of the user terminal 2 may generate a captured image at the timing when the user faces forward from multiple captured images. In this case, the processor 11 of the user terminal 2 may generate a captured image at the timing when the user faces forward using captured images in which the user faces in a direction symmetrical to the front. [Industrial Applicability]
[0192] The authentication method, authentication system, terminal, and server disclosed herein are useful as an authentication method, authentication system, terminal, and server for identity verification, and for shortening the time required to complete a procedure. [Explanation of symbols]
[0193] 1: Authentication system 2: User terminal 2A: Authentication machine 3: Authentication server 5: Communication network 7: Provider server 11: Processor 12: Memory 13: Touch panel display 14: Camera 15: Reading device 16: Communication interface 17: Storage 18: Speaker 21: Communications Department 22: Storage section 22A: Pose determination database 22B: Operation instruction database 23: Reading unit 24: Imaging unit 25:Notification Department 26: Operation authentication section 27: Face image acquisition unit 31: Pause indicator 32: Pose matching section 33: Motion verification section 41: Processor 42: Memory 43: Storage 44: Communication interface 51: Communications Department 52: Storage section 52A: User database 53: Facial feature acquisition unit 54: Face matching unit 55: Face information registration unit (authentication unit) 56: Service start processing unit 61: Document selection screen 61A: Button 62: PIN entry screen 62A: PIN code entry field 63: Reading execution screen 63A: Message 65: Registered notification screen 65A: Pop-up 66: First pause instruction screen 66A: Captured image 66B: Text 67: Pause authentication error screen 67A:Text 67B:Resume button 71: Second pause instruction screen 71A: Captured image 71B: Text 72: Verification execution screen 72A: Loading animation 73: Authentication completion screen 73A:Text 73B:Register button 73C:Facial image 73D: Captured image 75: Authentication completion screen 75A:Text 75B: Registration button 75C:Face photo 75D: Captured image 76: Registration completion screen 76A:Face photo 76B: Captured image 76C: Text 76D:Image 77: Notification screen 77A:Face photo 77B: Captured image 77C :Notification 77D:Resume button 77E:Exit button 101: Registration System
Claims
1. An authentication method for identity verification, comprising: Acquire first face information from a user's identification document; instructing the user to perform an authentication operation, and determining whether the instructed authentication operation has been performed; collating the first face information with second face information acquired by capturing an image of the user to determine whether they match; when it is determined that the instructed authentication operation has been performed and that the first face information and the second face information match, it is determined that the identity of the person has been confirmed; the second face information is acquired while the authentication operation is being performed, The authentication method includes a timing when the face faces forward in the authentication action.
2. The authentication method according to claim 1 , further comprising the steps of: recording at least one of the first face information and the second face information as face information for face authentication when it is determined that the person is identified.
3. The authentication method according to claim 1 , wherein the authentication action includes assuming two or more poses with different facial orientations.
4. The authentication method according to claim 3 , wherein the authentication action includes successively assuming two of the poses in which the face faces in symmetrical directions with respect to the front of the body.
5. An authentication system for identity verification, a first face information acquisition unit that acquires first face information from a user's identification document; an operation authentication unit that instructs the user to perform an authentication operation and determines whether the instructed authentication operation has been performed; a face matching unit that compares the first face information with second face information acquired by capturing an image of the user to determine whether they match; an authentication unit that certifies that identity verification has been performed when it is determined that the instructed authentication operation has been performed and that the first face information and the second face information match, The second face information is acquired when the authentication operation is being performed, and the authentication operation includes a timing when the face turns forward.
6. A terminal for performing authentication for identity verification, Acquire first face information from a user's identification document; instructing the user to perform an authentication operation, and determining whether the instructed authentication operation has been performed; collating the first face information with second face information acquired by capturing an image of the user to determine whether they match; when it is determined that the instructed authentication operation has been performed and that the first face information and the second face information match, it is determined that the identity of the person has been confirmed; the second face information is acquired while the authentication operation is being performed, The authentication operation of the terminal includes a timing when the face faces forward.
7. A server for performing authentication for identity verification, Acquire first face information from a user's identification document; instructing the user to perform an authentication operation, and determining whether the instructed authentication operation has been performed; collating the first face information with second face information acquired by capturing an image of the user to determine whether they match; when it is determined that the instructed authentication operation has been performed and that the first face information and the second face information match, it is determined that the identity of the person has been confirmed; the second face information is acquired while the authentication operation is being performed, The authentication operation includes timing when the face faces forward.
Citation Information
Patent Citations
Identity verification program, identity verification method and information processing apparatus
JP2020064541A
Method and system for verifying users
JP2021099802A
Information processing program, information processing method, and information processing apparatus
JP2022143810A
Personal authentication system
JP2023054831A
Facial recognition
US20140016837A1
Cited By
Biological information registration device, biological information registration method, and recording medium
WO2026185999A1
