Personal identification device and personal identification method

Abstract

To provide a method that improves a face checkup rate in identity confirmation by an own operation of a user when the user attempts to execute a trade that needs identity confirmation.SOLUTION: An identity confirmation device 100 includes: a display operating unit 102 that outputs a guidance for a user at a predetermined timing; an identity information reading unit (a front surface imaging unit 104, a rear surface imaging unit 105 and a non-contact IC reading unit 114) that reads, from an identity confirmation document, the own face image of a certain person; a face imaging unit 106 that obtains a user face image by imaging the user by a camera; a checkup unit 120 that determines whether the user is the certain person or not by executing a face checkup that checks up the own face image with the user face image by utilizing a face recognition algorithm; and a control unit 101 that decides the identity confirmation result on the basis of the determination result of the checkup unit 120. A guiding unit displays a motion image that guides the user in the appearance of a face and the direction both suitable for the face checkup before or while the checkup unit 120 is executing the face checkup.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an identity verification device and an identity verification method, and is suitable for use in an identity verification device and an identity verification method that verify the identity of a person by comparing a facial image captured by a camera mounted on the device with a facial image on an identity verification document. [Background technology]

[0002] In recent years, as the shift to cashless transactions and operational reforms have reduced the workload of counter staff, efforts are being made to improve efficiency by promoting self-service counter operations.In addition, with the aim of preventing money laundering, the outflow of funds to terrorist organizations, and fraudulent transactions, which have long been problems, new online identity verification methods have been established through legal reforms, and there is a demand for greater efficiency in counter operations that verify the identity of users.

[0003] In order to realize the efficiency of the above-mentioned identity verification process, it is conceivable to equip automated teller machines, certificate issuing machines, etc. with identity verification functions using facial recognition technology, and have users themselves perform identity verification operations. Such devices with identity verification functions authenticate whether the user is the person listed on the identity document by comparing the facial image of the user captured by a camera installed in the device with the facial image information stored in the contactless IC chip of the photo identification document or the facial image on the document (face matching). However, when the user performs identity verification operations themselves without the intervention of a teller or other person, it is expected that the user's facial image will not be captured under appropriate photographic conditions. Therefore, it is necessary to provide some kind of guidance (guidance) to the user to guide them so that they can capture an appropriate facial image for face matching.

[0004] For example, Patent Document 1 discloses a technology for displaying a guidance screen to a user when matching is not successful when performing personal authentication using a facial image in an automated teller machine. According to the technology disclosed in Patent Document 1, if matching cannot be performed successfully because the user is wearing a mask or the like, a current facial image with the mask or the like is displayed to the user, thereby guiding the user to remove the mask or the like. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-98955 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the reason why an appropriate facial image for face matching cannot be captured is not limited to the wearing of a mask or the like (user's appearance) as disclosed in Patent Document 1, but also includes various other factors such as the user's physique and posture. In automated teller machines (ATMs) and certificate issuing machines, the positions where a facial matching camera can be installed are limited, and the shooting direction of the facial matching camera is often fixed. Therefore, the position and orientation of the user's face are important for the facial matching camera to capture an appropriate facial image. For example, even if a user's face is captured so that it fits within the shooting range of the facial matching camera, the angle (orientation) of the face relative to the facial matching camera will vary depending on the user's height. In this case, differences between the captured facial image and the facial image on the identification document may occur, resulting in a decrease in the facial matching rate. To resolve this issue, it is necessary to guide the user to face the facial matching camera in an appropriate direction.

[0007] However, Patent Document 1 does not take into consideration guidance on how to correct factors such as the user's physique or posture, and it is possible that an appropriate facial image may not be captured due to factors other than wearing a mask. Furthermore, while Patent Document 1 mentions wearing a mask, other factors related to appearance that can reduce the facial matching rate include, for example, hairstyle, glasses, and hats. It is difficult to say that Patent Document 1 adequately discloses guidance on how to resolve such factors related to appearance.

[0008] The present invention has been made in consideration of the above points, and aims to propose an identity verification device and an identity verification method that can improve the facial matching rate in identity verification performed by the user's own operation by presenting the user with guidance for taking a facial image suitable for identity verification when the user attempts to carry out a transaction that requires identity verification. [Means for solving the problem]

[0009] In order to solve such problems, the present invention provides an identity verification device that performs identity verification on a user attempting to execute a transaction based on an identity verification document presented by the user, wherein the identity verification document is a medium on which information about a specific person who is authorized to execute the transaction is recorded and which contains at least a facial image of the specific person, and the identity verification device comprises: a guidance unit that outputs guidance to the user at a predetermined timing; an identity information reading unit that reads predetermined information including a principal's facial image, which is a facial image of the specific person, from the identity verification document; a face photographing unit that photographs the user with a camera to obtain a user's facial image; a matching unit that performs face matching that matches the principal's facial image with the user's facial image using a facial recognition algorithm to determine whether the user is the specific person; and a control unit that determines the result of the identity verification based on the result of the determination by the matching unit, wherein before or during the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that is suitable for the face matching.

[0010] In addition, in order to solve such problems, the present invention provides an identity verification method using an identity verification device that performs identity verification on a user attempting to execute a transaction based on an identity verification document presented by the user, wherein the identity verification document is a medium on which information about a specific person who is authorized to execute the transaction is recorded and which has at least a facial image of the specific person, and the identity verification device has: a guidance unit that outputs guidance to the user at a predetermined timing; an identity information reading unit that reads predetermined information including a principal face image that is a facial image of the specific person from the identity verification document; a face photographing unit that photographs the user with a camera to obtain a user face image; a matching unit that performs face matching that matches the principal face image with the user face image using a face recognition algorithm to determine whether the user is the specific person; and a control unit that determines the result of the identity verification based on the result of the determination by the matching unit, and wherein the identity verification method is characterized in that before or during the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching. [Effects of the Invention]

[0011] According to the present invention, when a user attempts to carry out a transaction that requires identity verification, it is possible to improve the facial matching rate in identity verification performed by the user's own operation. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram showing an automated teller machine 1 equipped with a personal identification device 100 according to a first embodiment of the present invention. [Figure 2] 2 is a block diagram showing an example of the internal configuration of the personal identification device 100. FIG. [Figure 3] 10 is a flowchart showing an example of a transaction processing procedure including identity verification processing. [Figure 4] FIG. 10 is a diagram showing an example (part 1) of guidance presented before face matching begins. [Figure 5] FIG. 10 is a diagram showing a second example of a guide presented before face matching begins. [Figure 6] 10A to 10C are diagrams illustrating an example of screen transitions of a moving image displayed before face matching starts. [Figure 7] FIG. 10 is a diagram showing an example of a guide presented at the start of face matching. [Figure 8] FIG. 10 is a diagram showing an example of guidance presented during face matching. [Figure 9] 10A to 10C are diagrams illustrating an example of screen transition of a moving image displayed during face matching. [Figure 10] FIG. 10 is a diagram showing an example of a notice presented when the result of face matching is abnormal. [Figure 11] 10A and 10B are diagrams illustrating an example of guidance presented during face pose matching. [Figure 12] 10A to 10C are diagrams illustrating an example of screen transition of a moving image displayed during face pose matching. [Figure 13] 10 is a flowchart showing an example of a transaction processing procedure including identity verification processing in the second embodiment. [Figure 14] 13 is a flowchart showing an example of a processing procedure of a face matching step in the third embodiment. [Figure 15] 13A to 13C are diagrams illustrating an example of screen transition of a screen animation during face matching in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] Note that the following description and drawings are examples for explaining the present invention, and have been omitted or simplified as appropriate for clarity of explanation. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. The present invention is not limited to the embodiments, and all application examples consistent with the concept of the present invention are included in the technical scope of the present invention. Those skilled in the art can make various additions and modifications to the present invention within the scope of the present invention. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be plural or singular.

[0015] In addition, in the following description, when describing elements of the same type without distinguishing between them, reference signs or common numbers in reference signs will be used, and when describing elements of the same type with distinction between them, the reference signs of those elements will be used or an ID assigned to those elements will be used instead of the reference signs.

[0016] Furthermore, although the following description may describe processing performed by executing a program, the program is executed by at least one processor (e.g., a CPU) to perform a predetermined process using storage resources (e.g., memory) and / or interface devices (e.g., communication ports) as appropriate, and therefore the processor may be the subject of the processing. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, node, storage system, storage device, server, management computer, client, or host having a processor. The subject of the processing performed by executing a program (e.g., a processor) may include a hardware circuit that performs part or all of the processing. For example, the subject of the processing performed by executing a program may include a hardware circuit that performs encryption and decryption or compression and decompression. The processor operates as a functional unit that realizes a predetermined function by operating in accordance with the program. Apparatuses and systems including a processor are apparatuses and systems that include these functional units.

[0017] A program may be installed on a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server includes a processor (e.g., a CPU) and storage resources, and the storage resources may further store a distribution program and a program to be distributed. The processor of the program distribution server may then execute the distribution program, causing the processor of the program distribution server to distribute the program to be distributed to other computers. In the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

[0018] (1) First embodiment (1-1) Configuration 1 is a diagram showing an automated teller machine 1 equipped with a personal identification device 100 according to a first embodiment of the present invention. The personal identification device 100 is installed, for example, inside the automated teller machine 1, and the detailed internal configuration thereof will be described later with reference to FIG.

[0019] The face matching camera 106 shown in Fig. 1 is one of the components of the identity verification device 100, and corresponds to the face photographing unit 106 shown in Fig. 2. When a user attempts to carry out a predetermined transaction using the automated teller machine 1, the identity verification device 100 mounted on the automated teller machine 1 requests the user to present an identity verification document, and performs "identity verification" by using the face image of the user captured by the face matching camera 106 to confirm whether the user is the person (principal person) identified by the identity verification document.

[0020] An identification document is a medium on which information about a specific person authorized to execute a transaction is recorded, and a facial image of that person is attached. The medium of the identification document may be a paper medium on which analog data is recorded, or an IC card or the like with a built-in IC chip on which digital data is recorded. The facial image attached to the identification document may be a direct attachment, such as by pasting a photograph of the face, or an indirect attachment, such as by storing facial image data in an IC chip.

[0021] Fig. 1 shows a schematic view of the photographing range of the face matching camera 106 mounted on the automated teller machine 1. The automated teller machine 1 is, for example, a machine with a height of approximately 1200 mm or less, and the face matching camera 106 is attached to its front (user side). That is, in the case of Fig. 1, the height of the face matching camera 106 from the ground is 1200 mm or less.

[0022] Here, users of the automated teller machine 1 vary in height, and if the user is too tall or too short, the user's face will be out of the center position of the image captured by face matching camera 106. Furthermore, even if the user's face falls within the image capture range of face matching camera 106, the orientation of the face relative to face matching camera 106 will be at various angles depending on the positional relationship between the user's eyes and face matching camera 106. For example, if the user is tall or the installation position of face matching camera 106 is low, the user will look down at face matching camera 106, and if the user is short or the installation position of face matching camera 106 is high, the user will look up at face matching camera 106.

[0023] If the face matching camera 106 is located below the user's eye level, the user may only turn their eyes toward the face matching camera 106 when taking a photograph, rather than facing their entire face toward the camera. In this case, the captured face image (hereinafter referred to as the user's face image) will be oriented differently from the face image on the identification document (hereinafter referred to as the "identity face image"). Furthermore, if the user is photographed facing sideways, the face image will also be oriented differently from the identity face image. Thus, if the user's face image is photographed in a different orientation from the identity face image, this may cause a decrease in the matching rate of face matching in identity verification, which may affect the execution of the transaction. Therefore, the identity verification device 100 according to this embodiment presents the user with easy-to-understand video (e.g., animation) or audio guidance (guidance) to enable the face matching camera 106 to capture a face image suitable for identity verification, thereby increasing the matching rate of face matching and reducing the impact on the transaction.

[0024] 2 is a block diagram showing an example of the internal configuration of the personal identification device 100. The personal identification device 100 is a device that can be operated by a user himself / herself to perform personal identification, and is configured to include a control unit 101, a display operation unit 102, an operator display operation unit 103, a front photographing unit 104, a back photographing unit 105, a face photographing unit 106, an IO control unit 111, a front illumination unit 112, a back illumination unit 113, a contactless IC reading unit 114, a voice input unit 115, a voice output unit 116, a power supply unit 117, and a matching unit 120. The matching unit 120 may be configured to be installed outside and connected to the personal identification device 100 as shown in FIG. 2, or may be configured to be incorporated inside the personal identification device 100.

[0025] The control unit 101 performs overall control of each component of the personal verification device 100. In the case of Fig. 2, the control unit 101 is connected to the matching unit 120 via USB (Universal Serial Bus) or Bluetooth (registered trademark).

[0026] 2, the control unit 101 not only controls the operation of each component such as the face photographing unit 106 connected to the control unit 101, but also controls the operation of each component such as the surface illumination unit 112 connected to the IO control unit 111, which controls input and output, via the IO control unit 111. For example, the control unit 101 can instruct the contactless IC reading unit 114 to read information from the IC chip of personal identification information, and can instruct the power supply unit 117, which controls the power supply of the personal identification device 100, to turn off the power, via the IO control unit 111.

[0027] The control unit 101 and the IO control unit 111 may be realized by software in which a processor executes a program, or may be realized in part or entirely by hardware circuits.

[0028] The display operation unit 102 is an input / output device for a user, and includes a display unit 107 that displays guidance and an operation unit 108 that accepts user operations. If the operation unit 108 has a touch panel function, the display unit 107 and the operation unit 108 can be realized by a touch display having a shared display operation screen. The user performs touch operations in accordance with instructions such as the guidance screen displayed by the display unit 107 or the voice output by the voice output unit 116, and the operation content is received by the operation unit 108. Specifically, when the user touches a software key displayed on the display operation screen, the operation unit 108 receives the operation content and transmits it to the control unit 101. For example, the operation unit 108 is used when input of predetermined information such as a personal identification number for identification is required to read information from an IC chip (contactless IC) mounted on an identification document.

[0029] By attaching a privacy film to the display / operation screen of the display / operation unit 102, it is possible to make it difficult for others to see the user's operations, thereby increasing security. The display / operation unit 102 may also be realized as an air-touch operation display. In this case, by displaying operation buttons in the air in addition to the guidance screen, the user can operate the unit in a clean manner without directly touching the screen. The unit may also be configured so that the user can input operation details by voice into the microphone of the voice input unit 115 and voice recognition is performed. In such a configuration, voice operation is possible, allowing the user to operate the unit in a clean manner.

[0030] The operator display operation unit 103 is an input / output device for the operator of the automated teller machine 1, and functionally includes a display unit 109 that displays images for the operator, and an operation unit 110 that accepts operations from the operator. The display and operations for the operator are not closely related to the main features of the present invention, so detailed explanations will be omitted.

[0031] The display operation screens of the display operation unit 102 and the operator display operation unit 103 may be implemented separately or may be implemented as a common screen. Furthermore, the display operation screens may use the screen of the automated teller machine 1.

[0032] The front photographing unit 104, back photographing unit 105, front illumination unit 112, back illumination unit 113, and contactless IC reading unit 114 are components for acquiring predetermined information from an identification document. The front photographing unit 104 and back photographing unit 105 have the function of photographing an identification document, and the front illumination unit 112 and back illumination unit 113 have the function of irradiating light from a light source to photograph the identification document. The contactless IC reading unit 114 has the function of reading information (IC registration information) stored in a contactless IC mounted on the identification document.

[0033] To photograph an identification document, the user follows the instructions displayed on the display operation screen of the display operation unit 102 and places the identification document in a predetermined location with the front side facing the front photographing unit 104. Then, the front photographing unit 104 and back photographing unit 105 photograph the identification document while the front lighting unit 112 and back lighting unit 113 emit light from their light sources to illuminate the identification document.

[0034] To explain the photographing of the identification document in more detail, the front illumination unit 112 has multiple normal light sources and illuminates the front of the identification document placed in a predetermined location. Using this illuminated light, the front photographing unit 104 can photograph the front of the identification document and transmits the photographed data to the control unit 101. On the other hand, the back illumination unit 113 has multiple transmitted light sources and illuminates the front side of the identification document placed in a predetermined location. The light illuminated from the transmitted light sources of the back illumination unit 113 passes through the identification document, and using this transmitted light, the back photographing unit 105 can photograph the back of the identification document and transmits the photographed data to the control unit 101.

[0035] By configuring as described above, the personal identification device 100 can photograph both sides of the personal identification document using the front photographing unit 104 and the back photographing unit 105 while the personal identification document is placed in a predetermined location, without the need to turn the document over.

[0036] The backside illumination unit 113 may be equipped with an ultraviolet light source (UV-C) for the purpose of sterilization. In this case, the backside illumination unit 113 can sterilize the surface of the identification document by irradiating it with the ultraviolet light source while the identification document is being verified.

[0037] Furthermore, if the personal identification document is equipped with a contactless IC, the contactless IC reading unit 114 reads the IC registration information stored in the contactless IC. Reading IC registration information by a contactless method can be done using existing technology, so a detailed explanation will be omitted. If input of predetermined information such as a PIN number is required to read the IC registration information, the user inputs the predetermined information on the display operation screen, and the operation unit 108 receives the input content. As a result, the personal identification device 100 can read the IC registration information while the personal identification document is placed in the predetermined location, without the need to turn the document over.

[0038] In this way, the personal identification device 100 can acquire, from the personal identification document, images and information required for determining the authenticity of the personal identification document and for verifying the identity of the user.

[0039] The face photographing unit 106 is a face matching camera that photographs the face of the user in order to verify the identity of the user. The face photographing unit 106 photographs the face in accordance with instructions from the control unit 101, and transmits the photographed face image (user face image) to the control unit 101.

[0040] The voice input unit 115 is, for example, a microphone, and receives voice input from the user. As described above, the voice input unit 115 may perform voice recognition on the input voice. The voice output unit 116 is, for example, a speaker, and outputs voice guidance.

[0041] The matching unit 120 has a "face matching" function that uses a known face recognition algorithm to match the face image on the identification document (personal face image) with the face image of the user (user face image) captured by the face capturing unit 106, thereby determining whether the user shown in the user face image is the same person as the person shown on the identification document. The matching unit 120 can output the result of the face matching, and if the user does not match the person on the identification document and is determined to be "abnormal (failed)", it can also output the cause of the abnormality.

[0042] Furthermore, the matching unit 120 has a "face pose matching" function that performs biometric matching using a known biometric authentication algorithm on multiple patterns of user face images taken at different times by the face photographing unit 106 to determine whether the user shown in the user face image (in other words, the user performing the identity verification operation) is the real user. Performing face pose matching can prevent user impersonation.

[0043] The mobile terminal 130 is a terminal carried by the user. The user connects the mobile terminal 130 to the matching unit 120 by, for example, any of WiFi, USB, or Bluetooth, reads the two-dimensional code displayed on the display unit 107 of the personal identification device 100 with the mobile terminal 130, and connects to the URL corresponding to the two-dimensional code. Then, at this URL, the user can execute transaction details, personal information consent, input of personal information, etc.

[0044] (1-2) Processing 3 is a flowchart showing an example of a transaction processing procedure including identity verification processing. Identity verification processing is processing for verifying whether a user is the person shown on an identity verification document, and is executed by the identity verification device 100 based on the user's own operation. When a user attempts to carry out a transaction with the automated teller machine 1, identity verification processing is carried out on the user to determine whether the user is a party that can be used for the transaction.

[0045] 3, first, the control unit (not shown) of the automated teller machine 1 that has detected the presence of a user displays a predetermined transaction selection screen and requests the user to select a transaction they wish to execute (step S101). In this example, it is assumed that a transaction that requires identity verification using an identity verification document has been selected. In this case, identity verification processing is performed by the identity verification device 100 in steps S102 to S117 before the execution of the selected transaction.

[0046] The identity verification process can be broadly divided into an information reading step (steps S102 to S106) that reads the information necessary for identity verification from the identity verification document, a face matching step (steps S107 to S110) that determines whether the user shown in the user's face image is the same person as the face image of the person in question, a face pose matching step (steps S111 to S113) that determines whether the user shown in the user's face image is a real person present in front of the identity verification device 100, and a comprehensive judgment step (steps S114 to S116) that makes a comprehensive judgment based on the results of each of the above steps as to whether the user is the person shown in the identity verification document.

[0047] In the information reading step, first, the control unit 101 of the personal identification device 100 requests the user to place the personal identification document by causing the display operation screen of the display operation unit 102 to display guidance and the audio output unit 116 to provide audio guidance (step S102). In response to this request, the user places the personal identification document in a predetermined position.

[0048] Strictly speaking, when a screen is displayed on the display operation screen of the display operation unit 102, the control unit 101 instructs the display unit 107 to display the screen, and the display unit 107 displays the screen on the display operation screen in response to this instruction. However, for the sake of simplicity, hereinafter, descriptions such as "the control unit 101 displays the screen on the display operation screen of the display operation unit 102," "the control unit 101 causes the display unit 107 to display the screen," or "the display unit 107 (or the display operation unit 102) displays the screen" will be used. Also, for the sake of simplicity, descriptions of the IO control unit 111 may be omitted for the control performed by the control unit 101 via the IO control unit 111. The above-mentioned omissions in descriptions apply to the overall operation of each unit controlled by the control unit 101.

[0049] If the personal identification document is equipped with a contactless readable IC chip (contactless IC), the user may be required to input specific information to read the information (registered IC information) stored in the contactless IC. The specific information may be, for example, a personal identification number. If the user needs to input the specific information to read the registered IC information from the personal identification document placed in step S102, the control unit 101 prompts the user to input the specific information by displaying the screen on the display unit 107 or by providing voice guidance from the audio output unit 116 (step S103). In response to this request, the user operates the display / operation unit 102 to input the specific information, such as a personal identification number, and the operation unit 108 transmits the input specific information to the control unit 101. The control unit 101 sends the received specific information to the verification unit 120, which then verifies whether the specific information is correct (for example, whether it is a personal identification number pre-registered in association with the contactless IC) and returns the authentication result to the control unit 101. The authentication of the predetermined information is not limited to the above method, and other generally known authentication methods may be used. If the predetermined information input by the user is authenticated as correct, the processing from step S104 onwards is carried out.

[0050] In step S104, the control unit 101 photographs an image of the face of the personal identification document placed in step S102 and reads the IC registration information of the contactless IC. More specifically, the control unit 101 photographs the front of the face of the document using the front photographing unit 104 and photographs the back of the face of the document using the back photographing unit 105. The control unit 101 also reads the IC registration information from the contactless IC using the contactless IC reading unit 114.

[0051] Next, the control unit 101 determines whether or not sufficient information required for identity verification has been acquired in step S104 (step S105). If it is determined that sufficient information has been acquired (YES in step S105), the information reading step is terminated and the process proceeds to step S107, which is the face matching step. On the other hand, if it is determined that sufficient information has not been acquired (NO in step S105), the control unit 101 issues an error notification corresponding to an error in reading the identity verification document (step S106), and then returns to step S104 to redo the acquisition of information from the identity verification document. The error notification is issued by at least one of displaying on the display operation screen of the display operation unit 102 or outputting audio by the audio output unit 116.

[0052] In the face matching step, first, the control unit 101 starts capturing an image of the user's face using the face capturing unit 106 (step S107). The control unit 101 then analyzes the captured face image of the user to detect whether the user is wearing a mask, sunglasses, or the like. After step S107, the face capturing unit 106 continues capturing the face image of the user continuously (including periodically repeating or intermittently repeating). It is preferable that the capturing of face images started in step S107 continues at least until face matching is completed successfully (YES in step S109) or face pose matching is completed successfully (YES in step S112). These user face images captured by the face capturing unit 106 are then used for face matching in step S108, which will be described later.

[0053] Next, the control unit 101 transmits the user's face image captured by the face capturing unit 106 and the user's face image included in the information acquired from the identification document in step S104 to the matching unit 120, and requests the matching unit 120 to perform face matching using this information. The matching unit 120 uses a face recognition algorithm to match the user's face image with the user's face image to determine whether the user shown in the user's face image is the user listed on the identification document, and returns the determination result to the control unit 101 (step S108). If the facial matching result is determined to be abnormal (the user is determined to be not the user), the matching unit 120 transmits the cause of the abnormality along with the determination result to the control unit 101. The user's face image used for face matching is selected from multiple patterns of images continuously captured by the face capturing unit 106, taking into consideration the position and orientation of the face, etc., and is suitable for matching. This selection of the user's face image may be performed by the matching unit 120 or by the control unit 101 within the automated teller machine 1.

[0054] Next, the control unit 101 checks whether the determination result of the face matching in step S108 is normal (step S109). For example, even if the user is the person listed on the identification document, if the user is wearing a mask, sunglasses, or the like, the face matching may not be determined to be normal. If the determination result of the face matching is normal (YES in step S109), the face matching step is terminated and the process proceeds to step S111 of the face pose matching step. If the determination result of the face matching is not normal (NO in step S109), the control unit 101 provides an abnormality notification corresponding to the abnormality in the face matching result (step S110), and then returns to step S109 to redo the face matching. The abnormality notification is provided by at least one of displaying on the display operation screen of the display operation unit 102 or outputting audio by the audio output unit 116.

[0055] In the face pose matching step, the control unit 101 transmits multiple patterns of user face images captured by the face photographing unit 106 to the matching unit 120 and requests that the matching unit 120 perform face pose matching using this information. The matching unit 120 uses a biometric authentication algorithm to match the user's biometrics from the multiple patterns of user face images, thereby determining whether the user shown in the user face images is a real person present in front of the personal identification device 100 (automated cash transaction machine 1), and responds with the determination result to the control unit 101 (step S111).

[0056] Next, the control unit 101 checks whether the determination result of the face pose matching in step S111 is normal (step S112). If the determination result is normal (YES in step S112), the face pose matching step ends and the process proceeds to the overall determination step of step S114. If the determination result in step S112 is not normal (NO in step S112), the control unit 101 issues an abnormality notification corresponding to the abnormality in the face pose matching result (step S113), and then returns to step S111 to perform face pose matching again.

[0057] In the comprehensive determination step, the control unit 101 uses the determination results of the face matching and the face pose matching to comprehensively determine whether the user is the person shown on the identification document based on predetermined determination criteria (step S114). The determination criteria are determined, for example, based on the type of transaction selected in step S101, the amount of money to be handled, etc. Therefore, depending on the transaction, even if either the face matching or the face pose matching results in an incorrect determination, the control unit 101 can determine in step S114 that the identification is correct. Furthermore, the information on the identification document read in the information reading step may be included in the determination factors of step S114.

[0058] Next, the control unit 101 checks whether the determination result of the comprehensive determination in step S114 is normal or not (step S115). If the determination result of the comprehensive determination is normal (YES in step S115), the process proceeds to step S117. At the stage of proceeding to step S117, the user is permitted to execute the transaction selected in step S101. On the other hand, if the determination result of the comprehensive determination is not normal (NO in step S115), the control unit 101 provides an abnormality notification corresponding to the abnormality in the comprehensive determination result (identity verification result) (step S116), and then returns to step S102 to redo the identity verification process.

[0059] In step S117, the control unit 101 displays the user's registration information on the display operation screen of the display operation unit 102 and asks the user to confirm it. The displayed registration information is, for example, personal information such as the user's name, date of birth, address, and telephone number, and also includes a security code and expiration date for personal identification information. The control unit 101 extracts this registration information from the information read from the personal identification document in step S104 and displays it on the display operation screen. If the user wants to change the displayed registration information, the user performs a correction operation on the display operation screen. When the operation unit 108 accepts the correction operation, the control unit 101 changes the registration information according to the operation content. The control unit 101 may display the changed registration information on the display operation screen and ask the user to confirm it.

[0060] Finally, the control unit 101 notifies the control unit of the automated teller machine 1 that identity verification for the transaction selected in step S101 has been successfully completed, and upon receiving this notification, the control unit of the automated teller machine 1 executes the transaction (step S118).

[0061] 3 is merely an example, and various modifications can be made to the execution of each step in this embodiment. Specifically, for example, the execution order of the face matching step and the face pose matching step may be reversed as long as they are performed after the information reading step. Furthermore, for example, since the face pose matching step does not require matching with an identity verification document, the control unit 101 may execute the processing within the automated teller machine 1. Furthermore, for example, the processing of checking and correcting the registered information in step S117 may not be executed.

[0062] (1-3) Screen example Below, the guidance presented to the user at each stage in the identity verification process will be described in detail with reference to specific examples of screens displayed on the display operation screen and audio output.

[0063] Fig. 4 is a diagram showing an example (part 1) of guidance presented before face matching begins. Fig. 4 shows an example of guidance to encourage the user to adopt an appropriate appearance before face matching is performed in step S108 of Fig. 3, taking into consideration that the matching rate of face matching decreases when a face image is taken while wearing sunglasses, a mask, or a hat. Specifically, Fig. 4 shows a pre-face matching start screen 210 displayed on the display operation screen of display operation unit 102, together with a character string 215 of voice guidance output from voice output unit 116.

[0064] On the screen 210 before face matching starts, an area 211 displays an image of a person wearing sunglasses, a mask, or a hat as a specific example of an inappropriate appearance for face matching. A person wearing sunglasses is an inappropriate appearance because the eyes are hidden, a person wearing a mask is an inappropriate appearance because at least one of the nose or mouth is hidden, and a person wearing a hat is an inappropriate appearance because more than a part of the facial contours are hidden. An area 212 displays an image of a person not wearing sunglasses, a mask, or a hat as a specific example of an appropriate appearance for face matching. Furthermore, a voice guidance character string 215 notifies the user that a photograph of the face will now be taken.

[0065] By presenting a guide such as that shown in Figure 4, the personal identification device 100 clearly encourages the user to pay attention to their appearance when photographing their face, and prevents a decrease in the face matching rate due to images of a user wearing sunglasses, a mask, or a hat.

[0066] Fig. 5 is a diagram showing a second example of guidance presented before face matching begins. Fig. 5 shows an example of guidance to encourage the user to adopt an appropriate appearance before face matching is performed in step S108 of Fig. 3, taking into consideration that the matching rate of face matching decreases when the eyebrows or face line (facial outline) are hidden by hair in a face image. Specifically, Fig. 5 shows a pre-face matching start screen 220 displayed on the display operation screen of display operation unit 102, together with a character string 225 of voice guidance output from voice output unit 116.

[0067] In the pre-face matching screen 220, an area 221 displays a face image of the person obtained by photographing an identification document and reading the face image. An area 222 displays a moving image of a person whose eyebrows are obscured by hair as a specific example of an inappropriate appearance for face matching, and an area 223 displays a moving image of a person whose facial lines are obscured by hair as another specific example of an inappropriate appearance for face matching. In addition, a character string 225 of voice guidance instructs the user to match their appearance to the appearance on the identification document during face matching and provides guidance on the photographing procedure for face matching.

[0068] Fig. 6 is a diagram showing an example of screen transitions of a moving image displayed before face matching begins. Fig. 6(A) to (D) sequentially show the screen transitions of an example of a moving image (animation) displayed in area 222 of Fig. 5, specifically showing the action of styling hair so that it does not cover the eyebrows. Fig. 6(E) to (F) sequentially show the screen transitions of an example of a moving image (animation) displayed in area 223 of Fig. 5, specifically showing the action of styling hair so that the facial line is not hidden.

[0069] 5 and 6, the personal identification device 100 clearly urges the user to pay attention to their appearance when photographing their face. As a result, the eyebrows and face line are not hidden by hair and the face is photographed with the same appearance as the face image on the personal identification document, preventing a decrease in the face matching rate.

[0070] Fig. 7 is a diagram showing an example of guidance presented at the start of face matching. Fig. 7 shows an example of guidance for encouraging the user to take an appropriate stance when face matching is started in step S108 of Fig. 3, taking into consideration that the matching rate of face matching may decrease due to factors such as the position and orientation of the user's face in the user's face image. Specifically, Fig. 7 shows a face matching start screen 230 displayed on the display operation screen of the display operation unit 102, together with a character string 235 of voice guidance output from the voice output unit 116.

[0071] In the face matching start screen 230, an image of the automatic teller machine 1 as seen from the front (user side) is displayed in an area 231. An image showing the ideal posture of the user in relation to the automatic teller machine 1 at the start of face matching, as viewed from the side, is displayed in an area 232. A voice guidance character string 235 provides guidance on the procedure for starting face matching. When the user presses a confirmation button 233, face matching starts.

[0072] Fig. 8 is a diagram showing an example of guidance presented during face matching. Fig. 8 shows an example of guidance for encouraging the user to take an appropriate stance during face matching in step S108 of Fig. 3, taking into consideration that the matching rate of face matching may decrease due to factors such as the position and orientation of the user's face in the user's face image. Specifically, Fig. 8 shows a face matching in progress screen 240 displayed on the display operation screen of the display operation unit 102, together with a character string 245 of audio guidance output from the audio output unit 116. When face matching starts, the display on the display operation screen transitions from the face matching start screen 230 of Fig. 7 to the face matching in progress screen 240 of Fig. 8, and a moving image is displayed.

[0073] On the face matching in progress screen 240, an area 241 displays a video showing the automatic teller machine 1 from the front. An area 242 displays a video showing the ideal posture of the user in relation to the automatic teller machine 1 from the side during face matching. Details of the video displayed in areas 242, 242 will be described in FIG. 9. Furthermore, a character string 245 of voice guidance guides the user on the actions required during face matching.

[0074] Fig. 9 is a diagram showing an example of screen transitions of moving images displayed during face matching. Fig. 9(A) to (I) sequentially show the screen transitions of an example of moving images (animation) displayed in areas 241 and 242 of face matching in progress screen 240 shown in Fig. 8. While face matching is being performed, the display operation screen repeatedly displays the animation displays of Fig. 9(A) to (I). Then, audio output unit 116 outputs audio guidance such as that shown in character string 245 in Fig. 8 in accordance with the transitions of the animation.

[0075] 9(A), the display operation screen is highlighted by a frame in area 241, and the user's actions of looking at the display operation screen are displayed in area 242. In accordance with the display in FIG. 9(A), the audio output unit 116 outputs guidance such as "Please face the flashing operation screen and remain still."

[0076] 9(B) to (C), in area 241, a frame display moves upward from the display operation screen, and face matching camera 106 is highlighted by the frame display. In area 242, an action of the user lifting their face from the display operation screen and looking at face matching camera 106 is displayed. In accordance with the displays in FIGS. 9(B) to (C), audio output unit 116 outputs guidance such as "Please face the flashing camera and remain still."

[0077] Next, in Figures 9(D) to (E), in area 241, a frame display moves upward from face matching camera 106, and a predetermined position above face matching camera 106 is highlighted by the frame display. Area 242 displays the action of the user lifting their face from face matching camera 106 and looking at the predetermined position. This "predetermined position" refers to the position in front of the user's eyes when they face forward. In accordance with the display of Figures 9(D) to (E), audio output unit 116 outputs guidance such as "Please face forward and remain still."

[0078] Next, in Figures 9(F) to (I), in area 241, the frame display moves downward from the predetermined position, and the display operation screen is highlighted by the frame display. In area 242, the action of the user lowering their face from the predetermined position and looking at the display operation screen is displayed. In accordance with the displays of Figures 9(F) to (I), the audio output unit 116 outputs guidance such as "Please face the flashing operation screen and remain still." Then, after Figure 9(I), the screen transitions to Figure 9(A), and the video display and audio output are repeated.

[0079] By providing guidance during face matching as described above, the personal identification device 100 can prompt the user to pay attention to the direction of the face, which can be a factor in reducing the matching rate. In other words, the user can be easily guided by video and audio to move their face up and down to take a picture at an appropriate direction, rather than just looking at the target with their eyes. As a result, the facial image of the user captured by the face matching camera (face photographing unit) 106 is more likely to match the facial angle of the facial image on the personal identification document, which is expected to have the effect of increasing the matching rate of face matching.

[0080] Furthermore, since the probability that the angle of the face will match the image on the identification document is low when only a face image is taken while looking at one point, by specifying multiple targets such as the display operation screen, the face matching camera, and a frontal view as shown in the guide in Figure 9 and taking face images while having the user slowly move their face up and down, it becomes easier to select a face image whose angle of face matches the image on the identification document from among the multiple face images taken during the period, and as a result, the face matching rate can be increased.

[0081] Fig. 10 is a diagram showing an example of a notice presented when the face matching result is abnormal. Fig. 10 shows a specific example of an abnormality notice corresponding to an abnormality in the face matching result performed in step S110 of Fig. 3, and shows an abnormal face matching result screen 250 displayed on the display operation screen of the display operation unit 102.

[0082] In the screen 250 for displaying an abnormal face matching result, an area 251 displays an image of the user's face obtained by photographing an identification document and reading the face image. An area 252 displays an image of the user's face that has been determined to be abnormal in face matching. An area 253 displays images showing examples of appearances that may be factors in determining an abnormality in face matching. In the case of FIG. 10 , the images displayed in area 253 are fixed images showing a state in which the user is wearing sunglasses and a state in which the user is wearing a mask, respectively, but the display in area 253 is not limited to this. For example, when an abnormality is determined in face matching, an image of appearance corresponding to the factor notified by matching unit 120 may be displayed in area 253.

[0083] Furthermore, the user's face image displayed in area 252 is preferably selected from among the multiple patterns of user's face images captured by face photographing unit 106, one that has a relatively high similarity to the user's face image displayed in area 251. A high similarity can be, for example, one in which the angle (orientation) of the face is close. By displaying a user's face image that has a high similarity to the user's face image in area 252 in this way, the user who views screen 250 for abnormal face matching result can accurately understand what points differ from the user's face image, and can approach face matching again with a strong focus on improving the differences. As a result, it can be expected to have the effect of increasing the success rate of face matching when re-executing.

[0084] As described above, by displaying the guidance shown in Fig. 10 when an abnormality is detected in face matching, the user can easily recognize the difference in appearance between the face image of the person in question and the face image of the user (areas 251, 252). Furthermore, the display of area 253 allows the user to easily guess the problem with appearance that caused the abnormality in face matching. As a result, the user will be encouraged to take another face image without sunglasses, a mask, a hat, etc., and a decrease in the face matching rate due to differences in appearance can be prevented.

[0085] Fig. 11 is a diagram showing an example of guidance presented during face pose matching. Fig. 11 shows an example of guidance for prompting the user to perform a facial pose action (action to change the direction of the face) during the execution of face pose matching in step S111 of Fig. 3. Specifically, Fig. 11 shows a face pose matching in progress screen 260 displayed on the display operation screen of the display operation unit 102.

[0086] As described above, face pose matching is a process for matching the biometrics of a user captured by the face matching camera (face photographing unit 106), and in order to confirm the biometrics, it is necessary to capture multiple patterns of camera-captured images by photographing the user with different face poses. Therefore, in area 261 of face pose matching screen 260, a moving image (animation) of the user moving their face relative to the face matching camera is displayed.

[0087] Fig. 12 is a diagram showing an example of screen transition of a moving image displayed during face pose matching. Fig. 12(A) to (D) sequentially show the screen transition of an example of a moving image (animation) displayed in area 261 of face pose matching in progress screen 260 shown in Fig. 11. Specifically, in Fig. 12(A), area 261 displays a state in which the user is looking straight ahead. Then, from Fig. 12(B) to (D), area 261 displays an animation of the user turning their face to the right. After the user has turned to the right in Fig. 12(D), the screen returns to Fig. 12(A), and area 261 displays a state in which the user is looking straight ahead.

[0088] 11 and 12, the personal identification device 100 can clearly communicate to the user the facial pose movements required for facial pose matching. As a result, the user can pay attention to the direction of their face and take a photograph with a face direction that increases the matching rate of facial pose matching.

[0089] 12, the user is requested to look straight ahead and then turn to the right, but the facial pose movements requested of the user for facial pose matching do not have to be limited to one type. For example, a plurality of types of facial pose movements and corresponding videos may be prepared in advance, and when performing facial pose matching, the control unit 101 may randomly select one of the movements and present the guidance. By having such a function, the personal identification device 100 can enhance the effectiveness of preventing impersonation.

[0090] 11 displays a button labeled "Another Action," and when this button is pressed, the control unit 101 may select one action from among the above-described plurality of types of face pose actions that is different from the action for the face pose currently being guided on the face pose matching screen 260, and display a video corresponding to the action for the selected face pose in the area 261. By having such a function, even when a user has difficulty performing an action for a particular face pose, the personal identification device 100 can guide the user to actions that the user can perform and perform face pose matching, thereby improving convenience for the user and increasing the success rate of face pose matching.

[0091] As described above, the identity verification device 100 according to the first embodiment performs identity verification processing, and when a user attempts to perform a transaction requiring identity verification, guidance (guidance) is presented to the user to capture a face image suitable for identity verification, thereby improving the face matching rate in identity verification performed by the user's own operation. In particular, in face verification, which compares the face image on an identity verification document with the user's face image, the user is clearly informed by image (video), audio, or both of suitable conditions for improving the face matching rate, including not only appearance (hairstyle, whether or not wearing glasses, a hat, a mask, etc.) but also the position and orientation of the face, allowing the user to easily proceed with the identity verification operation. Furthermore, in face pose verification, which verifies that the user performing the identity verification operation is the real user, the required face pose movements are clearly guided by video, which allows the user's movements to be smoothly guided, thereby improving the face pose verification rate when the user is not an impersonator.

[0092] (2) Second embodiment An identity verification device 100 according to a second embodiment of the present invention will be described. The second embodiment is an improvement over the first embodiment, in which the number of candidate user face images used for face matching is increased in identity verification processing executed based on user operation. The configuration of the identity verification device 100 according to the second embodiment is the same as that of the first embodiment, so the same will be reused and description will be omitted.

[0093] FIG. 13 is a flowchart showing an example of a transaction processing procedure including identity verification processing according to the second embodiment.

[0094] 13, when the automated teller machine 1 detects a user, the control unit 101 of the personal identification device 100 starts capturing a facial image of the user using the face capturing unit 106 (step S201). Furthermore, the control unit 101 analyzes the captured facial image of the user and detects whether the user is wearing a mask, sunglasses, or the like. The process of step S201 corresponds to the process of step S107 in FIG. 3. At the time of step S201, it is not necessary to notify the user that capturing of a facial image will begin.

[0095] After step S201, steps S101 to S118, excluding step S107, are executed in the same manner as in Fig. 3. These steps are the same as those described in Fig. 3, and therefore will not be described again. That is, the processing procedure shown in Fig. 13 differs from the processing procedure shown in Fig. 3 in the first embodiment in the execution timing of step S201 (step S107 in Fig. 3).

[0096] In the second embodiment, after step S201, the face photographing unit 106 continues to photograph the face images of the user continuously (including periodically or intermittently). It is preferable that the photographing of face images started in step S201 continues at least until face matching ends normally (YES in step S109) or face pose matching ends normally (YES in step S112). These face images of the user photographed by the face photographing unit 106 are used for face matching in step S108, which will be described later. That is, compared to the identity verification process in the first embodiment, in which the face photographing unit 106 starts photographing when face matching starts, in the second embodiment, the face photographing unit 106 can photograph face images for face matching before the user performs a predetermined operation to start face matching in step S108 (for example, pressing a confirmation button to start photographing, etc.).

[0097] In the second embodiment, the personal identification device 100 can acquire more candidates for a user's face image to be used for face matching than in the first embodiment, and can more easily extract a user's face image that closely matches the face image on the personal identification document during face matching. As a result, the personal identification device 100 according to the second embodiment can achieve a higher face matching rate than in the first embodiment.

[0098] Furthermore, if the user's face image detects that the user is wearing a mask, sunglasses, etc., the system may point out this problem before starting face matching and prompt the user to remove the mask, sunglasses, etc. This can prevent face matching failures due to differences in appearance in advance, thereby increasing the face matching rate.

[0099] Furthermore, the facial images of the user captured by the face photographing unit 106 after step S201 are basically used for face matching in step S108, but for example, when only confirming the liveness of the user standing in front of the automated teller machine 1 in face pose matching in step S111, these user facial images may also be used for face pose matching. In this case, it is possible to confirm that the user is moving (alive) from multiple patterns of captured user face images before starting face pose matching in step S111, which makes it possible to process face pose matching easily or in a short time.

[0100] As described above, according to the personal identification device 100 of the second embodiment, in addition to the effects obtained in the first embodiment, it is possible to further increase the matching rate of face matching.

[0101] (3) Third embodiment An identity verification device 100 according to a third embodiment of the present invention will now be described. The third embodiment is an improvement on the facial matching process (steps S108 to S110 in FIGS. 3 and 13) described in the first or second embodiment, and changes the video guidance content so that when facial matching is re-executed within the same transaction, a user's facial image suitable for facial matching can be efficiently captured based on multiple patterns of user's facial images captured in the previously executed facial matching. The configuration of the identity verification device 100 according to the third embodiment is the same as that of the first or second embodiment, so this will be reused and description will be omitted.

[0102] In the third embodiment, when performing at least the first facial matching in the same transaction, the control unit 101 displays a guidance video (screen animation during facial matching) and has the user change the direction of their face while capturing multiple patterns of user facial images taken by the facial capturing unit 106, and detects the position to which the user's face was directed in the video when a user facial image suitable for facial matching was captured (hereinafter referred to as the priority position).

[0103] Here, the "user face image suitable for face matching" is a user face image with a high matching rate in face matching by the matching unit 120, specifically, a user face image with a facial orientation similar to that of the user's own face image. Therefore, the control unit 101 can extract the "user face image suitable for face matching" by comparing multiple patterns of user face images with the user's own face image in terms of facial orientation. As another extraction method, the control unit 101 may extract a user face image determined to have a high matching rate based on the result of face matching by the matching unit 120 as the "user face image suitable for face matching." In any case, the control unit 101 can determine which scene in the guidance video the extracted "user face image suitable for face matching" corresponds to based on the capture timing of the "user face image suitable for face matching," and detect the position that was highlighted in the corresponding scene as the "priority position."

[0104] Furthermore, the detection of the "priority position" by the control unit 101 may be performed only when the determination result in face matching is abnormal (face matching has failed).

[0105] Fig. 14 is a flowchart showing an example of the processing procedure of the face matching step in the third embodiment. The processing shown in Fig. 14 is a modification of the processing of the face matching step (steps S108 to S110 in Fig. 3 and Fig. 13) described in the first or second embodiment. The "screen (X)" and "guidance screen (X)" described in the processing in Fig. 14 correspond to the respective scenes (A) to (I) of the screen animation during face matching shown as an example in Fig. 9.

[0106] 14, when the face matching step starts, the control unit 101 determines whether or not it is necessary to display a guidance screen (A) in the screen animation during face matching, based on information indicating the priority position (step S301). Specifically, if the guidance screen (A) is a scene in which the priority position is highlighted, the control unit 101 determines that it is necessary to display the guidance screen (A) (YES in step S301), displays the guidance screen (A) in step S302, and proceeds to the next process. On the other hand, if the guidance screen (A) is not a scene in which the priority position is highlighted, the control unit 101 determines that it is not necessary to display the guidance screen (A) (NO in step S301), and proceeds to the next process without displaying the guidance screen (A). Note that if this is the first face matching within the same transaction, the priority position has not been detected, and therefore the control unit 101 always determines that it is necessary to display the guidance screen (A).

[0107] Thereafter, similar to steps S301 to S302, the control unit 101 sequentially determines whether each scene (guidance screens (B) to (I)) of the animation during face matching is necessary based on the information indicating the priority position, and if it determines that it is necessary, it displays the target scene, and if it determines that it is not necessary, it skips displaying the target scene (steps S303 to S308).

[0108] When the determination of necessity for all scenes of the face matching animation is completed, the control unit 101 determines whether or not a user's face image suitable for face matching by the matching unit 120 has been captured (step S309). If a user's face image suitable for face matching has been captured (YES in step S309), the control unit 101 requests the matching unit 120 to perform face matching using the user's face image, and proceeds to step S310.

[0109] On the other hand, if a user's face image suitable for face matching has not been captured in step S309 (NO in step S309), the process returns to step S301, and video display (steps S301 to S308) is repeated based on the necessity determination for each scene. This repetition is performed up to a predetermined number of times, and if a user's face image suitable for face matching has not been captured even after the predetermined number of repetitions, the process may proceed to step S311, which will be described later.

[0110] Then, in step S310, control unit 101 checks whether the determination result of face matching by matching unit 120 is normal or not, and if it is normal (YES in step S310), ends the face matching step.

[0111] On the other hand, if the face matching determination result is abnormal in step S310 (NO in step S310), control unit 101 provides an abnormality notification corresponding to the abnormality in the face matching result (step S311), and returns to step S301. As described above, control unit 101 detects the "priority position" for the current face matching image capture and acquires information indicating the priority position at least during the period from step S311 to returning to step S301.

[0112] 14, if step S309 or step S310 is determined to be NO and a user face image suitable for face matching cannot be captured even after repeating the video display (steps S301 to S308) based on the necessity determination for each scene a predetermined number of times, the control unit 101 may increase the number of scenes to be displayed in the screen animation during face matching. Specifically, the control unit 101 may increase the number of scenes determined to be necessary in the necessity determination by lowering the evaluation standard for the "priority position," or may display all scenes without performing the necessity determination. By adopting such a modification, it is expected to be effective in preventing a situation in which face matching does not progress.

[0113] Fig. 15 is a diagram showing an example of screen transitions of the screen animation during face matching in the third embodiment. Specifically, Fig. 15 shows screen transitions of the screen animation during face matching that are displayed when face matching is retried when the priority position is only the "display operation screen." In the case of the screen animation during face matching in Fig. 9, three positions, for example, "bottom (display operation screen)," "middle (face matching camera 106)," and "top (front position)" can be cited as positions where highlighting is performed by frame display.

[0114] When the priority position is only the "display operation screen," referring to the screen animation during face matching shown in FIG. 9, the scenes highlighting the display operation screen in area 241 correspond to guidance screens (A) and (I). When such conditions are applied to the processing of FIG. 14, among the necessity determinations for each scene (steps S301, S303, S305, and S307), a determination of "necessary" is made in steps S301 and S307, and guidance screens (A) and (I) are displayed in steps S302 and S308. However, since guidance screen (I) is the same as guidance screen (A), it is treated as guidance screen (A) in FIG. 15. Therefore, in FIG. 15, guidance screen (A) is repeatedly displayed, and as shown in area 242, the user's face can be guided toward the highlighted display operation screen (i.e., downward).

[0115] As shown in Figure 15, by displaying an image guiding the user to a preferred position as a priority in the animation during face matching when face matching is re-executed, the user's face can be guided in a direction that is easy to use for face matching, thereby improving the efficiency of capturing face images for face matching.

[0116] As a variation of the animation during face matching during re-execution, rather than preferentially displaying only the scene guiding to the priority position, it is also possible to preferentially display scenes before and after guiding to a predetermined range including the priority position. Specifically, for each scene in FIG. 9 , if the priority position is "bottom (display operation screen)," not only guidance screen (A) but also nearby guidance screens (B) and (H) may be preferentially displayed. Furthermore, if the priority position is "middle (face matching camera 106)," not only guidance screen (C) but also nearby guidance screens (B) and (D) may be preferentially displayed. According to this variation, a video that changes the orientation of the user's face in the vertical direction around the priority position can be displayed. Therefore, if a position that guides the user to a more suitable face angle exists near the priority position, the possibility of capturing a user's face image in the most suitable orientation increases, resulting in an effect of improving the matching rate of face matching.

[0117] As described above, according to the third embodiment, by performing the processing of the face matching step shown in Fig. 14, the personal identification device 100 can display, in a moving image display of guidance when face matching is re-executed, an image that emphasizes a "priority position" where a user's face image suitable for face matching can be captured, based on multiple patterns of user's face images captured in a previously executed face matching. As a result, the user can be efficiently guided to a priority position where the user's face can be easily detected for face matching, and the matching rate of face matching can be efficiently increased while shortening the time required to re-execute face matching. [Explanation of symbols]

[0118] 1. Automatic teller machine 100 Personal identification device 101 Control section 102 Display operation section 103 Operator display and operation section 104 Surface Photography Unit 105 Backside Photography Section 106 Face capture unit (face matching camera) 107,109 Display section 108,110 Operation unit 111 IO control unit 112 Surface lighting section 113 Backside lighting unit 114 Contactless IC reader 115 Audio input section 116 Audio output section 117 Power supply section 120 Collation Unit 130 mobile devices

Claims

1. An identity verification device that verifies the identity of a user attempting to execute a transaction based on the identity verification documents presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; an operation unit that accepts operations by the user; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, the face photographing unit continuously photographs the user and acquires a plurality of patterns of face images of the user before the face matching is performed; the face photographing unit starts continuous photographing of the user before the user performs an operation on the operation unit to start the face matching, The matching unit performs the face matching using a user face image extracted from the plurality of patterns of user face images. An identity verification device characterized by:

2. An identity verification device that verifies the identity of a user who is about to execute a transaction based on the identity verification documents presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, Before the face matching is performed, the guide unit displays an image showing a face appearance that is inappropriate for the face matching. An identity verification device characterized by:

3. An identity verification device that verifies the identity of a user who is about to execute a transaction based on an identity verification document presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, Before the face matching is performed, the guide unit displays the face image of the person and an image showing a face appearance inappropriate for the face matching. An identity verification device characterized by:

4. An identity verification device that verifies the identity of a user who is about to execute a transaction based on an identity verification document presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, the face photographing unit continuously photographs the user and acquires a plurality of patterns of face images of the user before the face matching is performed; the matching unit performs the face matching using a user face image extracted from the plurality of patterns of user face images, During the execution of the face matching, the guide unit displays a video prompting the user to move the direction of the face in an up and down direction relative to the camera. An identity verification device characterized by:

5. An identity verification device that verifies the identity of a user who is about to execute a transaction based on an identity verification document presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, the face photographing unit continuously photographs the user and acquires a plurality of patterns of face images of the user before the face matching is performed; the matching unit performs the face matching using a user face image extracted from the plurality of patterns of user face images, When the face matching is determined to be abnormal, the guidance unit displays the face image of the person, the face image of the user, and an image showing a face appearance inappropriate for the face matching, The matching unit re-executes the face matching based on an operation by the user. An identity verification device characterized by:

6. If the face matching is determined to be abnormal, the guiding unit displays, as the user face image, a user face image having a face orientation similar to that of the user's own face image among the plurality of patterns of user face images.

6. The personal identification device according to claim 5.

7. When the face matching is determined to be abnormal, the guiding unit displays an image corresponding to the appearance that caused the abnormality as an image showing a facial appearance that is inappropriate for the face matching.

6. The personal identification device according to claim 5.

8. the matching unit, in addition to performing the face matching, uses a biometric authentication algorithm to perform face pose matching to match the biometrics of the user based on a plurality of patterns of user face images acquired by the face photographing unit photographing the user performing a predetermined movement; The control unit determines a result of the identity verification based on the determination results of the face matching and the face pose matching.

6. The personal identification device according to claim 1, claim 4, or claim 5.

9. An identity verification device that verifies the identity of a user who is about to execute a transaction based on an identity verification document presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, the face photographing unit continuously photographs the user and acquires a plurality of patterns of face images of the user before the face matching is performed; the matching unit performs the face matching using a user face image extracted from the plurality of patterns of user face images, the matching unit, in addition to performing the face matching, uses a biometric authentication algorithm to perform face pose matching to match the biometrics of the user based on a plurality of patterns of user face images acquired by the face photographing unit photographing the user performing a predetermined movement; the control unit determines a result of the identity verification based on determination results of the face matching and the face pose matching, During the execution of the face pose matching, the guidance unit displays a video prompting the user to perform the predetermined action. An identity verification device characterized by:

10. An identity verification device that verifies the identity of a user attempting to execute a transaction based on an identity verification document presented by the user, the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; Equipped with before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, the face photographing unit continuously photographs the user and acquires a plurality of patterns of face images of the user before the face matching is performed; the matching unit performs the face matching using a user face image extracted from the plurality of patterns of user face images, the matching unit, in addition to performing the face matching, uses a biometric authentication algorithm to perform face pose matching to match the biometrics of the user based on a plurality of patterns of user face images acquired by the face photographing unit photographing the user performing a predetermined movement; the control unit determines a result of the identity verification based on determination results of the face matching and the face pose matching, When a predetermined operation is performed by the user, the predetermined movement required in the face pose matching is changed. An identity verification device characterized by:

11. The appearance inappropriate for face matching is a face state in which at least one of the eyes, nose, mouth, eyebrows, or a part or more of the facial contours is hidden.

8. The personal identification device according to claim 2, claim 3, claim 5, or claim 7.

12. The guidance unit outputs a sound related to an image in accordance with the display of the image.

11. The personal identification device according to claim 1, wherein the personal identification device is a personal identification device.

13. A method for verifying the identity of a user who is about to execute a transaction, using an identity verification device to verify the identity of the user based on an identity verification document presented by the user, comprising: the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; and before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, The control unit detects a position where the user's face is oriented in a direction suitable for the face matching when the face matching is performed, and when the face matching is performed again within the same transaction, causes the guidance unit to preferentially display an image that guides the user to orient the face at the detected position in the video displayed by the guidance unit. A method for verifying identity, characterized in that

14. A method of identity verification using an identity verification device for verifying the identity of a user who is about to execute a transaction based on an identity verification document presented by the user, comprising: the personal identification document is a medium on which information about a specific person authorized to execute the transaction is recorded, and which includes at least a facial image of the specific person; The personal identification device a guidance unit that outputs guidance to the user at a predetermined timing; a face photographing unit that photographs the user with a camera to acquire a face image of the user; a matching unit that performs face matching by using a face recognition algorithm to match a face image of the specific person obtained from the identification document with the user's face image, thereby determining whether the user is the specific person; a control unit that determines a result of the identity verification based on a result of the determination by the matching unit; and before or during the execution of the face matching by the matching unit, the guidance unit displays a video that guides the user to a facial appearance and orientation that are suitable for the face matching, The control unit detects a position where the user's face is oriented in a suitable direction for the face matching when the face matching is performed, and when the face matching is performed again within the same transaction, causes the guidance unit to preferentially display an image that guides the user to orient the face in a peripheral range including the detected position in the video displayed by the guidance unit. A method for verifying identity, characterized in that

Images