Multi-factor authentication system, multi-factor authentication method, and multi-factor authentication information acquisition system

Abstract

This multifactor authentication system: acquires biometric information of a person being authenticated from information about a predetermined site acquired by a sensor; performs a first authentication process using the acquired biometric information; on the basis of the position of the predetermined site that has been detected by a detection device, receives an input of authentication information for use in a second authentication process and performs the second authentication process; and performs authentication of the person being authenticated on the basis of a result of the first authentication process and a result of the second authentication process.

Description

[Technical Field]

[0001] The present disclosure relates to a multi-factor authentication system, a multi-factor authentication method, and a multi-factor authentication information acquisition system. [Background technology]

[0002] Patent Document 1 discloses a personal authentication device that is equipped with an input key and a fingerprint identification device. The personal authentication device accepts the input of a personal identification number on a liquid crystal display screen, and by erasing the personal identification number to make it transparent, it can be used as a fingerprint authentication window onto which a finger can be placed, and by reading the fingerprint image of the finger placed on the fingerprint authentication window, personal authentication is performed using the personal identification number and fingerprint authentication. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2001-242952 Summary of the Invention [Problem to be solved by the invention]

[0004] However, contact-type personal authentication devices have the risk of the residual fingerprint remaining on the fingerprint authentication window after fingerprint reading being exploited, and the personal identification number being leaked based on the residual fingerprint, etc. Furthermore, in recent years, from the viewpoint of hygiene, such as preventing infectious diseases, there has been a demand for personal authentication devices that allow the input of personal identification numbers and the acquisition of fingerprint images in a contactless manner.

[0005] The present disclosure has been devised in view of the above-described conventional circumstances, and aims to provide a multi-factor authentication system, a multi-factor authentication method, and a multi-factor authentication information acquisition system that more effectively prevent leakage of authentication-related information used for personal authentication. [Means for solving the problem]

[0006] The present disclosure provides a multi-factor authentication system including a detection device that detects a predetermined body part of a person to be authenticated, a sensor that acquires information about the predetermined body part, a first authentication device that can communicate with the sensor and executes a first authentication process, a second authentication device that can communicate with the detection device and executes a second authentication process different from the first authentication process, and a terminal device that can communicate among the detection device, the first authentication device, and the second authentication device, wherein the first authentication device executes the first authentication process based on information about the predetermined body part acquired by the sensor, the terminal device accepts contactless input of authentication information by the person to be authenticated based on the position of the predetermined body part detected by the detection device and transmits it to the second authentication device, the second authentication device executes the second authentication process based on the authentication information, and the terminal device authenticates the person to be authenticated based on a result of the first authentication process and a result of the second authentication process, and the multi-factor authentication system executes the first authentication process and the acquisition of the authentication information in parallel.

[0007] The present disclosure also provides a multi-factor authentication method performed by a multi-factor authentication system that performs a first authentication process and a second authentication process different from the first authentication process to authenticate a person to be authenticated, the multi-factor authentication method including: detecting a predetermined body part of the person to be authenticated; acquiring information about the detected predetermined body part; acquiring biometric information of the person to be authenticated from the acquired information about the predetermined body part; performing the first authentication process using the acquired biometric information; accepting input of authentication information to be used in the second authentication process contactlessly by the person to be authenticated based on the detected position of the predetermined body part; performing the second authentication process; and authenticating the person to be authenticated based on a result of the first authentication process and a result of the second authentication process, wherein the multi-factor authentication system performs the first authentication process and the acquisition of the authentication information in parallel.

[0008] The present disclosure also provides a multi-factor authentication information acquisition system including a sensor that acquires information about a specific body part used in a first authentication process of a person to be authenticated, a detection device that detects the specific body part, and a terminal device that accepts input of authentication information to be used in a second authentication process different from the first authentication process based on the position of the specific body part detected by the detection device, and acquires the authentication information. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to more effectively prevent leakage of authentication-related information used for personal authentication. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of a multi-factor authentication system according to a first embodiment. [Figure 2] FIG. 1 shows an example of the internal configuration of a first terminal device, a finger position detection device, and a biometric authentication device according to a first embodiment. [Figure 3] FIG. 1 shows an example of the internal configuration of a PIN authentication device and a second terminal device according to a first embodiment. [Figure 4] 10 is a flowchart showing an example of an operation procedure of a first terminal device according to the first embodiment. [Figure 5] 10 is a flowchart showing an example of an operation procedure of a first terminal device according to the first embodiment. [Figure 6] 10 is a flowchart showing an example of an operation procedure of a first terminal device according to the first embodiment. [Figure 7] 10 is a flowchart showing an example of an operation procedure of a first terminal device according to the first embodiment. [Figure 8] 1 is a timing chart showing an example of the overall operation procedure of a multi-factor authentication system according to a first embodiment; [Figure 9] A diagram explaining an example of an infrared camera and camera placement in a multi-factor authentication system [Figure 10] FIG. 10 is a diagram illustrating a first example of a pointer display on a first terminal device. [Figure 11] FIG. 10 is a diagram illustrating a second example of pointer display on the first terminal device. [Figure 12] An example of the authentication result screen when the authentication result is "OK" [Figure 13] An example of the authentication result screen when the authentication result is "NG" DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, with reference to the drawings as appropriate, embodiments that specifically disclose the configurations and operations of a multi-factor authentication system, a multi-factor authentication method, and a multi-factor authentication information acquisition system according to the present disclosure will be described in detail. However, more detailed explanations than necessary may be omitted. For example, detailed explanations of well-known matters or redundant explanations of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0012] The overall configuration of a multi-factor authentication system 100 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an example of the overall configuration of the multi-factor authentication system 100 according to the first embodiment. The multi-factor authentication system 100 according to the first embodiment is a system that performs user authentication using biometric authentication and personal identification number (hereinafter referred to as "PIN") authentication.

[0013] The multi-factor authentication system 100 according to the first embodiment includes a first terminal device P1, a finger position detection device C1, a biometric authentication device P2, a PIN authentication device P3, a second terminal device P4, a camera C2, a light L1, and a network NW. Note that the network NW is not essential and may be omitted.

[0014] The first terminal device P1 is connected to the finger position detection device C1 via a Universal Serial Bus (hereinafter referred to as "USB") cable or the like to enable wired communication, and is connected to the biometric authentication device P2, the PIN authentication device P3, and the second terminal device P4 via a network NW to enable wireless communication and data transmission / reception. Note that the wireless communication here refers to communication via a wireless local area network (LAN) such as Wi-Fi (registered trademark), a wide area network (WAN), or the like.

[0015] The first terminal device P1 controls the finger position detection device C1 to accept a PIN input by a user, and generates and outputs a user authentication result based on the biometric authentication result transmitted from the biometric authentication device P2 and the PIN authentication result transmitted from the PIN authentication device P3.

[0016] The finger position detection device C1 is connected to the first terminal device P1 via a USB cable or the like to enable wired communication and transmits and receives data. The finger position detection device C1 accepts user operations on various screens displayed on the first terminal device P1 (e.g., authentication start screens SC11, SC12, SC13, SC14, PIN authentication screen SC10, and authentication result screens SC15, SC16, etc.). The finger position detection device C1 converts the content of the user operations into electrical signals and transmits them to the first terminal device P1.

[0017] The biometric authentication device P2 is wirelessly connected to the first terminal device P1 and the second terminal device P4 via the network NW to transmit and receive data. The biometric authentication device P2 performs biometric authentication based on an image captured by the camera C2 and transmits the biometric authentication result to the first terminal device P1.

[0018] The PIN authentication device P3 is connected to the first terminal device P1 via the network NW so as to be able to communicate wirelessly with the first terminal device P1, and transmits and receives data to the first terminal device P1. The PIN authentication device P3 performs PIN authentication based on the PIN input result transmitted from the first terminal device P1, and transmits the PIN authentication result to the first terminal device P1.

[0019] The second terminal device P4 is connected to the first terminal device P1 and the biometric authentication device P2 via the network NW so as to be able to communicate wirelessly, and is connected to the camera C2 and the light L1 so as to be able to communicate wired using a USB cable or the like, to transmit and receive data. The second terminal device P4 controls the camera C2 and the light L1, and transmits images captured by the camera C2 to the biometric authentication device P2.

[0020] Camera C2, an example of a sensor, is connected to second terminal device P4 so as to be able to communicate via wired communication and transmits and receives data. Camera C2 is controlled by second terminal device P4 to capture an image of a region including a user's fingertip or part of a finger and transmits the captured image (an example of information related to a predetermined part) to second terminal device P4.

[0021] The light L1 is controlled by the second terminal device P4 to illuminate an area including the user's fingertip or part of the finger.

[0022] The network NW wirelessly connects the first terminal device P1, the biometric authentication device P2, the PIN authentication device P3, and the second terminal device P4, and transmits and receives data between them. The network NW is not essential and may be omitted. In such a case, the multi-factor authentication system 100 uses a USB cable or the like instead of the network NW to realize data transmission and reception between the first terminal device P1, the biometric authentication device P2, the PIN authentication device P3, and the second terminal device P4.

[0023] Next, the internal configurations of the first terminal device P1, finger position detection device C1, biometric authentication device P2, PIN authentication device P3, and second terminal device P4 in embodiment 1 will be described with reference to Fig. 2 and Fig. 3, respectively. Fig. 2 is a diagram showing an example of the internal configurations of the first terminal device P1, finger position detection device C1, and biometric authentication device P2 in embodiment 1. Fig. 3 is a diagram showing an example of the internal configurations of the PIN authentication device P3 and second terminal device P4 in embodiment 1. Note that the internal configurations of the various devices shown in Fig. 2 and Fig. 3 are merely examples and are not limiting.

[0024] The first terminal device P1 includes a communication unit 10, a processor 11, a memory 12, and a monitor 13. In the example shown in Fig. 2, for ease of understanding, the finger position detection device C1 and the finger position detection control unit 111 are illustrated as being communicatively connected, but the finger position detection device C1 may be connected to the communication unit 10 so as to be able to send and receive data therebetween.

[0025] The communication unit 10 transmits and receives data to and from the communication unit 20 of the biometric authentication device P2, the finger position detection device C1, the communication unit 30 of the PIN authentication device P3, and the communication unit 40 of the second terminal device P4.

[0026] The communication unit 10 transmits control commands output from the finger position detection control unit 111 of the processor 11 to various devices (such as an LED (Light-Emitting Diode) C11, infrared cameras C12 and C13, etc.) that constitute the finger position detection device C1, and also acquires electrical signals (control commands) that indicate user operations transmitted from the finger position detection device C1 and outputs them to the finger position determination unit 112 of the processor 11.

[0027] Furthermore, the communication unit 10 transmits a control command output from the finger position determination unit 112 of the processor 11 to the biometric authentication device P2 and to the PIN authentication device P3. The communication unit 10 outputs to the processor 11 each of the biometric authentication result transmitted from the biometric authentication device P2 and the PIN authentication result transmitted from the PIN authentication device P3.

[0028] Processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with memory 12. Specifically, processor 11 references programs and data stored in memory 12 and executes the programs to realize functions such as finger position detection control unit 111, finger position determination unit 112, image processing unit 113, and PIN input processing unit 114.

[0029] The finger position detection control unit 111 controls the LED C11 and the two infrared cameras C12 and C13, and causes each of the two infrared cameras C12 and C13 to capture an image of an area including at least a part of the user's fingertip.

[0030] Finger position determination unit 112 acquires each of a plurality of captured images captured by each of two infrared cameras C12 and C13. Finger position determination unit 112 processes each of the plurality of captured images and calculates the position (three-dimensional coordinates) of at least one of the user's fingers relative to monitor 13. Finger position determination unit 112 outputs the calculated position information (coordinate information) of at least one of the user's fingers for each finger to image processing unit 113 and PIN input processing unit 114. Note that the image processing may be performed by image processing unit 113.

[0031] The image processing unit 113 generates various screens based on the finger coordinate information output from the finger position determination unit 112, the biometric authentication result transmitted from the biometric authentication device P2, or the PIN authentication result transmitted from the PIN authentication device P3, and outputs the generated various screens to the monitor 13 for display.

[0032] PIN input processing unit 114 analyzes the PIN (numbers, alphabets, symbols, etc.) input by user operation and the input contents (PIN, input information, etc.) on various screens displayed on monitor 13, based on the finger coordinate information output from finger position determination unit 112. PIN input processing unit 114 outputs the analyzed PIN information to communication unit 10, which transmits it to PIN authentication device P3. PIN input processing unit 114 also outputs the analyzed input information (for example, information on selection (pressing) of key icon "Del" on authentication start screen SC11) to image processing unit 113.

[0033] The memory 12 includes, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when executing each process of the processor 11, and a read only memory (hereinafter referred to as "ROM") that stores programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM stores programs that define the operation of the processor 11.

[0034] The monitor 13 may be configured using a display device such as a Liquid Crystal Display (LCD) or an organic electroluminescence (EL). The monitor 13 outputs and displays various screens (e.g., authentication start screens SC11 to SC14, PIN authentication screen SC10, and authentication result screens SC15 and SC16) output from the processor 11. The first terminal device P1 may include a speaker (not shown) and may assist the user in entering the PIN by audio output using the speaker, or may notify the user of the authentication result of the user authentication.

[0035] The finger position detection device C1 detects the position of the user's fingertip relative to the monitor 13 of the first terminal device P1. The finger position detection device C1 includes an LED C11 and two infrared cameras C12 and C13.

[0036] The LEDC11 is configured with at least one infrared LED and illuminates the finger of a user operating the monitor 13 of the first terminal device P1 in a non-contact manner. The LEDC11 is controlled by the finger position detection control unit 111 of the first terminal device P1 to control the turning on / off of the infrared LED.

[0037] Each of the two infrared cameras C12 and C13 is a stereo camera and includes at least an image sensor (not shown) and a lens (not shown). The image sensor is a solid-state imaging element such as a Charged-Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS), and converts an optical image formed on the imaging surface into an electrical signal.

[0038] Each of the two infrared cameras C12 and C13 is controlled by the processor 11 of the first terminal device P1 to capture infrared rays emitted from the hand of a user operating the monitor 13 of the first terminal device P1 in a non-contact manner. Each of the two infrared cameras C12 and C13 transmits each of the captured images to the first terminal device P1.

[0039] The biometric authentication device P2 performs biometric authentication by comparing the biometric information of a user (e.g., a fingerprint, vein pattern, etc.) with the biometric information of each of multiple users registered in advance. The biometric authentication device P2 includes a communication unit 20, a processor 21, a memory 22, a biometric information database DB1, etc.

[0040] The communication unit 20 transmits and receives data to and from the communication unit 10 of the first terminal device P1 and the communication unit 40 of the second terminal device P4.

[0041] The communication unit 20 outputs a control command requesting the start of biometric authentication transmitted from the first terminal device P1, at least one captured image transmitted from the second terminal device P4, etc. to the processor 21. The communication unit 20 also transmits a control command to start imaging processing output from the processor 21 to the second terminal device P4, and transmits biometric authentication result information output from the processor 21 to the first terminal device P1.

[0042] The processor 21 is configured using, for example, a CPU or FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 references the programs and data stored in the memory 22 and executes the programs to realize functions such as a command analysis unit 211 and a biometric information authentication unit 212.

[0043] The command analysis unit 211 analyzes the control command (command) transmitted from the first terminal device P1. Based on the analysis result by the command analysis unit 211, the processor 21 generates a control command to start capturing an image of the user's fingertip by the camera C2, and transmits the generated control command to the second terminal device P4.

[0044] The biometric information authentication unit 212 extracts the user's biometric information from the captured image transmitted from the second terminal device P4. The biometric information authentication unit 212 compares the extracted user's biometric information with the biometric information of each of multiple users stored (registered) in the biometric information database DB1, and performs biometric authentication. Note that the extraction process and comparison process of the user's biometric information using the user's finger image may use known techniques (for example, the technique disclosed in Japanese Patent Application Laid-Open No. 2018-124999). The biometric information authentication unit 212 generates biometric authentication result information based on the biometric authentication result, outputs it to the communication unit 20, and causes it to be transmitted to the first terminal device P1.

[0045] The biometric authentication result information includes at least information on the biometric authentication result, and may include information such as a user name, a matching score between the extracted biometric information of the user and each of the biometric information of multiple users stored (registered) in the biometric information database DB1, etc.

[0046] The memory 22 includes, for example, a RAM as a work memory used when executing each process of the processor 21, and a ROM for storing programs and data that define the operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM has written therein programs that define the operation of the processor 21.

[0047] The biometric information database DB1 is a storage medium device such as a hard disk drive (hereinafter referred to as "HDD") or a solid state drive (hereinafter referred to as "SSD"), and registers (stores) each piece of biometric information of a plurality of users that has been registered in advance by an administrator (for example, a specific person such as a security guard or manager of the facility or building where the first terminal device P1 is installed, or an employee of the store where the first terminal device P1 is installed). The biometric information stored in the biometric information database DB1 is biometric information that can be authenticated based on the fingerprints or veins of each of the user's multiple fingers, and is registered (stored) in association with information about the user (for example, a user name, a user ID, the user's date of birth, or a user's facial image, etc.).

[0048] Note that, although the biometric information database DB1 shown in FIG. 2 is illustrated as being configured integrally with the biometric authentication device P2, it may also be configured separately from the biometric authentication device P2 and realized as an external storage device externally connected to enable data communication.

[0049] The PIN authentication device P3 compares the PIN entered into the first terminal device P1 with the PIN of the user identified by biometric authentication and registered in advance, and performs PIN authentication. The PIN authentication device P3 includes a communication unit 30, a processor 31, a memory 32, a PIN database DB2, etc.

[0050] The communication unit 30 transmits and receives data to and from the communication unit 10 of the first terminal device P1. The communication unit 30 outputs a control command, which is transmitted from the first terminal device P1 and requests the start of PIN authentication, to the processor 31. The communication unit 30 also transmits PIN authentication result information, which is output from the processor 31, to the first terminal device P1.

[0051] Processor 31 is configured using, for example, a CPU or FPGA, and performs various processes and controls in cooperation with memory 32. Specifically, processor 31 references the programs and data stored in memory 32 and executes the programs to realize functions such as command analysis unit 311 and PIN authentication unit 312.

[0052] The command analysis unit 311 analyzes the control command transmitted from the first terminal device P1, and based on the analysis result, the command analysis unit 311 causes the PIN authentication unit 312 to start PIN authentication.

[0053] The PIN authentication unit 312 refers to the PIN database DB2 based on information about the user identified from the biometric authentication result transmitted from the first terminal device P1 (for example, the user name, the user ID, the user's date of birth, or the user's facial image), and extracts a PIN associated with the information about the user. The PIN authentication unit 312 compares the PIN extracted from the PIN database DB2 with the PIN transmitted from the first terminal device P1, and performs PIN authentication.

[0054] If the user cannot be identified from the biometric authentication result (for example, if the user is not registered in the biometric information database DB1, and the biometric authentication result is "NG"), the PIN authentication unit 312 compares each of the multiple PINs registered (stored) in the PIN database DB2 with the PIN sent from the first terminal device P1, and performs PIN authentication. If the user cannot be identified from the biometric authentication result, the PIN authentication unit 312 may determine that PIN authentication is not possible, and determine that the PIN authentication result is "NG".

[0055] The PIN authentication unit 312 generates PIN authentication result information based on the PIN authentication result, outputs the PIN authentication result information to the communication unit 30, and causes it to be transmitted to the first terminal device P1. The PIN authentication result information includes information on the PIN authentication result and information on the user.

[0056] The memory 32 includes, for example, a RAM as a work memory used when the processor 31 executes each process, and a ROM for storing programs and data that define the operation of the processor 31. The RAM temporarily stores data or information generated or acquired by the processor 31. The ROM stores programs that define the operation of the processor 31.

[0057] The PIN database DB2 is a storage medium device such as an HDD or SSD, and registers (stores) the PIN information of each of multiple users that has been registered in advance by an administrator (for example, a specific person such as a security guard or administrator of the facility or building where the first terminal device P1 is installed, or an employee of the store where the first terminal device P1 is installed). The PIN database DB2 registers (stores) the PIN information in association with information about the user (for example, the user name, the user's date of birth, or the user's facial image).

[0058] Note that, although the PIN database DB2 shown in FIG. 3 is illustrated as being configured integrally with the biometric authentication device P2, it may also be configured separately from the biometric authentication device P2 and realized as an external storage device externally connected to enable data communication.

[0059] The second terminal device P4 controls the camera C2 and the light L1 to capture an image used for biometric authentication, the image including at least a portion of the user's finger or fingertip. The second terminal device P4 includes a communication unit 40, a processor 41, and a memory 42.

[0060] The communication unit 40 transmits and receives data to and from the communication unit 10 of the first terminal device P1 and the communication unit 20 of the biometric authentication device P2. The communication unit 40 outputs a control command, transmitted from the biometric authentication device P2, requesting the start of imaging to the processor 41. The communication unit 40 also transmits the captured image output from the processor 41 to the biometric authentication device P2.

[0061] The processor 41 is configured using, for example, a CPU or FPGA, and performs various processes and controls in cooperation with the memory 42. Specifically, the processor 41 references the programs and data stored in the memory 42 and executes the programs to realize the functions of a command analysis unit 411, a camera control unit 412, a lighting power supply unit 413, etc.

[0062] The processor 41 outputs the captured image captured by the camera C2 to the communication unit 40, and causes it to be transmitted to the biometric authentication device P2.

[0063] The command analysis unit 411 analyzes the control command (command) transmitted from the biometric authentication device P2. Based on the analysis result, the command analysis unit 411 generates a control command for controlling the camera control unit 412 or the lighting power supply unit 413, and outputs the generated control command to the camera control unit 412 or the lighting power supply unit 413.

[0064] Based on the control command output from the command analysis unit 411, the camera control unit 412 controls the start or end of imaging by the camera C2.

[0065] The lighting power supply unit 413 controls the lighting L1 to turn on or off based on the control command output from the command analysis unit 411.

[0066] The memory 42 includes, for example, a RAM as a work memory used when the processor 41 executes each process, and a ROM for storing programs and data that define the operation of the processor 41. The RAM temporarily stores data or information generated or acquired by the processor 41. The ROM stores programs that define the operation of the processor 41.

[0067] Next, the operation procedure of the first terminal device P1 will be described with reference to Fig. 4 to Fig. 7. Fig. 4 is a flowchart showing an example of the operation procedure of the first terminal device P1 in the first embodiment. Fig. 5 is a flowchart showing an example of the operation procedure of the first terminal device P1 in the first embodiment. Fig. 6 is a flowchart showing an example of the operation procedure of the first terminal device P1 in the first embodiment. Fig. 7 is a flowchart showing an example of the operation procedure of the first terminal device P1 in the first embodiment.

[0068] Furthermore, in the multi-factor authentication system 100 according to the present embodiment, an example will be described in which the PIN is made up of only a predetermined number of numbers, but the PIN is not limited to numbers and may include letters such as alphabets or symbols.

[0069] The processor 11 of the first terminal device P1 displays an authentication start screen SC11 (see FIG. 10) on the monitor 13. The processor 11 generates a control command to turn on the LED C11 and a control command to start capturing images for each of the two infrared cameras C12 and C13, and transmits these to the finger position detection device C1.

[0070] The processor 11 performs image processing on each of the multiple captured images transmitted from each of the two infrared cameras C12 and C13, and calculates the user's fingertip position (three-dimensional coordinates). Based on the calculated user's fingertip position (three-dimensional coordinates), the processor 11 determines whether the user's fingertip is within an area (hereinafter referred to as the "recognition area") corresponding to the authentication start screen SC11 displayed on the monitor 13 (St11). Note that the recognition area here may be the imaging area AR1 (see FIG. 9) of each of the two infrared cameras C12 and C13, or the imaging area AR2 (see FIG. 9) of the camera C2.

[0071] When the processor 11 determines in the process of step St11 that the user's fingertip is present within the recognition area (St11, YES), it generates and displays a pointer PT11 (see FIG. 10) on the authentication start screen SC11 at a position corresponding to the calculated position of the user's fingertip (St12). The pointer PT11 may be a so-called cursor or the like.

[0072] On the other hand, if the processor 11 determines in the processing of step St11 that the user's fingertip is not within the recognition area (St11, NO), it generates an authentication start screen SC13 (see FIG. 11) including an instruction MSG1 for the user to insert the fingertip into the recognition area, and outputs and displays it on the monitor 13 (St13). The processor 11 returns to the processing of step St11.

[0073] The processor 11 performs image processing on each of the multiple captured images transmitted from each of the two infrared cameras C12 and C13, and determines whether or not the pointers PT11 and PT13 (see Figures 10 and 11) are on the start button BT11 on the authentication start screens SC11, SC13, and SC14 (see Figures 10 and 11) (i.e., whether or not the user's fingertip is positioned at the position corresponding to the start button BT11) (St14).

[0074] If the processor 11 determines in the processing of step St14 that the pointer PT11, PT13 is located on the start button BT11 of the authentication start screen SC11, SC13, SC14 (St14, YES), it calculates the distance between the fingertip position and the monitor 13 based on the calculated fingertip position. The processor 11 determines whether the calculated distance between the fingertip position and the monitor 13 is within a specified distance (St15). Note that the distance between the fingertip position and the monitor 13 indicates the distance between the surface of the monitor 13 in the X direction and the detected position of the user's fingertip. The specified distance is the distance at which the user's finger or fingertip is included within the imaging area AR2 (see FIG. 9) of the camera C2.

[0075] On the other hand, if the processor 11 determines in the processing of step St14 that the pointers PT11 and PT13 are not positioned on the start button BT11 of the authentication start screens SC11, SC13 and SC14 (St14, NO), it generates an authentication start screen (not shown) including an instruction (not shown) to move the pointers PT11 and PT13 onto the start button BT11, and outputs and displays it on the monitor 13 (St16). The processor 11 returns to the processing of step St14.

[0076] When the processor 11 determines in the processing of step St15 that the distance between the calculated fingertip position and the monitor 13 is within a specified distance (St15, YES), it generates an authentication start screen (not shown) including a finger stillness instruction (not shown) to prompt the user to still the movement of his or her fingers, and outputs and displays the screen on the monitor 13 (St17). Note that the finger stillness instruction is displayed for a display period corresponding to the time (e.g., 1.5 seconds, 3 seconds, etc.) during which the camera C2 captures the image of the user's finger or fingertip.

[0077] On the other hand, if the processor 11 determines in the processing of step St15 that the distance between the calculated fingertip position and the monitor 13 is not within the specified distance (St15, NO), it generates an authentication start screen SC14 (see FIG. 11) including an instruction MSG2 to the user to bring the finger closer to the monitor 13, and outputs and displays it on the monitor 13 (St18). The processor 11 returns to the processing of step St14.

[0078] The processor 11 determines whether the distance between the calculated fingertip position and the monitor 13 is within a specified distance, or whether the distance between the user's fingertip position and the monitor 13 is within a specified distance and whether the user's fingertip has been continuously detected for a specified time from the time when it is determined that the distance between the calculated fingertip position and the monitor 13 is within a specified distance, or from the time when an authentication start screen (not shown) including a finger stillness instruction (not shown) is displayed on the monitor 13 (St19).

[0079] In the process of step St19, if the processor 11 determines that the distance between the position of the user's fingertip and the monitor 13 is within a specified distance and that the user's fingertip has been continuously detected for a specified time (St19, YES), it generates a control command to turn off the LED C11 (i.e., to turn off the infrared light irradiated onto the user's fingers) and transmits the control command to the finger position detection device C1 (St20). As a result, the infrared light irradiated by the LED C11 is turned off, and the camera C2 can capture an image suitable for biometric authentication and extraction of biometric information.

[0080] On the other hand, if the processor 11 determines in the processing of step St19 that the distance between the user's fingertip position and the monitor 13 is within a specified distance and that the user's fingertip has not been continuously detected for a specified time (St19, NO), it returns to the processing of step St14.

[0081] The processor 11 generates a control command requesting the start of biometric authentication, and outputs the generated control command to the communication unit 10 to cause it to be transmitted to the biometric authentication device P2 (St21).

[0082] After processing step St21, the processor 11 proceeds to PIN authentication processing, generates a control command to control the LED C11 to light up (i.e., to turn on the infrared light irradiated onto the user's fingers), and transmits the control command to the finger position detection device C1 (St22). As a result, the infrared light irradiated by the LED C11 is turned on, and the camera C2 can detect the position of the user's fingertip relative to the monitor 13.

[0083] The processor 11 generates a PIN authentication screen SC10 (see FIG. 9), and outputs it to the monitor 13 for display (St23).

[0084] The processor 11 acquires biometric authentication result information from the biometric authentication device P2 (St24).

[0085] The processor 11 calculates the user's finger position (three-dimensional coordinates) based on the captured images captured by the two infrared cameras C12 and C13. Based on the calculated user's fingertip position, the processor 11 determines whether the user's fingertip is within the recognition area corresponding to the PIN authentication screen SC10 displayed on the monitor 13 (St25).

[0086] If the processor 11 determines in the processing of step St25 that the user's fingertip is within the recognition area (St25, YES), it determines whether or not the user's finger has performed a click operation (specifically, a click motion) on the PIN authentication screen SC10 based on the captured images captured by each of the two infrared cameras C12 and C13 (St26).

[0087] On the other hand, if the processor 11 determines in the process of step St25 that the user's fingertip is not present within the recognition area (St25, NO), it executes the process of step St25 again.

[0088] If it is determined in the processing of step St26 that the user's finger has performed a click operation (specifically, a click motion) on the PIN authentication screen SC10 (St26, YES), the processor 11 calculates the position of the user's fingertip when it was determined that the click operation was performed. The processor 11 accepts a click operation by the user on the PIN authentication screen SC10 at a position corresponding to the position of the click operation (the calculated position of the user's fingertip) (St27).

[0089] On the other hand, if the processor 11 determines in the process of step St26 that the user's finger has not performed a click operation (specifically, a click motion) on the PIN authentication screen SC10 (St26, NO), the processor 11 returns to the process of step St25.

[0090] The processor 11 determines whether the click operation received is a number input operation (St28).

[0091] When processor 11 determines in the process of step St28 that the operation accepted by the click operation is an operation for inputting numbers (St28, YES), processor 11 determines whether the currently input number has a predetermined number of digits or less (St29).

[0092] On the other hand, if the processor 11 determines in the processing of step St28 that the operation accepted by the click operation is not a numeric input operation (St28, NO), it determines whether or not "Delete" (for example, the operation key KY12 "Del" shown in FIG. 9) was input by the click operation (St30).

[0093] If the processor 11 determines in the processing of step St29 that the currently entered numbers are less than the predetermined number of digits (St29, YES), it displays the entered numbers on the PIN authentication screen SC10 (see Figure 9) following the already entered numbers (St31).

[0094] On the other hand, if the processor 11 determines in the processing of step St29 that the currently entered number is not less than the predetermined number of digits (St29, NO), it deletes the first digit of the entered numbers and displays the entered number as the lowest digit on the PIN authentication screen SC10 (St32).

[0095] When it is determined in the process of step St30 that "Delete" has been input by a click operation (St30, YES), the processor 11 deletes all the input numbers (St34).

[0096] On the other hand, if the processor 11 determines in the processing of step St30 that "Delete" has not been input by a click operation (St30, NO), it determines whether "Enter" (for example, the operation key KY13 "Ent" shown in FIG. 9) has been input by a click operation (St33).

[0097] If the processor 11 determines in the processing of step St33 that "Enter" has been input by clicking (St33, YES), it determines whether or not the user has been identified based on the biometric authentication result (St35).

[0098] On the other hand, if the processor 11 determines in the process of step St33 that the "Enter" has not been input by a click operation (St33, NO), the process returns to the process of step St25.

[0099] In the process of step St35, if the processor 11 determines that the user has been identified based on the biometric authentication result (St35, YES), it acquires the user name (information about the user) included in the biometric authentication result information (St36).

[0100] On the other hand, if the processor 11 determines in the process of step St35 that the user has not been identified based on the biometric authentication result (St35, NO), it provisionally sets the user name to "Unknown" (St37). It goes without saying that the provisionally set user name is not limited to this.

[0101] Here, if the processor 11 determines in the processing of step St35 based on the biometric authentication result that the user has not been identified (St35, NO), the processor 11 may determine that the biometric authentication processing has timed out and determine that the biometric authentication result is "NG." Also, if the processor 11 determines in the processing of step St35 based on the biometric authentication result that the user has not been identified (St35, NO), the processor 11 may determine that the biometric authentication result is "NG" and proceed to the processing of step St42.

[0102] The processor 11 associates the input numeric information of the predetermined number of digits, the user name information based on the biometric authentication result, and a control command to execute the user name and PIN authentication, and transmits them to the PIN authentication device P3, causing the PIN authentication process to be executed (St38).

[0103] The processor 11 determines whether or not PIN authentication has been completed based on the PIN authentication result information transmitted from the PIN authentication device P3 (St39).

[0104] In the processing of step St39, if the processor 11 acquires the PIN authentication result information transmitted from the PIN authentication device P3 and determines that the PIN authentication has been completed (St39, YES), it determines whether the biometric authentication result acquired from the biometric authentication device P2 and the PIN authentication result transmitted from the PIN authentication device P3 are both "OK" (St40).

[0105] On the other hand, if the processor 11 cannot acquire the PIN authentication result information transmitted from the PIN authentication device P3 in the process of step St39 and determines that the PIN authentication has not been completed (St39, NO), the processor 11 returns to the process of step St25.

[0106] If the processor 11 determines in the processing of step St40 that both the biometric authentication result and the PIN authentication result are "OK" (St40, YES), it generates an authentication result screen SC15 (see FIG. 12) including the user authentication result "OK" and outputs it to the monitor 13 for display (St41).

[0107] On the other hand, if the processor 11 determines in the processing of step St40 that both the biometric authentication result and the PIN authentication result are not "OK" (St40, NO), it generates an authentication result screen SC16 (see FIG. 13) including the user authentication result "NG", and outputs and displays it on the monitor 13 (St42).

[0108] The processor 11 calculates the position (three-dimensional coordinates) of the user's fingertip based on the captured images captured by the two infrared cameras C12 and C13. Based on the calculated position of the user's fingertip and whether or not a click operation has been performed, the processor 11 determines whether or not a selection (input) operation has been performed on the Return button BT12 (see FIGS. 12 and 13) displayed on the authentication result screens SC15 and SC16 (St43).

[0109] In the process of step St43, if the processor 11 determines that the Return button BT12 displayed on the authentication result screen SC15, SC16 has been selected (input) (St43, YES), the processor 11 returns to the process of step St14.

[0110] On the other hand, if the processor 11 determines in the process of step St43 that the Return button BT12 displayed on the authentication result screens SC15 and SC16 has not been selected (input) (St43, NO), the processor 11 returns to the process of step St43.

[0111] In the multi-factor authentication system 100, the biometric authentication processing performed in step St15 and steps St17 to St23, respectively, and the PIN authentication processing performed in steps St24 to St34 and steps St38 to St39, respectively, may be performed in the reverse order.

[0112] As a result, the multi-factor authentication system 100 according to the first embodiment can acquire biometric information used for biometric authentication and a PIN used for PIN authentication, without the user's fingers coming into contact with the monitor 13 and the camera C2. Furthermore, when performing user authentication based on biometric authentication and PIN authentication (multi-factor authentication), the multi-factor authentication system 100 performs the biometric authentication process, and the acceptance of the input operation of the PIN used for PIN authentication and the acquisition of the PIN in parallel, thereby further shortening the time required for multi-factor authentication and making the overall processing of multi-factor authentication more efficient.

[0113] Next, an operation procedure of the multi-factor authentication system 100 will be described with reference to Fig. 8. Fig. 8 is a timing chart showing an example of the overall operation procedure of the multi-factor authentication system 100 according to the first embodiment.

[0114] At time T11, the first terminal device P1 controls the finger position detection device C1 to detect or measure the user's fingertip position on the authentication start screen SC11 (see FIG. 10) displayed on the monitor 13, based on each of the multiple captured images taken by the two infrared cameras C12 and C13. The first terminal device P1 generates an authentication start screen (not shown) including a finger stillness instruction (not shown) for the user to still the movement of their fingers, and outputs and displays it on the monitor 13.

[0115] At time T12, the first terminal device P1 controls to turn off the LEDC11 (i.e., turns off the infrared light irradiated onto the user's fingers). After turning off the LEDC11, the first terminal device P1 generates a control command requesting the start of biometric authentication and transmits it to the biometric authentication device P2.

[0116] At time T13, the biometric authentication device P2 starts biometric authentication processing based on the control command transmitted from the first terminal device P1, generates a control command (image capture instruction) to capture an image of the user's fingertip, and transmits it to the second terminal device P4. The second terminal device P4 controls the camera C2 to capture an image of the user's fingertip based on the control command transmitted from the biometric authentication device P2. Here, the first terminal device P1 controls the LED C11 to turn off after displaying a finger still instruction on the monitor 13. After the finger still instruction is displayed, the second terminal device P4 controls the camera C2 to capture an image of the user's finger or fingertip during the display period of the finger still instruction (for example, 1.5 seconds, 3 seconds, etc.).

[0117] At time T14, after the display period of the finger-still instruction has elapsed, the first terminal device P1 determines that the image capturing process by the camera C2 has been completed, and controls the LED C11 to light up (i.e., turns on the infrared light irradiated onto the user's fingers), and accepts the user's PIN input operation. In addition, the biometric authentication device P2 generates a control command to request the transmission of a captured image of an area including at least the user's finger or fingertip, and transmits the control command to the second terminal device P4.

[0118] At time T15, the second terminal device P4 transmits the captured image captured by the camera C2 to the biometric authentication device P2 based on the control command (image transmission instruction) transmitted from the biometric authentication device P2. The biometric authentication device P2 performs biometric authentication processing using the captured image transmitted from the second terminal device P4.

[0119] At time T16, the biometric authentication device P2 generates biometric authentication result information that associates the biometric authentication result with information about the user identified by the biometric authentication process (for example, the user name), and transmits the information to the first terminal device P1.

[0120] At time T17, the first terminal device P1 associates the input PIN (numeric) information and information (e.g., user name) included in the biometric authentication result information and related to the user identified by the biometric authentication process with a control command requesting the execution of PIN authentication process, and transmits them to the PIN authentication device P3. Based on the control command transmitted from the first terminal device P1, the PIN authentication device P3 compares the input PIN with the PIN corresponding to the identified user, and executes the PIN authentication process.

[0121] At time T18, the PIN authentication device P3 generates PIN authentication result information based on the PIN authentication result and transmits it to the first terminal device P1. The first terminal device P1 executes comprehensive authentication (user authentication) based on the biometric authentication result and the PIN authentication result. The first terminal device P1 generates authentication result screens SC15 and SC16 based on the comprehensive authentication (user authentication) result and outputs them to the monitor 13 for display.

[0122] Next, the arrangement of two infrared cameras C12 and C13 (stereo cameras) that capture images of the user's fingertip movements to accept PIN input and a camera C2 that captures images used for biometric authentication will be described with reference to Fig. 9. Fig. 9 is a diagram illustrating an example of the arrangement of the infrared cameras C12 and C13 and the camera C2 in the multi-factor authentication system 100.

[0123] Camera C2 is placed in front of (on the X-direction side of) the first terminal device P1. Infrared cameras C12 and C13 are placed further in front of (on the X-direction side of) camera C2. Note that, although FIG. 9 shows an example in which the two infrared cameras C12 and C13 and camera C2 are placed side by side on a horizontal plane (XY plane), this is not limiting. The two infrared cameras C12 and C13 and camera C2 may be placed at a predetermined angle with respect to the horizontal plane (XY plane) so as to be able to capture images of the user's fingers or fingertips that are more suited to the intended use.

[0124] The light L1 is disposed in a position where it can illuminate the user's fingertip located within the imaging area AR2 of the camera C2. The imaging area AR2 (an example of an acquisition area) of the camera C2 is an imaging area where it can capture an image suitable for extracting biometric information used for biometric authentication. The light L1 may be disposed between the first terminal device P1 and the camera C2, or between the camera C2 and each of the infrared cameras C12 and C13. A plurality of lights L1 may be disposed on both sides of the camera C2 along the Y-axis direction.

[0125] Furthermore, the LED C11 may be disposed in a position capable of illuminating a user's finger or fingertip located within each of the imaging areas AR1 of the two infrared cameras C12 and C13. The imaging areas AR1 of each of the two infrared cameras C12 and C13 are imaging areas capable of capturing images of user input operations on various screens displayed on the monitor 13. The LED C11 may be disposed between the camera C2 and each of the infrared cameras C12 and C13, or may be disposed in front of each of the infrared cameras C12 and C13 (on the X-axis direction). The LED C11 may be disposed along the periphery of the infrared cameras C12 and C13, or multiple LEDs C11 may be disposed on both sides of the infrared cameras C12 and C13 along the Y-axis direction.

[0126] The imaging areas AR1 and AR2 may or may not overlap each other.

[0127] 9 includes a PIN display area INP, a PIN key display area KY11, and operation keys KY12, KY13, and KY14. The PIN display area INP is an area that can display a PIN of a predetermined number of digits, and displays the entered PIN.

[0128] In this example, the PIN display area INP of the PIN authentication screen SC10 displays a four-digit PIN "1234," with the first digit being "1," the second digit being "2," the third digit being "3," and the fourth digit being "4" (the least significant digit). Note that when a PIN with more than a predetermined number of digits is entered, the first digit, "1," is deleted from the PIN display area INP, and each digit is moved up to the next highest digit, and the newly entered PIN is displayed as the least significant digit.

[0129] For example, if a new PIN "5" is entered, the PIN display area INP will delete the first digit, PIN "1," and display a total of four digits, PIN "2345," with the first digit being PIN "2," the second digit being PIN "3," the third digit being PIN "4," and the fourth digit (lowest digit) being PIN "5."

[0130] The PIN key display area KY11 displays keys such as numbers, letters, symbols, etc. used for the PIN. The PIN authentication screen SC10 shown in Fig. 9 shows an example in which only the numbers "0" to "9" are displayed.

[0131] Operation key KY12 is a key that can accept a user operation for deleting an input PIN. When processor 11 determines that operation key KY12 has been operated by the user to input (select), it deletes at least one of the input PINs. The PINs to be deleted may be all of the input PINs, the PIN with the lowest digit, or any one of the input PINs selected by the user.

[0132] Operation key KY13 is a key that can accept a user operation for performing PIN authentication based on the input PIN. When processor 11 determines that operation key KY13 has been operated by the user to input (select), it performs PIN authentication based on the input PIN.

[0133] Operation key KY14 is a key that can accept a user operation for returning (transitioning) the screen displayed on monitor 13 from PIN authentication screen SC10 to authentication start screen SC11 (see FIG. 10 ). When processor 11 determines that the user has operated operation key KY14 for input (selection), it causes monitor 13 to display authentication start screen SC11.

[0134] Next, examples of pointer display on various screens will be described with reference to Fig. 10. Fig. 10 is a diagram illustrating pointer display example 1 on the first terminal device P1. Note that the size of the pointer shown in Fig. 10 is an example and is not limited to this.

[0135] The processor 11 performs image processing on each of the multiple captured images captured by the two infrared cameras C12 and C13, and calculates the distances D1 and D2 between the surface of the monitor 13 and the user's fingertip. Based on the calculated distances D1 and D2, the processor 11 determines the sizes of the pointers PT11 and PT12 to be displayed on various screens (authentication start screens SC11 and SC12 in the example shown in FIG. 10) displayed on the monitor 13.

[0136] The pointers PT11 and PT12 are displayed larger as the calculated distance decreases (i.e., the distance between the surface of the monitor 13 and the user's fingertip is closer), and smaller as the calculated distance increases (i.e., the distance between the surface of the monitor 13 and the user's fingertip is farther). For example, on the authentication start screens SC11 and SC12, the distance between the user's fingertip and the surface of the monitor 13 increases from distance D1 to distance D2. In such a case, the processor 11 changes the size of the pointer to be smaller (pointer PT11 → pointer PT12).

[0137] This allows the user to intuitively understand the position or distance of his or her fingertip relative to the surface of the monitor 13 based on the size of the pointer displayed on the monitor 13.

[0138] Next, examples of pointer display on various screens will be described with reference to Fig. 11. Fig. 11 is a diagram illustrating a pointer display example 2 on the first terminal device P1. Note that the size of the pointer shown in Fig. 11 is an example and is not limited to this.

[0139] The processor 11 processes each of the multiple captured images taken by the two infrared cameras C12 and C13, and calculates the distances D3 and D4 between the surface of the monitor 13 and the user's fingertip. The processor 11 determines whether to display a pointer on the monitor 13 based on whether the calculated distances D3 and D4 are within the recognition area or a predetermined distance.

[0140] For example, on the authentication start screen SC13, the processor 11 calculates the position of the user's fingertip (three-dimensional coordinates) and the distance D3 between the surface of the monitor 13 and the user's fingertip. The processor 11 determines that the calculated position of the user's fingertip is not within the recognition area, omits the process of displaying a pointer on the authentication start screen SC13, and generates the authentication start screen SC13 including an instruction MSG1 to the user to insert their fingertip into the recognition area, "Bring your finger closer to the screen until a green pointer appears," and displays the generated screen on the monitor 13.

[0141] If the calculated position of the user's fingertip is not within the recognition area, the calculated distance D3 is determined to be not within the specified distance (that is, distance D3>specified distance).

[0142] Furthermore, the processor 11 calculates the position of the user's fingertip (three-dimensional coordinates) and the distance D4 between the surface of the monitor 13 and the user's fingertip on the authentication start screen SC14. The processor 11 determines that the calculated position of the user's fingertip is within the recognition area and that the calculated distance D4 is not within the specified distance (i.e., distance D4>specified distance), and executes processing to display a pointer PT13 on the authentication start screen SC14, as well as generates an authentication start screen SC14 including an instruction MSG2 "Bring your finger closer to the screen" to prompt the user to insert their fingertip into the recognition area, and displays the generated screen on the monitor 13.

[0143] This allows the first terminal device P1 to guide the position of the user's finger or fingertip to a position suitable for capturing an image used for biometric authentication and a position or distance suitable for a PIN input operation on the PIN authentication screen SC10. Furthermore, by displaying instructions MSG1 and MSG2 or showing / hiding the pointer, the user can understand whether the position and distance of his or her finger or fingertip is a position suitable for capturing an image used for biometric authentication and whether the position and distance are suitable for a PIN input operation on the PIN authentication screen SC10.

[0144] Next, an authentication result screen SC15 that is displayed on the monitor 13 when the authentication result of the multi-factor authentication (i.e., the user authentication result) is "OK" will be described with reference to Fig. 12. Fig. 12 is a diagram illustrating an example of the authentication result screen SC15 when the authentication result is "OK". It goes without saying that the authentication result screen SC15 shown in Fig. 12 is just an example and is not limited to this.

[0145] The authentication result screen SC15 is generated by the processor 11 and output to and displayed on the monitor 13. The processor 11 generates the authentication result screen SC15 when the authentication results of both the biometric authentication and the PIN authentication are "OK" based on the biometric authentication result information transmitted from the biometric authentication device P2 and the PIN authentication result information transmitted from the PIN authentication device P3.

[0146] The authentication result screen SC15 includes authentication result information RST11 "Authentication result OK", detailed information RST12, and a Return button BT12.

[0147] The detailed information RST12 includes information about the fingerprint authentication result and information about the PIN authentication result. The information about the fingerprint authentication result includes, for example, the authentication result "result: OK", information about the user (here, the user name) "name: XXX", and the matching score of the biometric information (fingerprint or vein) "score: 191".

[0148] The information about the PIN authentication result includes, for example, the authentication result "result: OK" and information about the user (here, the user name) "name: XXX".

[0149] Note that the various information included in the detailed information RST12 is merely an example and is not limited to this. Furthermore, the information about the user is not limited to the user name, but may be, for example, a facial image of the user. The matching score may be any numerical value such as 1 to 100 or 0 to 200, a symbol such as O, △, or ×, or an alphabet such as A, B, C, etc. Note that the detailed information RST12 is not essential and may be omitted.

[0150] The Return button BT12 is a button that can accept a user operation to terminate user authentication for this user. When the processor 11 determines that the Return button BT12 has been input (selected) by the user, it generates an authentication start screen SC11 and outputs it to the monitor 13 for display.

[0151] As described above, the multi-factor authentication system 100 can notify the user of the user authentication (multi-factor authentication) result, the biometric authentication result, and the PIN authentication result.

[0152] Next, an authentication result screen SC16 that is displayed on the monitor 13 when the authentication result of the multi-factor authentication (i.e., the user authentication result) is "NG" will be described with reference to Fig. 13. Fig. 13 is a diagram illustrating an example of the authentication result screen SC16 when the authentication result is "NG". It goes without saying that the authentication result screen SC16 shown in Fig. 13 is just an example and is not limited to this.

[0153] The authentication result screen SC16 is generated by the processor 11 and output to and displayed on the monitor 13. The processor 11 generates the authentication result screen SC16 when at least one of the biometric authentication and PIN authentication results is "NG" based on the biometric authentication result information transmitted from the biometric authentication device P2 and the PIN authentication result information transmitted from the PIN authentication device P3.

[0154] The authentication result screen SC16 includes authentication result information RST21 "Authentication result NG", detailed information RST22, and a Return button BT12.

[0155] The detailed information RST22 includes information about the fingerprint authentication result and information about the PIN authentication result. The information about the fingerprint authentication result includes, for example, the authentication result "result: NG", information about the user (here, the user name) "name: unknown", and the matching score of the biometric information (fingerprint or vein) "score: 125".

[0156] The information about the PIN authentication result includes, for example, the authentication result "result: NG" and information about the user (here, the user name) "name: unknown."

[0157] It should be noted that the various types of information included in the detailed information RST22 are merely examples and are not limited to these. Furthermore, the information about the user is not limited to the user name, but may be, for example, a facial image of the user. The matching score may be any numerical value such as 1 to 100 or 0 to 200, a symbol such as O, △, X, or alphabets such as A, B, C, etc. It should be noted that the detailed information RST22 is not essential and may be omitted.

[0158] As described above, the multi-factor authentication system 100 can notify the user of the user authentication (multi-factor authentication) result, the biometric authentication result, and the PIN authentication result.

[0159] As described above, the multi-factor authentication system 100 according to the first embodiment includes a finger position detection device C1 (an example of a detection device) that detects a predetermined part (e.g., a user's finger or fingertip) of a user (an example of a person to be authenticated), a camera C2 (an example of a sensor) that acquires information about the predetermined part (e.g., a captured image), a biometric authentication device P2 (an example of a first authentication device) that can communicate with the camera C2 and executes a biometric authentication process (an example of a first authentication process), a PIN authentication device P3 (an example of a second authentication device) that can communicate with the finger position detection device C1 and executes a PIN authentication process (an example of a second authentication process) that is different from the biometric authentication process, and a first terminal device P1 (an example of a terminal device) that can communicate among the finger position detection device C1, the biometric authentication device P2, and the PIN authentication device P3. The biometric authentication device P2 executes the biometric authentication process based on the information about the predetermined part acquired by the camera C2. The first terminal device P1 accepts a contactless input of a PIN (an example of authentication information) by the user and transmits it to the PIN authentication device P3. The PIN authentication device P3 executes PIN authentication processing based on the PIN. The first terminal device P1 executes user authentication based on the results of the biometric authentication processing and the PIN authentication processing. The multi-factor authentication system 100 executes the biometric authentication processing and the PIN acquisition in parallel.

[0160] As a result, the multi-factor authentication system 100 according to the first embodiment can acquire biometric information used for biometric authentication and a PIN used for PIN authentication, without the user's fingers coming into contact with the monitor 13 and the camera C2. Furthermore, when performing user authentication based on biometric authentication and PIN authentication (multi-factor authentication), the multi-factor authentication system 100 performs the biometric authentication process, and the acceptance of the input operation of the PIN used for PIN authentication and the acquisition of the PIN in parallel, thereby further shortening the time required for multi-factor authentication and making the overall processing of multi-factor authentication more efficient.

[0161] As described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment displays various screens (examples of input screens, such as the PIN authentication screen SC10) capable of accepting PIN input on the monitor 13, and accepts the PIN input based on the position of the predetermined part detected by the finger position detection device C1. This allows the multi-factor authentication system 100 according to the first embodiment to accept the PIN input without the user's fingers touching the monitor 13.

[0162] As described above, the camera C2 in the multi-factor authentication system 100 according to the first embodiment is disposed between the monitor 13 and the finger position detection device C1. As a result, in the multi-factor authentication system 100 according to the first embodiment, the camera C2 that captures an image used for biometric authentication is disposed at a position closer to the monitor 13 than the finger position detection device C1, making it easier to capture an image of a predetermined part (finger or fingertip) of the user operating the monitor 13. Therefore, the multi-factor authentication system 100 can capture an image that is more suitable for biometric authentication, thereby improving the accuracy of biometric authentication.

[0163] As described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment calculates the position of the predetermined part detected by the finger position detection device C1, generates a pointer PT indicating the position of an operation by the user based on the position of the predetermined part, and displays the pointer PT on various screens at the position of the predetermined part on the various screens. This allows the multi-factor authentication system 100 according to the first embodiment to visualize the user operation position indicated by the user's finger or fingertip, thereby assisting the user in entering a PIN without contact. The user can understand the current operation position on various screens displayed on the monitor 13 even in a contactless state.

[0164] As described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment determines whether to display or hide the pointer PT on various screens based on the position of a predetermined body part. Thus, when the user's finger or fingertip is in a position that is not suitable for capturing an image used for biometric authentication or for accepting a PIN input operation, the multi-factor authentication system 100 according to the first embodiment can notify the user that a PIN input operation is not being accepted by hiding the pointer PT, or when the user's finger or fingertip is in a position that is suitable for capturing an image used for biometric authentication or for accepting a PIN input operation, the multi-factor authentication system 100 can display the pointer PT to notify the user that a PIN input operation is being accepted. In other words, the user can intuitively understand whether a PIN input operation is possible or not by viewing or hiding the pointer PT.

[0165] Furthermore, as described above, when the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment determines, based on the position of the predetermined part, that the distance between the monitor 13 and the predetermined part is within a specified distance, the first terminal device P1 displays the pointer PT on various screens. This allows the multi-factor authentication system 100 according to the first embodiment to notify the user that a PIN input operation is being accepted by displaying the pointer PT. The display of the pointer PT allows the user to intuitively understand the current operation position indicated by the user's fingertip and that a PIN input operation is possible.

[0166] Furthermore, as described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment changes the size of the pointer PT displayed on various screens based on the position of the predetermined body part. As a result, the multi-factor authentication system 100 according to the first embodiment can notify the position or distance of the user's fingertip relative to the surface of the monitor 13 based on the size of the pointer displayed on the monitor 13. The user can intuitively understand the position or distance of the user's fingertip relative to the surface of the monitor 13.

[0167] Furthermore, as described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment calculates the position of the predetermined part detected by the finger position detection device C1, and when it determines based on the position of the predetermined part that the predetermined part is not within the imaging area AR2 (an example of an acquisition area) where information about the predetermined part can be acquired (imaged) by the camera C2, it generates and displays various screens (e.g., the authentication start screen SC13 shown in FIG. 11 ) including an instruction (e.g., instruction MSG1) to move the predetermined part into the imaging area AR2 of the camera C2. This allows the multi-factor authentication system 100 according to the first embodiment to determine whether the user's finger or fingertip is located in a position where the biometric authentication process, the acceptance of a PIN input operation used for PIN authentication, and the acquisition of the PIN can be performed in parallel. Furthermore, the user can intuitively understand based on the instruction MSG1 whether the position of the user's finger or fingertip is suitable for biometric authentication and PIN input.

[0168] Furthermore, as described above, when the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment determines that the predetermined part is within the imaging area AR2 of the camera C2, it generates and displays various screens including a finger still instruction (an example of an instruction) requesting that the predetermined part be still. As a result, the multi-factor authentication system 100 according to the first embodiment can more effectively suppress deterioration of the image quality of the captured image (for example, camera shake) by temporarily keeping the user's hand or fingertip still.

[0169] As described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment generates a control command to cause the finger position detection device C1 to suspend (turn off) detection of a predetermined part and start biometric authentication processing, and transmits the generated control command to the biometric authentication device P2. Based on the control command, the biometric authentication device P2 causes the camera C2 to acquire information about the predetermined part (capture an image). As a result, the multi-factor authentication system 100 according to the first embodiment can more effectively suppress deterioration in image quality (for example, flicker) of the captured image by using the LED C11 (infrared light) of the finger position detection device C1.

[0170] As described above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment starts accepting inputs to various screens after causing the finger position detection device C1 to resume (ON) detection of the predetermined part. This enables the multi-factor authentication system 100 according to the first embodiment to detect the position of the user's finger or fingertip with higher accuracy and to more accurately acquire the PIN entered by the user.

[0171] As a result of the above, the first terminal device P1 in the multi-factor authentication system 100 according to the first embodiment authenticates the user when the results of both the biometric authentication process and the PIN authentication process are successful (i.e., the authentication result is "OK"). As a result, the multi-factor authentication system 100 according to the first embodiment can perform multi-factor authentication based on biometric authentication and PIN authentication without the user's fingers coming into contact with the monitor 13 and the camera C2.

[0172] As described above, the multi-factor authentication system 100 (an example of a multi-factor authentication information acquisition system) according to the first embodiment includes a camera C2 (an example of a sensor) that acquires information about a specific body part (e.g., a user's finger or fingertip) used in a biometric authentication process (an example of a first authentication process) of a user (an example of a person to be authenticated), a finger position detection device C1 (an example of a detection device) that detects the specific body part, and a first terminal device P1 (an example of a terminal device) that accepts input of a PIN (an example of authentication information) used in a PIN authentication process (an example of a second authentication process) that is different from the biometric authentication process based on the position of the specific body part detected by the finger position detection device C1, and acquires the PIN (an example of authentication information).

[0173] As a result, the multi-factor authentication system 100 according to the first embodiment can acquire biometric information used for biometric authentication and a PIN used for PIN authentication without the user's fingers coming into contact with the monitor 13 and the camera C2.

[0174] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0175] This application is based on a Japanese patent application (Patent Application No. 2022-114029) filed on July 15, 2022, the contents of which are incorporated herein by reference. [Industrial Applicability]

[0176] The present disclosure is useful as a proposal for a multi-factor authentication system, a multi-factor authentication method, and a multi-factor authentication information acquisition system that more effectively prevent leakage of authentication-related information used for personal authentication. [Explanation of symbols]

[0177] 10, 20, 30, 40 Communications Department 11,21,31,41 processors 12, 22, 32, 42 memory 13 Monitor 100 Multi-Factor Authentication Systems 111 Finger position detection control unit 112 Finger position determination section 113 Image processing section 114 PIN input processing section 211,311,411 Command analysis section 212 Biometric Information Authentication Department 312 PIN authentication section C1 Finger position detection device C2 Camera C11 LED C12, C13 infrared camera DB1 Biological Information Database DB2 PIN database L1 Lighting NW Network

Claims

1. a detection device for detecting a predetermined part of a person to be authenticated; a sensor for acquiring information about the predetermined part; a first authentication device capable of communicating with the sensor and executing a first authentication process; a second authentication device capable of communicating with the detection device and performing a second authentication process different from the first authentication process; A multi-factor authentication system including the detection device, and a terminal device capable of communicating between the first authentication device and the second authentication device, the first authentication device, performing the first authentication process based on the information about the predetermined body part acquired by the sensor; The terminal device accepting contactless input of authentication information by the person to be authenticated based on the position of the predetermined part detected by the detection device, and transmitting the input to the second authentication device; the second authentication device, performing the second authentication process based on the authentication information; The terminal device performing authentication of the person to be authenticated based on a result of the first authentication process and a result of the second authentication process; The multi-factor authentication system comprises: The first authentication process and the acquisition of the authentication information are executed in parallel. Multi-factor authentication system.

2. The terminal device displaying an input screen on a monitor that can accept input of the authentication information; accepting input of the authentication information based on the position of the predetermined part detected by the detection device; The multi-factor authentication system of claim 1 .

3. The sensor is disposed between the monitor and the detection device. The multi-factor authentication system of claim 2 .

4. The terminal device calculating a position of the predetermined part detected by the detection device; generating a pointer indicating an operation position by the person to be authenticated based on the position of the predetermined part, and displaying the pointer on the input screen at a position corresponding to the position of the predetermined part on the input screen; The multi-factor authentication system of claim 2 .

5. The terminal device determining whether to display or hide the pointer on the input screen based on the position of the predetermined portion; The multi-factor authentication system of claim 4 .

6. The terminal device When it is determined based on the position of the predetermined part that the distance between the monitor and the predetermined part is within a specified distance, the pointer is displayed on the input screen. The multi-factor authentication system of claim 5 .

7. The terminal device changing the size of the pointer displayed on the input screen based on the position of the predetermined part; The multi-factor authentication system of claim 4 .

8. The terminal device calculating a position of the predetermined part detected by the detection device; when it is determined based on the position of the predetermined part that the predetermined part is not within an acquisition area of ​​the sensor where information about the predetermined part can be acquired, an input screen including an instruction to move the predetermined part into the acquisition area of ​​the sensor is generated and displayed. The multi-factor authentication system of claim 2 .

9. The terminal device When it is determined that the predetermined part is within the acquisition area of ​​the sensor, an input screen including an instruction to request that the predetermined part be kept still is generated and displayed. The multi-factor authentication system of claim 8 .

10. The terminal device generating a control command to cause the detection device to suspend detection of the predetermined part and start the first authentication process, and transmitting the control command to the first authentication device; the first authentication device, causing the sensor to acquire information about the predetermined part based on the control command; The multi-factor authentication system of claim 1 .

11. The terminal device and after causing the detection device to resume detection of the predetermined portion, starting to accept the input on the input screen. The multi-factor authentication system of claim 2 .

12. The terminal device when the result of the first authentication process and the result of the second authentication process are both successful authentication, authenticate the person to be authenticated; The multi-factor authentication system of claim 1 .

Images