Authentication server, authentication system, authentication device, authentication method, and authentication program

Abstract

This invention provides an authentication server, authentication system, authentication device, authentication method, and authentication program that can reduce processing load while ensuring security. [Solution] The authentication server 2 includes a processor 11 that acquires captured images and performs authentication processing. Based on processing instruction information 42, the processor selectively executes, for each frame constituting the captured image, a biometric information matching process that compares it with registered biometric information and determines whether the person to be authenticated is a registered person, a body expression matching process that determines whether the body expression of the person to be authenticated satisfies the conditions, and a liveness determination process that confirms whether it is a living being, thereby authenticating the person to be authenticated. The processing instruction information indicates whether or not the biometric information matching process, body expression matching process, and liveness determination process should be executed for each frame, and each process is set to be executed in at least one frame.

Description

Technical Field

[0001] The present disclosure relates to an authentication server, an authentication system, an authentication device, an authentication method, and an authentication program for authenticating an authentication target person.

Background Art

[0002] In recent years, services using face recognition technology (hereinafter referred to as face recognition services) have become widespread in order to manage access to facilities and perform payment procedures for goods and services. In face recognition technology, face matching is performed by comparing the face information of an authentication target person (i.e., the person to be authenticated) with information regarding a pre-registered face image (hereinafter referred to as a registered image). Thereby, it is possible to determine whether the authentication target person is a legitimate registered person.

[0003] On the other hand, by making a legitimate user load a face image (for example, a photograph), fraud may occur where the person pretends to be that person and passes the authentication. In order to prevent such fraud, in addition to normal face recognition processing, a face recognition device is known that instructs a person to take a specific expression and determines that the expression estimated from the captured image satisfies a predetermined condition (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] If the normal face recognition processing described in Patent Document 1 and other authentication processes are always performed, a highly secure face recognition device can be configured. However, since the processing load increases, a technique capable of reducing the processing load while ensuring security has been demanded.

[0006] Therefore, the main purpose of this disclosure is to provide an authentication server, authentication system, authentication device, authentication method, and authentication program that can reduce processing load while ensuring security. [Means for solving the problem]

[0007] The authentication server of this disclosure includes a processor that acquires an image captured by an imaging device, including an object to be authenticated, and performs authentication-related processing. The processor selectively performs, based on predetermined processing instruction information, a biometric information matching process for each frame constituting the image, which involves acquiring biometric information of the object to be authenticated, comparing the acquired biometric information with pre-registered registered biometric information, and determining whether the object to be authenticated is a registered person; a body expression matching process, which involves acquiring a body expression of the object to be authenticated and determining whether the acquired body expression satisfies predetermined conditions; and a liveness determination process, which confirms whether the object to be authenticated is a living organism. Based on the results of these processes, the server authenticates the object to be authenticated. The processing instruction information indicates whether the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is configured to be performed in at least one of the frames.

[0008] The authentication system of this disclosure comprises an authentication terminal equipped with an imaging device that acquires an image of an object to be authenticated captured by the imaging device, and an authentication server that acquires the image from the authentication terminal and performs authentication-related processing. The authentication server selectively performs, based on predetermined processing instruction information, a biometric information matching process for each frame constituting the image of the captured image, which involves acquiring biometric information of the object to be authenticated, comparing the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person, a body expression matching process for acquiring a body expression of the object to be authenticated and determining whether the acquired body expression satisfies predetermined conditions, and a liveness determination process to confirm whether the object to be authenticated is a living organism. Based on the results of these processes, the authentication server authenticates the object to be authenticated. The processing instruction information indicates whether the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is configured to be performed in at least one of the frames.

[0009] The authentication device of this disclosure includes a processor that acquires an image captured by an imaging device, including an object to be authenticated, and performs authentication processing. The processor selectively performs, based on predetermined processing instruction information, a biometric information matching process for each frame constituting the image, which involves acquiring biometric information of the object to be authenticated, comparing the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person; a body expression matching process, which involves acquiring a body expression of the object to be authenticated and determining whether the acquired body expression satisfies predetermined conditions; and a liveness determination process, which confirms whether the object to be authenticated is a living organism. Based on the results of these processes, the device authenticates the object to be authenticated. The processing instruction information indicates whether the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is configured to be performed in at least one of the frames.

[0010] The authentication method of this disclosure is performed by a processor that acquires an image captured by an imaging device, including the object to be authenticated, and performs authentication-related processing. The processor selectively performs the following processes for each frame constituting the image based on predetermined processing instruction information: a biometric information matching process that acquires biometric information of the object to be authenticated, compares the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person; a body expression matching process that acquires a body expression of the object to be authenticated and determines whether the acquired body expression satisfies predetermined conditions; and a liveness determination process that confirms whether the object to be authenticated is a living organism. Based on the results of these processes, the processor authenticates the object to be authenticated. The processing instruction information indicates whether the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is configured to be performed in at least one frame.

[0011] The authentication program of this disclosure is an authentication program that acquires an image captured by an imaging device, including an object to be authenticated, and causes a computer to perform information processing related to authentication. The information processing includes, based on predetermined processing instruction information, selectively executing for each frame constituting the image: a biometric information matching process that acquires biometric information of the object to be authenticated, compares the acquired biometric information with pre-registered registered biometric information, and determines whether the object to be authenticated is a registered person; a body expression matching process that acquires a body expression of the object to be authenticated and determines whether the acquired body expression satisfies predetermined conditions; and a liveness determination process that confirms whether the object to be authenticated is a living organism. Based on the results of these processes, the program authenticates the object to be authenticated. The processing instruction information indicates whether the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is configured to be performed in at least one of the frames. [Effects of the Invention]

[0012] This disclosure makes it possible to provide an authentication server, authentication system, authentication device, authentication method, and authentication program that can reduce processing load while ensuring security. [Brief explanation of the drawing]

[0013] [Figure 1] Overall configuration diagram of the authentication system according to the first embodiment [Figure 2] Block diagram showing the configuration of the authentication server shown in Figure 1. [Figure 3] The authentication server's functional block diagram is shown in Figure 1. [Figure 4] Figure 2 is an explanatory diagram showing an example of a processing information table. [Figure 5] This figure shows the first example of the settings screen for the processing information table displayed on the administrator terminal. [Figure 6] This diagram shows an example of a warning screen that may be displayed on the administrator terminal when configuring the processing information table. [Figure 7] This figure shows a second example of the settings screen for the processing information table displayed on the administrator terminal. [Figure 8] An example of a pull-down menu item for biometric information matching processing, and an example of a table showing the relationship between each item, the corresponding matching method, and the corresponding matching environment. [Figure 9] An example of a pull-down menu item corresponding to the body expression matching process, and an example of a table showing the relationship between the corresponding body expression instruction (expression instruction) and the corresponding judgment method. [Figure 10] Block diagram showing the configuration of the user terminal [Figure 11] User terminal functional block diagram [Figure 12] Block diagram showing the configuration of the administrator terminal. [Figure 13] Functional block diagram of the administrator terminal [Figure 14] An example of an error display screen shown on the administrator terminal's display unit. [Figure 15] An example of a history display screen shown on the administrator terminal's display unit. [Figure 16]An example of a pop-up showing the details of the processing result of the authentication process [Figure 17] In the setting process of the processing instruction information, when the setting information that satisfies the setting condition (A) is input, and when the setting information that satisfies the setting condition (B) is input once, a sequence diagram showing the operations of the respective authentication servers and the administrator terminal [Figure 18] A flowchart showing the flow of the setting process of the processing instruction information in the authentication server [Figure 19] A sequence diagram showing the operations of the authentication server and the user terminal in the authentication process [Figure 20] A flowchart showing the flow of the authentication process in the authentication server [Figure 21] An explanatory diagram showing an example of the physical expression performed by the authentication target in the authentication process and the timing of the process [Figure 22] In the authentication system according to the second embodiment, a sequence diagram showing the operations of the authentication server and the user terminal after the authentication server acquires the start request signal [Figure 23] In the authentication system of the second embodiment, a flowchart showing the flow of the process in the authentication server after the authentication server acquires the start request signal [Figure 24] An explanatory diagram showing an example of (A) the first processing information table, (B) the second processing information table, and (C) the third processing information table [Figure 25] An example of a group setting screen for grouping user terminals [Figure 26] An example of a processing instruction setting screen for inputting the association between each group and the processing instruction table [Figure 27] A functional block diagram showing modified examples of the authentication server and the user terminal according to the first embodiment and the second embodiment [Figure 28] An explanatory diagram showing a modified example of the physical expression performed by the authentication target in the authentication process and the timing of the process

Mode for Carrying Out the Invention

[0014] The first invention made to solve the above problem is an authentication server (2) comprising a processor (11) that acquires an image captured by an imaging device (4) including an object to be authenticated and performs authentication processing, wherein the processor selectively executes, based on predetermined processing instruction information (42), a biometric information matching process for each frame constituting the image, which involves acquiring biometric information of the object to be authenticated, comparing the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person, a body expression matching process which involves acquiring a body expression of the object to be authenticated and determining whether the acquired body expression satisfies predetermined conditions, and a liveness determination process which confirms whether the object to be authenticated is a living being, and authenticates the object to be authenticated based on the results of these processes, wherein the processing instruction information indicates whether or not the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is set to be performed in at least one of the frames.

[0015] According to this, processing is performed on each frame constituting the captured image based on processing instruction information indicating whether or not biometric information matching, body expression matching, and liveness determination processing are required. Therefore, by changing the processing instruction information, the number of times and frames for which each of the biometric information matching, body expression matching, and liveness determination processes are executed can be set, thereby reducing the processing load while ensuring security.

[0016] Furthermore, the second invention further includes a communication device (13) for communicating with an administrator terminal (5) used by an administrator, and the processor is configured to acquire setting information input at the administrator terminal via the communication device and to set the processing instruction information corresponding to the setting information.

[0017] According to this, processing instruction information can be set by input from the administrator terminal.

[0018] Furthermore, the third invention is configured such that, in the second invention, when the number of executions of at least one of the biometric information matching process, the bodily expression matching process, and the liveness determination process in the processing instruction information corresponding to the setting information is less than a predetermined threshold, the processor transmits warning information to the administrator terminal via the communication device.

[0019] According to this, when the number of executions of at least one of the biometric information matching process, body expression matching process, and liveness determination process falls below a threshold and security can no longer be ensured, a warning message can be sent to the administrator terminal.

[0020] Furthermore, the fourth invention is configured such that, in the second invention, when the number of executions of the biometric information matching process in the processing instruction information corresponding to the setting information is greater than either the number of executions of the bodily expression matching process or the number of executions of the liveness determination process, the processor transmits warning information to the administrator terminal via the communication device.

[0021] According to this, if the number of executions of the biometric information matching process is greater than the number of executions of either the body expression matching process or the liveness determination process, and the processing load is high, a warning message can be sent to the administrator terminal.

[0022] Furthermore, the fifth invention is configured such that, in the first invention, the processor sets the processing instruction information based on at least one of the date and time of authentication and the day of the week on which authentication is performed.

[0023] According to this, processing instruction information can be set according to the date, time, and day of the week.

[0024] Furthermore, the sixth invention is configured such that, in the first invention, the processor sets the processing instruction information based on the captured image acquired by the imaging device.

[0025] According to this, processing instruction information can be appropriately set according to the status of the object being authenticated.

[0026] Furthermore, the seventh invention further comprises a communication device for communicating with an authentication terminal equipped with the imaging device, and the processor is configured to set the processing instruction information based on the identification information of the authentication terminal obtained via the communication device.

[0027] According to this, processing instruction information can be set for each authentication terminal.

[0028] Furthermore, the eighth invention is an authentication system (1) comprising an imaging device (4) and an authentication terminal (3) which acquires an image captured by the imaging device including an object to be authenticated, and an authentication server (2) which acquires the image captured from the authentication terminal and performs authentication processing, wherein the authentication server selectively performs, based on predetermined processing instruction information (42), a biometric information matching process for each frame constituting the image, which acquires biometric information of the object to be authenticated, compares the acquired biometric information with pre-registered registered biometric information and determines whether the object to be authenticated is a registered person, a body expression matching process which acquires a body expression of the object to be authenticated and determines whether the acquired body expression satisfies predetermined conditions, and a liveness determination process which confirms whether the object to be authenticated is a living being, and authenticates the object to be authenticated based on the results of these processes, wherein the processing instruction information indicates whether or not the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is set to be performed in at least one of the frames.

[0029] According to this, processing is performed on each frame constituting the captured image based on processing instruction information indicating whether or not biometric information matching, body expression matching, and liveness determination processing are required. Therefore, by changing the processing instruction information, the number of times and frames for which each of the biometric information matching, body expression matching, and liveness determination processes are executed can be set, thereby reducing the processing load while ensuring security.

[0030] Furthermore, the ninth invention is an authentication device (3) comprising a processor (11) that acquires an image captured by an imaging device (4) including an object to be authenticated and performs authentication processing, wherein the processor selectively executes, based on predetermined processing instruction information (42), a biometric information matching process for each frame constituting the image, which acquires biometric information of the object to be authenticated, compares the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person, a body expression matching process which acquires a body expression of the object to be authenticated and determines whether the acquired body expression satisfies predetermined conditions, and a liveness determination process which confirms whether the object to be authenticated is a living organism, and authenticates the object to be authenticated based on the results of these processes, wherein the processing instruction information indicates whether or not the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is set to be performed in at least one of the frames.

[0031] According to this, processing is performed on each frame constituting the captured image based on processing instruction information indicating whether or not biometric information matching, body expression matching, and liveness determination processing are required. Therefore, by changing the processing instruction information, the number of times and frames for which each of the biometric information matching, body expression matching, and liveness determination processes are executed can be set, thereby reducing the processing load while ensuring security.

[0032] Furthermore, the tenth invention is implemented by a processor (11) that acquires an image captured by an imaging device (4) including an object to be authenticated and performs authentication processing. Based on predetermined processing instruction information (42), the processor selectively performs the following for each frame constituting the image: a biometric information matching process that acquires biometric information of the object to be authenticated, compares the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person; a body expression matching process that acquires a body expression of the object to be authenticated and determines whether the acquired body expression satisfies predetermined conditions; and a liveness determination process that confirms whether the object to be authenticated is a living organism. Based on the results of these processes, the processor authenticates the object to be authenticated. The processing instruction information indicates whether the biometric information matching process, the body expression matching process, and the liveness determination process should be performed for each frame, and each process is set to be performed in at least one of the frames.

[0033] According to this, processing is performed on each frame constituting the captured image based on processing instruction information indicating whether or not biometric information matching, body expression matching, and liveness determination processing are required. Therefore, by changing the processing instruction information, the number of times and frames for which each of the biometric information matching, body expression matching, and liveness determination processes are executed can be set, thereby reducing the processing load while ensuring security.

[0034] Furthermore, the 11th invention is an authentication program that acquires an image captured by an imaging device (4) including an object to be authenticated, and causes a computer to perform information processing related to authentication, wherein the information processing includes, based on predetermined processing instruction information (42), selectively executing for each frame constituting the image a biometric information matching process that acquires biometric information of the object to be authenticated, compares the acquired biometric information with pre-registered registered biometric information to determine whether the object to be authenticated is a registered person, acquires a physical representation of the object to be authenticated, and determines whether the acquired physical representation satisfies predetermined conditions, and a liveness determination process that confirms whether the object to be authenticated is a living organism, and authenticating the object to be authenticated based on the results of these processes, wherein the processing instruction information indicates whether or not the biometric information matching process, the physical representation matching process, and the liveness determination process should be performed for each frame, and each process is set to be performed in at least one of the frames.

[0035] According to this, processing is performed on each frame constituting the captured image based on processing instruction information indicating whether or not biometric information matching, body expression matching, and liveness determination processing are required. Therefore, by changing the processing instruction information, the number of times and frames for which each of the biometric information matching, body expression matching, and liveness determination processes are executed can be set, thereby reducing the processing load while ensuring security.

[0036] The embodiments of this disclosure will be described below with reference to the drawings.

[0037] (First Embodiment) Figure 1 is an overall configuration diagram of the authentication system according to the first embodiment of this disclosure. Figure 2 is a block diagram showing the configuration of the authentication server. Figure 3 is a functional block diagram of the authentication server. Figure 4 is an explanatory diagram showing an example of the processing information table shown in Figure 2.

[0038] Figure 5 shows the first example of the processing information table settings screen, and Figure 6 shows an example of the warning screen. Figure 7 shows the second example of the processing information table settings screen.

[0039] Figure 8 shows an example of a pull-down menu item corresponding to biometric information matching processing, along with an example table showing the relationship between the corresponding matching method and the corresponding matching environment. Figure 9 shows an example of a pull-down menu item corresponding to bodily expression matching processing, along with an example table showing the relationship between the corresponding bodily expression instruction (expression instruction) and the corresponding judgment method.

[0040] Figure 10 is a block diagram showing the configuration of a user terminal. Figure 11 is a functional block diagram of a user terminal. Figure 12 is a block diagram showing the configuration of an administrator terminal, and Figure 13 is a functional block diagram of an administrator terminal.

[0041] As shown in Figure 1, the authentication system 1 includes an authentication server 2 operated by a provider of authentication services. The authentication system 1 further includes one or more user terminals 3. Each user terminal 3 is used by a user who is the subject of authentication (authenticated person) of the authentication service. The user terminal 3 functions as an authentication device that receives input from the subject of authentication, causes the authentication server 2 to perform authentication, and displays the authentication result. Each user terminal 3 may be owned and used by a corresponding specific subject of authentication. Each user terminal 3 is equipped with an imaging device 4 for imaging the subject of authentication. The subject of authentication may include, for example, an operator who operates the user terminal 3.

[0042] The authentication system 1 may further include at least one administrator terminal 5 used by an administrator who manages the authentication service.

[0043] The authentication server 2 and each user terminal 3 can communicate with each other via a publicly known communication network 6 such as the Internet. However, the authentication server 2 and each user terminal 3 may also send and receive necessary information and data to and from each other via other information processing devices (i.e., indirectly) not shown.

[0044] The authentication server 2 is a computer for performing the information processing necessary to provide authentication services, and as shown in Figure 2, it comprises a processor 11, a storage device 12, and a communication device 13. The processor 11 is composed of a CPU or MPU, and the storage device 12 may consist of memory 14 such as RAM or ROM and storage 15 such as HDD or SSD. The communication device 13 includes a network interface equipped with an antenna and communication circuits. In addition, the authentication server 2 may be equipped with input devices such as a mouse or keyboard, and known hardware such as a display (neither of which are shown).

[0045] The authentication server 2 obtains information related to the object to be authenticated from the image captured by the imaging device 4 and performs three determinations using that information. When the result of each determination satisfies predetermined conditions, it is determined that the object to be authenticated should be authenticated. In other words, the authentication server 2 functions as an authentication device that obtains an image containing the object to be authenticated, captured by the imaging device 4, and determines whether or not the object to be authenticated should be authenticated. Hereinafter, the information processing that obtains an image, determines whether or not the object to be authenticated should be authenticated, and performs authentication based on the determination result will be referred to as the authentication process.

[0046] The first type of determination is biometric determination based on biometric information. Authentication server 2 stores the biometric information of individuals registered in authentication system 1 as registered biometric information. Authentication server 2 obtains the biometric information of the person to be authenticated from the imaging results of imaging device 4. Subsequently, authentication server 2 compares the obtained biometric information with the registered biometric information (also called biometric information comparison) and determines whether the obtained biometric information matches the biometric information of the registered person.

[0047] Hereinafter, the process by which authentication server 2 acquires the biometric information of the person to be authenticated, compares the acquired biometric information with pre-registered biometric information, and determines whether or not the person to be authenticated is a registered person will be referred to as the biometric information comparison process.

[0048] In this embodiment, the registered biometric information is the facial information of the registered person (hereinafter referred to as "facial registration information"), which is a so-called facial feature. The authentication server 2 performs facial matching using the facial registration information and the image of the person to be authenticated obtained from the user terminal 3. However, the biometric information matching performed by the authentication system 1 is not limited to facial information, and other biometric information-based matching methods may be employed. Other biometric information-based matching methods may include matching based on iris information such as iris pattern, shape, and intensity; matching based on fingerprint information such as fingerprint pattern, shape, and characteristics; matching using vein patterns in the palms, wrists, forehead, eyelids, etc.; and matching using pulse rate.

[0049] The second determination is a body expression determination (also called body expression matching), which involves obtaining the body expression to be authenticated and determining whether the obtained body expression meets the prescribed conditions.

[0050] In this embodiment, the user terminal 3 instructs the person to be authenticated to perform a predetermined physical expression (hereinafter referred to as an instruction expression), captures an image using the imaging device 4, and transmits it to the authentication server 2. The authentication server 2 obtains the physical expression of the person to be authenticated from the imaging result and determines whether the physical expression satisfies predetermined conditions. This determines whether the captured image of the person performed a physical expression that matches the instructed content. The physical expressions instructed by the user terminal 3 (hereinafter referred to as an instruction expression) may include, for example, closing the eyes, opening and closing the mouth, turning left or right, covering the mouth with a hand, etc.

[0051] Furthermore, the term "bodily expression" here includes the posture, facial expression, and state taken as a result of the movement (action) being authenticated. It may also include the posture, facial expression, and state taken during the movement being authenticated.

[0052] For example, if the instruction is to open the mouth, the acquired bodily expression is determined to meet the predetermined conditions if it is determined that the subject of authentication made the movement of opening the mouth or if it is determined that the subject of authentication assumed a posture with the mouth open.

[0053] Hereinafter, the process by which authentication server 2 obtains the body representation to be authenticated and determines whether the obtained body representation satisfies predetermined conditions will be referred to as the body representation matching process.

[0054] The third determination is a so-called liveness determination, which confirms whether the subject being authenticated, as captured by the imaging device 4, is a living human being (i.e., a living organism). If the imaging device 4 captures, for example, a person's image displayed on a tablet device, it is determined that the person's image captured by the imaging device 4 is not a living organism.

[0055] Various known methods can be applied to the liveness determination performed by the authentication server 2. For example, one method of liveness determination may involve acquiring reflection information and texture information from captured images and using a machine learning model to determine whether or not the subject to authentication is a living organism. Other methods of liveness determination may include liveness determination using temperature sensors or infrared sensors, skin texture analysis, or point cloud imaging.

[0056] In addition, the liveness determination performed by the authentication server 2 may also be based on pulse or heart rate. Specifically, the authentication server 2 may determine whether or not the person being authenticated is a living being by analyzing subtle color changes and blood flow in the face and fingers based on the captured image to obtain pulse or heart rate.

[0057] The liveness determination performed by authentication server 2 may also involve instructing the authentication target to perform a predetermined action and determining whether the instructed action has been performed. Furthermore, the liveness determination performed by authentication server 2 may be based on multiple methods.

[0058] Hereafter, the process performed by authentication server 2 to verify whether the subject of authentication is a living organism will be referred to as the liveness determination process.

[0059] After the authentication process begins, the authentication server 2 takes images using the imaging device within a predetermined time (hereinafter referred to as the imaging time) and executes the biometric information matching process, bodily expression matching process, and liveness determination process at the corresponding timings (i.e., selectively). Subsequently, the authentication server 2 determines whether or not to authenticate the subject based on the results of each determination. However, within the imaging time, the biometric information matching process, bodily expression matching process, and liveness determination process are executed at least once. The authentication server 2 sends the determination result of whether or not to authenticate (i.e., the authentication result) to the user terminal 3.

[0060] As shown in Figure 3, the authentication server 2 has a communication unit 21, a storage unit 22, and a control unit 23 as functional units for executing this series of processes. The communication unit 21 may mainly consist of a communication device 13. The control unit 23 may be configured by the processor 11 executing an authentication program and performing authentication processing in order to implement the authentication method. The storage unit 22 may mainly consist of a storage device 12.

[0061] The communication unit 21 communicates wirelessly or via wired connection with other devices (in this case, user terminal 3, administrator terminal 5, etc.) via the communication network 6 in accordance with a known communication protocol.

[0062] The memory unit 22 stores registered biometric information and various information necessary to perform a series of processes. The registered biometric information may be pre-stored in the memory unit 22.

[0063] The control unit 23 comprises a GUI control unit 31, a registration processing unit 32, an image acquisition unit 33, a processing instruction unit 34, a biometric information matching unit 35, a bodily expression matching unit 36, a liveness determination unit 37, an authentication determination unit 38, a result notification unit 39, and an instruction information setting unit 40. The image acquisition unit 33, the processing instruction unit 34, the biometric information matching unit 35, the bodily expression matching unit 36, the liveness determination unit 37, the authentication determination unit 38, the result notification unit 39, and the instruction information setting unit 40 can each be configured by the processor 11 cooperating with the storage device 12 to execute an authentication program.

[0064] The GUI control unit 31 can control the GUI (Graphical User Interface) of the user terminal 3 by transmitting various information to the user terminal 3 via the communication unit 21.

[0065] For example, the GUI control unit 31 can send various information to the user terminal 3 to generate a screen, and the user terminal 3 can display various information. In this way, the GUI control unit 31 can control the screen display on the user terminal 3. The GUI control unit 31 can cause the user terminal 3 to accept input from the authentication target and send the acquired input information to the user terminal 3.

[0066] In the authentication process, the GUI control unit 31 transmits information indicating an instruction expression for the authentication target in the physical representation matching process to the user terminal 3. Upon receiving the information indicating the instruction expression, the user terminal 3 displays the corresponding expression instruction on its display at an appropriate time.

[0067] The GUI control unit 31, like the user terminal 3, can transmit various information to the administrator terminal 5 via the communication unit 21, thereby displaying various information on the GUI (Graphical User Interface) of the administrator terminal 5. This allows the GUI control unit 31 to control the screen display of the administrator terminal 5. The GUI control unit 31 can also cause the administrator terminal 5 to accept input from the administrator and transmit the acquired input information to the administrator terminal 5.

[0068] The registration processing unit 32 operates the user terminal 3 via the GUI control unit 31 and executes a registration process to register a person who wishes to register for the authentication service. In the registration process, the registration processing unit 32 first obtains various information (such as name) including biometric information of the person who wishes to register for the authentication service from the user terminal 3. Next, the registration processing unit 32 stores the obtained information as registration information in the storage unit 22. The registration information includes registered biometric information.

[0069] Furthermore, if the registration processing unit 32 acquires a face image captured by the imaging device 4 during the registration process, it may extract facial features and store them in the storage unit 22 as registered biometric information.

[0070] In the authentication process, the image acquisition unit 33 sends an imaging instruction signal to the user terminal 3, instructing it to transmit the image captured by the imaging device 4. The user terminal 3 transmits the image captured by the imaging device 4 to the image acquisition unit 33, and the image acquisition unit 33 receives the image captured by the imaging device 4 from the user terminal 3. The imaging device 4 is positioned to capture images of the authentication target, such as the user seeking authentication.

[0071] The images acquired by the image acquisition unit 33 are so-called moving images, consisting of one or more frames. Each frame corresponds to a single still image. However, in cases where, for example, pulse or heart rate is used for biometric information matching or liveness determination, or when actions such as blinking are used for bodily expression matching, a group of images or a moving image consisting of multiple still images may be used instead of individual frames.

[0072] In this embodiment, the image acquisition unit 33 sequentially acquires captured images from the user terminal 3 frame by frame and outputs them to the processing instruction unit 34. The image acquisition unit 33 should intermittently output frames to the processing instruction unit 34 within the shooting time.

[0073] The processing instruction unit 34 determines, for each frame acquired from the image acquisition unit 33, whether or not to execute the biometric information matching process, the bodily expression matching process, and the liveness determination process, based on the processing instruction information 42 stored in the storage unit 22.

[0074] In this embodiment, a processing instruction table 43, in which each frame and its corresponding processing instruction information 42 are recorded, is stored in the storage unit 22. Figure 4 shows an example of the processing instruction table 43 stored in the storage unit 22. The processing instruction information 42 indicates whether or not the biometric information matching process, the bodily expression matching process, and the liveness determination process should be executed for each frame. In the processing instruction table 43 shown in Figure 4, ○ is recorded for processes that need to be executed, and × is recorded for processes that do not need to be executed. Note that the timing setting information may include frames in which none of the biometric information matching process, the bodily expression matching process, or the liveness determination process should be executed. The processing instruction table 43 records the processing instruction information 42 corresponding to all frames output during the shooting time.

[0075] In the processing instruction table 43, the biometric information matching process, the bodily expression matching process, and the liveness determination process are each configured to be executed in at least one frame.

[0076] In the processing instruction table 43, it is preferable that the three processes—biometric information matching, bodily expression matching, and liveness determination—are not executed in a single frame. For example, it is preferable that the processes—biometric information matching, bodily expression matching, and liveness determination—are each executed in different frames.

[0077] The number of biometric information matching processes in the processing instruction table 43 should be set to be less than or equal to the number of body expression matching processes, and also less than or equal to the number of liveness determination processes.

[0078] Generally, the processing load of biometric information matching is greater than that of bodily representation matching, and also greater than that of liveness determination. Therefore, by setting the number of biometric information matching operations to be less than or equal to the number of bodily representation matching operations and less than or equal to the number of liveness determination operations, it is possible to prevent the processing load required for authentication from becoming too large.

[0079] When the processing instruction unit 34 determines that it is necessary to perform biometric information matching, it outputs the acquired frame to the biometric information matching processing unit and instructs it to perform biometric information matching. When the processing instruction unit 34 determines that it is necessary to perform body expression matching, it outputs the acquired frame to the body expression matching processing unit and instructs it to perform body expression matching. When the processing instruction unit 34 determines that it is necessary to perform liveness determination, it outputs the acquired frame to the liveness determination processing unit and instructs it to perform liveness determination.

[0080] When the biometric information matching unit 35 receives an instruction from the processing instruction unit 34 to execute the biometric information matching process, it detects the face to be authenticated from the frame acquired by the image acquisition unit 33 and performs face matching (an example of biometric information matching) by comparing it with the registered face information. The biometric information matching unit 35 outputs the matching result of the face matching to the authentication determination unit 38.

[0081] The biometric information matching unit 35 includes a face detection unit 44 and a face matching unit 45 for performing face matching. Note that if the biometric information matching unit 35 performs biometric information matching other than face matching, other configurations may be adopted.

[0082] The face detection unit 44 detects a person's face from the frame acquired by the image acquisition unit 33 and extracts the region of the detected face. This allows the face detection unit 44 to acquire the face image to be authenticated.

[0083] Furthermore, when the image acquisition unit 33 acquires video instead of frames (for example, when heart rate is used for liveness determination), the face detection unit 44 should acquire a still image from the video in which the face region is most clearly visible, and obtain the face image to be authenticated.

[0084] The face detection unit 44 outputs the detected face image to be authenticated to the face matching unit 45.

[0085] The face matching unit 45 compares the face image to be authenticated, acquired by the face detection unit 44, with the registered face information (i.e., performs face matching). Based on this, the face matching unit 45 determines whether the face image belongs to a registered person. The face matching unit 45 outputs the matching result to the authentication determination unit 38.

[0086] Furthermore, the face matching performed by the face matching unit 45 may be based on facial features. For example, the face matching unit 45 may obtain the facial features of the target by detecting facial feature points from the face image to be authenticated. The face matching unit 45 may then perform face matching by comparing the obtained facial features of the target with registered face information, which is the facial features of a registered person. In addition, the face matching performed by the face matching unit 45 may be a 1:N matching, or, for example, if the user terminal 3 is owned and used by a specific person, it may be a so-called 1:1 matching, which matches the facial features of that specific person.

[0087] When the body representation matching unit 36 ​​receives an instruction from the processing instruction unit 34 to execute the body representation matching process, it detects the body representation to be authenticated from the frame acquired by the image acquisition unit 33 and determines whether it matches the body representation (instructed representation) instructed to be authenticated (i.e., performs body representation matching). The biometric information matching unit 35 outputs the result of the body representation matching determination to the authentication determination unit 38.

[0088] The body representation matching unit 36 ​​may, for example, detect body representations by placing feature points on a face image, acquiring feature quantities, or by detecting the movement of those feature points. However, the feature points and feature quantities acquired by the body representation matching unit 36 ​​are for identifying body representations, and the position of the feature points and the calculation of the feature quantities may differ from those used to identify individuals.

[0089] If the instruction displayed on the user terminal 3 is to close the eyes, the body expression matching unit 36 ​​may detect that the person being authenticated has closed their eyes by setting feature points on the corners of the eyes and the upper and lower eyelids and determining whether the upper and lower eyelids are close together.

[0090] To determine that the instruction displayed on the user terminal 3 is an action to make a smile, the body expression matching unit 36 ​​may place feature points in positions where changes are likely to occur when making a facial expression (for example, both eyes, eyebrows, cheeks, forehead, corners of the mouth, etc.), and detect that the person being authenticated is making a smile based on the movement and position of these points.

[0091] If the instruction displayed on the user terminal 3 is to turn left or right, the body expression matching unit 36 ​​may place feature points on the nose and eyebrows, respectively, estimate the rotation angle of the face based on the relative positions of these feature points, and determine that the authentication target is turning left or right.

[0092] The body expression matching unit 36 ​​may also determine whether or not an instruction expression is being made by quantifying the detected posture of the person to be authenticated.

[0093] For example, if the instruction is to close the eyes, the body expression matching unit 36 ​​may quantify the distance between the feature points on the upper and lower eyelids, and if that distance is below a threshold, it may determine that the subject of authentication has closed its eyes and that it matches the instruction.

[0094] For example, if the instruction is to make a smile, the degree to which the corners of the mouth are raised and the degree to which the eyes are narrowed are quantified based on the position of feature points placed at the corners of the mouth and the distance between feature points on the upper and lower eyelids, and a score (hereinafter referred to as the smile score or smile index) is calculated. By comparing this smile score with a threshold, it may be possible to determine whether or not a smile is being made.

[0095] If the instruction is to turn left or right, the body expression matching unit 36 ​​may determine that the face is turning left or right when the estimated rotation angle of the face exceeds a predetermined threshold, and may determine that it matches the instruction.

[0096] Thus, since the body expression matching performed by the body expression matching unit 36 ​​is based on the body expression of the subject to authentication, it can be performed even with just one frame. Furthermore, if a group of images consisting of multiple still images or a video is used instead of individual frames, the body expression matching performed by the body expression matching unit 36 ​​can be performed based on that group of images or video, taking into account the movements and actions of the subject to authentication.

[0097] Furthermore, the body expression matching unit 36 ​​determines that a match has been made when it detects a body expression that matches the body expression (instruction expression) that was instructed to the authentication target. The body expression matching unit 36 ​​outputs the acquired determination result (also called the matching result) to the authentication determination unit 38.

[0098] When the liveness determination unit 37 receives an instruction to execute the liveness determination process from the processing instruction unit 34, it executes the liveness determination process using the frame acquired by the image acquisition unit 33 and outputs the determination result to the authentication determination unit 38.

[0099] Specifically, the liveness determination unit 37 detects a person's face from the frame acquired by the image acquisition unit 33. Then, the liveness determination unit 37 extracts a predetermined region containing the detected face from the frame and acquires reflection information, texture information, etc., in the extracted region. Subsequently, the liveness determination unit 37 uses a machine learning model to confirm that there are no images displayed on a tablet device or smartphone, etc., and if this is confirmed, it determines that the subject of authentication is a living person. The liveness determination unit 37 outputs the determination result obtained in this way to the authentication determination unit 38.

[0100] The authentication determination unit 38 determines whether or not to authenticate the target of authentication based on the processing results of the biometric information matching process, the bodily expression matching process, and the liveness determination process.

[0101] The authentication determination unit 38 may authenticate the authentication target if it can match the registered person in at least one biometric information matching process performed within the shooting time after the start of the authentication process, determines that an instruction expression was made in at least one bodily expression matching process, and determines that the authentication target is a living being in at least one liveness determination process.

[0102] In addition, the authentication determination unit 38 may authenticate the authentication target if, in the biometric information matching process, it is determined that the registered person has been matched with the biometric information a predetermined first number of times or more, in the bodily expression matching process, it is determined that the instruction expression has been made a predetermined second number of times or more, and in the liveness determination process, it is determined that the authentication target is a living being a predetermined third number of times or more.

[0103] When the result notification unit 39 obtains the authentication result from the authentication determination unit 38, it controls the GUI control unit 31 to display the authentication result on the user terminal 3.

[0104] The result notification unit 39 may send an error notification signal to the administrator terminal 5 to notify it of an authentication error if the authentication determination unit 38 determines that the authentication target should not be authenticated. The result notification unit 39 may also store the authentication result of the authentication determination unit 38, along with an identification number (identification information; hereinafter referred to as terminal ID) for identifying the user terminal 3 that transmitted the captured image, in the storage unit 22 as an authentication log.

[0105] The instruction information setting unit 40 executes a setting process to set the processing instruction information 42. Specifically, in the setting process, the instruction information setting unit 40 controls the screen display of the administrator terminal 5 via the GUI control unit 31, obtains the setting input entered on the administrator terminal 5, and sets the processing instruction information 42 corresponding to the setting information. The setting process is not executed while the authentication process is in progress, but is performed before or after the authentication process (i.e., before the next authentication process).

[0106] The instruction information setting unit 40 displays the setting screen 46 on the administrator terminal 5 during the setting process. Figure 5 shows one example of the setting screen 46 (hereinafter referred to as the first display example).

[0107] In the first example display, the settings screen 46 is provided with input fields 47 for each frame, indicating whether or not to perform biometric information matching processing, bodily expression matching processing, and liveness processing. Each input field 47 is composed of a checkbox 47A.

[0108] When the administrator enters information in the input field 47 and presses the registration button 48, setting information indicating whether or not to perform biometric information matching processing, bodily expression matching processing, and liveness processing for each frame is sent to the instruction information setting unit 40. When the instruction information setting unit 40 obtains the setting information from the administrator terminal 5, it sets the processing instruction information 42 to correspond to (match) the obtained setting information.

[0109] The instruction information setting unit 40 is preferably configured to determine whether the setting information (more specifically, the processing instruction information 42 corresponding to the setting information) satisfies the conditions required to ensure the required security level (hereinafter referred to as setting conditions). When the instruction information setting unit 40 determines that the setting conditions are not met, it is preferable to send warning information to the administrator terminal 5 and display a warning screen 49 on the administrator terminal 5.

[0110] For example, in the configuration information, the biometric information matching process, the bodily expression matching process, and the liveness determination process may each be configured not to be executed in at least one frame from the start to the end of authentication. In that case, the instruction information setting unit 40 should determine that the configuration conditions are not met. That is, the configuration conditions should be set so that the number of times the biometric information matching process is executed, the number of times the bodily expression matching process is executed, and the number of times the liveness determination process is executed from the start to the end of authentication are each 1 or more.

[0111] Figure 6 shows an example of a warning screen 49 displayed on the administrator terminal 5 when the configuration information does not meet the configuration conditions. In the example shown in Figure 6, a pop-up 50 is displayed prompting the administrator to set the frame in which to execute a process that is not executed in any of the frames from the start to the end of authentication.

[0112] In addition, the setting conditions may include setting the execution count of at least one of the biometric information matching process, bodily expression matching process, and liveness determination process to be above a predetermined threshold. This allows a warning screen 49 to be displayed on the administrator terminal 5 when the required security level is not ensured.

[0113] In addition, the settings may be configured such that the number of executions of the biometric information matching process is less than the number of executions of the bodily expression matching process, and the number of executions of the biometric information matching process is less than the number of executions of the liveness determination process. This prevents the number of executions of the biometric information matching process from exceeding either the number of executions of the bodily expression matching process or the number of executions of the liveness determination process, thereby preventing an increase in the processing load required for authentication.

[0114] Figure 7 shows another example of the settings screen 46 (hereinafter referred to as the second display example). In the example shown in Figure 7, the input fields 47 for the biometric information matching process, body expression matching process, and liveness determination process on the settings screen 46 are composed of pull-down menus 47B. In Figure 7, the pull-down menus 47B that appear when inputting into the input fields 47 for the biometric information matching process, body expression matching process, and liveness determination process are shown in order to illustrate the contents of each pull-down menu 47B.

[0115] In the second display example, the pull-down menu 47B corresponding to the biometric information matching process is provided with options (items) for setting the range of the face to be used for face matching. In the example shown in Figure 7, three options are shown for the range of the face to be used for matching: the entire face (indicated as "whole face"), the upper half of the face (indicated as "upper face"), and the lower half of the face (indicated as "lower face").

[0116] Figure 8 shows an example of the items in the pull-down menu 47B corresponding to the biometric information matching process, along with the corresponding matching method and matching environment, presented as a table. As shown in Figure 8, the upper half of the face (upper face) is used for face matching in the area above the tip of the nose, and is selected, for example, when the user terminal 3 is installed in an environment where a mask is required. The lower half of the face (lower face) is used for face matching in the area below the tip of the nose, and is selected, for example, when the user terminal 3 is installed in a manufacturing site where protective eyewear is required. In this way, the administrator can select and set the contents of the biometric information matching process and body expression matching process on the settings screen 46 according to the environment in which the matching will be performed.

[0117] In addition, biometric information matching processes may involve using the upper half of the face (also called the upper face) for facial matching in combination with fingerprint authentication and / or vein authentication, or using the lower half of the face (also called the lower face) for facial matching in combination with fingerprint authentication and / or vein authentication. Figure 8 shows examples of corresponding items, matching methods, and corresponding bodily representations.

[0118] The pull-down menu 47B, which corresponds to the body expression matching process, is provided with options (items) for setting the body expression to be indicated in the body expression matching. In the example shown in Figure 7, the options include turning the face left and right, turning the face up and down, blinking, and opening and closing the mouth. The GUI control unit 31 should set the instruction for the body expression to be displayed on the user terminal 3 according to the selection result of the options in this pull-down menu 47B.

[0119] Figure 9 shows an example of the items in the pull-down menu 47B corresponding to the body expression matching process, along with the corresponding expression instructions and corresponding judgment methods, all summarized in a table.

[0120] Note that in Figure 1, for the sake of explanation, only one authentication server 2 is shown. However, in the authentication system 1, at least some of the functions of the authentication server 2 described above may be realized by the cooperation of multiple computers.

[0121] User terminal 3 includes information devices (also called electronic devices) with communication functions, such as smartphones, tablet devices, or notebook PCs, carried by the user. However, information terminals installed in stores or public facilities (for example, kiosk terminals) may also be used as user terminal 3. In addition, user terminal 3 may be control devices such as those for opening and closing doors and gates, or equipment for manufacturing products installed in factories, or equipment for inspecting products.

[0122] The user terminal 3 performs tasks such as displaying information about the authentication target, capturing images of the authentication target, and receiving input from the authentication target, based on instructions from the authentication server 2. As shown in Figure 10, the user terminal 3 includes, in addition to the imaging device 4, a processor 52, a storage device 53, a communication device 54, an input device 55, and a display device 56. The processor 52 is composed of a CPU or MPU, and the storage device 53 may be composed of memory 57 such as RAM or ROM and storage 58 such as HDD or SSD. The communication device 54 includes a network interface equipped with an antenna and communication circuits.

[0123] The input device 55 is comprised of known hardware such as a touch panel display or input buttons that allow user input. The display device 56 may be comprised of known hardware such as a liquid crystal display. The touch panel display used as the input device 55 may also function as the display device 56.

[0124] As shown in Figure 11, the user terminal 3 has a communication unit 61, an imaging unit 62, an operation unit 63, a display unit 64, a control unit 65, and a storage unit 66.

[0125] The communication unit 61 is equipped with a communication device 54 and has the same configuration as the communication unit 21 described above. The communication unit 61 communicates wirelessly or via wired connection with other devices (in this case, the authentication server 2, etc.) via the communication network 6 according to a known communication protocol.

[0126] The operation unit 63 includes an input device 55 and accepts input operations from the authentication target. The display unit 64 is equipped with a display device 56 and displays information on the screen for the user.

[0127] The imaging unit 62 includes the imaging device 4 and acquires the image to be authenticated.

[0128] In this embodiment, the imaging device 4 is configured with a camera that acquires an image by detecting visible light. The imaging device 4 may also be configured with an infrared camera that acquires an image by detecting infrared light.

[0129] The imaging device 4 is positioned to capture images of the operator operating the control unit 63. The imaging device 4 is preferably positioned near the upper center of the display unit 64 to capture the user's face during operation. The imaging unit 62 can acquire (generate) an image including the face of the operator operating the user terminal 3 by capturing images from the imaging device 4.

[0130] The storage unit 66 stores data and information necessary for processing the user terminal 3. The storage unit 66 also stores an authentication application program, which is one of the control programs for the user terminal 3 to execute processing related to the facial recognition service. The authentication application program is pre-installed on the user terminal 3.

[0131] The control unit 65 includes one or more processors 52. By executing authentication application programs stored in the storage unit 66, the control unit 65 can perform processing related to authentication services used by the user (for example, processing required when registering the user's biometric information (facial information) and processing for authentication). In addition, the control unit 65 can comprehensively control the operation of the user terminal 3.

[0132] Administrator terminal 5 is a computer used by the administrator who manages the authentication service to configure and manage the authentication process necessary to provide the authentication service. Administrator terminal 5 includes information devices with communication capabilities, such as desktop PCs, notebook PCs, smartphones, and tablet devices.

[0133] As shown in Figure 12, the administrator terminal 5 comprises a processor 71, a storage device 72, a communication device 73, an input device 74, and a display device 75. The processor 71 is composed of a CPU or MPU, and the storage device 72 may consist of memory 76 such as RAM or ROM and storage 77 such as HDD or SSD. The communication device 73 includes a network interface equipped with an antenna and communication circuits. The input device 74 may consist of a mouse, keyboard, touch panel display, etc. The display device 75 comprises known hardware such as a liquid crystal display. A touch panel display used as the input device 74 may also function as the display device 75.

[0134] As shown in Figure 13, the administrator terminal 5 includes a communication unit 79, an operation unit 80, a display unit 81, a storage unit 82, and a control unit 83.

[0135] The communication unit 79 includes a communication device 73 and has the same configuration as the communication unit 21 described above. The communication unit 79 communicates wirelessly or via wired connection with other devices (in this case, the authentication server 2) via the communication network 6 according to a known communication protocol.

[0136] The operation unit 80 is equipped with an input device 74 to receive operation input from the administrator. The display unit 81 is equipped with a display device 75 to display information on the screen for the administrator.

[0137] The memory unit 82 is composed of a storage device 72 including memory 76 and storage 77, and stores data and information necessary for processing the administrator terminal 5.

[0138] The control unit 83 includes one or more processors 71. Based on instructions from the instruction information setting unit 40, the control unit 83 (processor 71) works in cooperation with the display unit 81 and the operation unit 80 to receive setting inputs for processing instruction information 42 from the administrator. The control unit 83 (processor 71) transmits the received setting inputs to the authentication server 2 via the communication unit 79. When the authentication server 2 (specifically the instruction information setting unit 40) receives the setting inputs for processing instruction information 42 from the control unit 83, it sets the processing instruction information 42 based on the received setting inputs.

[0139] In addition, when the control unit 83 receives an error notification signal, it causes the display unit 81 to display an error display screen 84. Figure 14 shows an example of the error display screen 84 displayed on the display unit 81 of the administrator terminal 5.

[0140] On the error display screen 84, a list of user terminals 3 on which the authentication application program is installed is displayed by icons 85. On the error display screen 84, the icon 85 corresponding to the user terminal 3 that sent the captured image that was determined to be unauthenticated should be displayed in color (colored by dots in Figure 14; for example, red).

[0141] The error display screen 84 shown in Figure 14 displays a pop-up 86 that provides information to identify the process in which the error occurred. For example, in the biometric information matching process, if face matching fails, the pop-up 86 should display a message to that effect (such as "Face matching error detected"). Similarly, in the body expression matching process, if a body expression corresponding to an instruction expression is not detected, the pop-up 86 should also display a message to that effect (such as "Body expression matching error detected"). Furthermore, in the liveness determination process, if the authentication target is determined not to be a living organism, the pop-up 86 should also display a message to that effect (such as "Liveness determination error detected").

[0142] Furthermore, the pop-up 86 may also display the device name of the user terminal 3 where the error was detected, along with the identification number (terminal ID) of the user terminal 3. Alternatively, the device name and terminal ID may be displayed along with the identification number (user ID) of the user who owns the user terminal 3 where the error was detected.

[0143] The popup 86 may also be provided with a confirmation button 87 for displaying the authentication log. When the administrator clicks (or touches) the confirmation button 87, the control unit 83 retrieves the authentication log from the authentication server 2 and displays a history display screen 88, including the retrieved authentication log, on the display unit 81. Figure 15 shows an example of the history display screen 88 displayed on the display unit 81 of the administrator terminal 5.

[0144] The history display screen 88 shown in Figure 15 displays a table 89 showing the date and time when authentication was initiated, as well as the authentication result. In addition, the table 89 shown in Figure 15 also includes information about the authenticated user (in Figure 15, the user's employment category) and the terminal ID of the terminal on which authentication was performed.

[0145] In the history display screen 88 shown in Figure 15, when the administrator selects a row (corresponding to one authentication process) from table 89 and double-clicks, a pop-up 90 showing the details of the authentication process results is displayed. Figure 16 shows an example of the pop-up 90 showing the details of the authentication process results.

[0146] As shown in Figure 16, the popup 90 may display the device name of the user terminal 3 on which the authentication process was performed, the terminal ID, and information to identify the process in which the error occurred. In addition, the popup 90, which shows details of the authentication process results, may also include the user ID of the user who owns the user terminal 3.

[0147] Next, the setting process for the processing instruction information 42, which is executed in the authentication system 1 according to the first embodiment, will be described.

[0148] Figure 17(A) is a sequence diagram showing the operation of the authentication server 2 and administrator terminal 5 when setting information that satisfies the setting conditions is entered during the setting process of processing instruction information 42. Figure 17(B) is a sequence diagram showing the operation of the authentication server 2 and administrator terminal 5 when setting information that satisfies the setting conditions is entered once during the setting process of processing instruction information 42. Figure 18 is a flowchart showing the flow of the setting process of processing instruction information 42 on the authentication server 2.

[0149] In Figures 17(A), 17(B), and 18, the same code (ST101, ST104-ST107) is assigned to the corresponding processes, with the exception of steps ST102, ST103, and ST108.

[0150] The setting process for the processing instruction information 42 is performed when the administrator makes a predetermined input to the administrator terminal 5, such as when the authentication system 1 is installed, and a setting process start command is sent to the authentication server 2.

[0151] As shown in Figure 18, in the setting process of the processing instruction information 42, the authentication server 2 first sends an instruction to the administrator terminal 5 to display the setting screen 46 (ST101). When the administrator terminal 5 receives the instruction to display the setting screen 46, it displays the setting screen 46 (see Figure 5) on the display unit 81 and accepts input of setting information from the administrator (ST102). When the administrator terminal 5 accepts the input of setting information, it sends the entered setting information to the authentication server 2 (ST103).

[0152] The authentication server 2 (specifically, its processor 11) receives configuration information from the administrator terminal 5 (ST104) and generates processing instruction information 42 corresponding to the acquired configuration information. Subsequently, the authentication server 2 (specifically, its processor 11) determines whether the generated processing instruction information 42 satisfies the configuration conditions (ST105).

[0153] If the processing instruction information 42 satisfies the setting conditions (Yes in ST105 in Figure 18), the authentication server 2 sets the processing instruction information 42 by recording the generated processing instruction information 42 in the processing instruction table 43 (see also ST106 and Figure 17(A)). Once the setting is complete, the authentication server 2 may display a message on the display unit 81 of the administrator terminal 5 indicating that the setting is complete.

[0154] If the processing instruction information 42 does not meet the setting conditions (No in ST105 in Figure 18), the authentication server 2 instructs the administrator terminal 5 to display the warning screen 49 (ST108), causing the administrator terminal 5 to display the warning screen 49 (see Figure 6). When the specified input is received on the warning screen 49, the administrator terminal 5 displays the setting screen 46 (see Figure 5) again to accept input of setting information (ST102).

[0155] Next, we will describe the details of the authentication process performed by the authentication system 1 according to the first embodiment.

[0156] Figure 19 is a sequence diagram showing the operation of authentication server 2 and user terminal 3 during the authentication process, and Figure 20 is a flowchart showing the flow of the authentication process on authentication server 2.

[0157] In Figures 19 and 20, with the exception of steps ST111A, ST111B, ST114, and ST117, the corresponding processes are assigned the same code (ST111, ST112, ST113, ST115, ST116, ST118).

[0158] When the user terminal 3 performs a prescribed procedure to request authentication (for example, launching an authentication application program), the user terminal 3 sends a start request signal requesting the start of the authentication process. Upon receiving this start request signal, the authentication server 2 starts the authentication process shown in Figure 17. The start request signal includes a terminal ID (identification number) to identify the user terminal 3 as the source. After sending the start request signal, the user terminal 3 should start imaging with the imaging device 4.

[0159] As shown in Figures 19 and 20, during the authentication process, the processor 11 of the authentication server 2 acquires one frame of captured images from the user terminal 3 (ST111). When acquiring a frame, as shown in Figure 19, the processor 11 of the authentication server 2 may be configured to send a frame request to the user terminal 3 (ST111A), and the user terminal 3, upon receiving the frame request, may send one frame of captured images (ST111B).

[0160] Once frame acquisition is complete, the processor 11 of the authentication server 2 refers to the processing instruction table 43 stored in the storage device 12 (storage unit 22) and obtains processing instruction information 42 corresponding to the acquired frame (ST112). The processing instruction information 42 records whether or not biometric information matching processing, body expression matching processing, and liveness determination processing should be performed for the acquired frame (see Figure 4).

[0161] Once the acquisition of the corresponding processing instruction information 42 is complete, the processor 11 of the authentication server 2 executes the processing that is recorded as necessary based on the necessity indicated in the processing instruction information 42, and stores the processing result in the storage device 12 (ST113).

[0162] Once the execution of the process recorded as necessary in the processing instruction information 42 is completed, the processor 11 of the authentication server 2 determines whether the acquired frame corresponds to the last frame of the shooting time (hereinafter referred to as the final frame) (ST114), as shown in Figure 20.

[0163] If the frame does not correspond to the final frame (No in ST114), the processor 11 of the authentication server 2 will again acquire the frame, acquire the corresponding processing instruction information 42, and execute the processing recorded in the processing instruction information 42 in order. In this way, the processor 11 of the authentication server 2 will sequentially acquire the frame, acquire the corresponding processing instruction information 42, and execute the processing recorded in the processing instruction information 42 until the final frame (see Figure 19).

[0164] If it is the final frame (Yes in ST114), the processor 11 of the authentication server 2 determines whether or not to authenticate based on the processing results of all frames within the shooting time (ST115). In addition, if it is the final frame, the processor 11 of the authentication server 2 may send an instruction signal to the user terminal 3, causing the authentication server 2 to display a message indicating that it is making a decision on whether or not to authenticate.

[0165] In this case, the processor 11 of the authentication server 2 may be configured to determine that authentication should be performed when each of the biometric information matching process, body expression matching process, and liveness determination process is successful in any frame within the shooting time. That is, the processor 11 of the authentication server 2 may be configured to determine that authentication should be performed when matching (face matching) is possible in at least one biometric information matching process within the shooting time, matching is possible in at least one body expression matching process within the shooting time, and the authentication target is determined to be a living being in at least one liveness determination process within the shooting time.

[0166] Once the determination of whether or not authentication should be performed is complete, the processor 11 of the authentication server 2 notifies the user terminal 3 of the result (ST116), and displays the authentication result on the user terminal 3 (ST117, see Figure 19). In addition, the processor 11 of the authentication server 2 also records the terminal ID of the user terminal 3 and the authentication result in the authentication log of the storage device 12 (storage unit 22) (ST118), and finishes the authentication process.

[0167] Next, the operation and effects of authentication system 1 will be explained.

[0168] Figure 21 shows the relationship between the physical expressions made by the person being authenticated and the timing of the processing during the authentication process. In Figure 21, the upper section shows the user making instruction expressions after the authentication process has started, and the lower section shows the processing performed in each frame based on the processing instruction information 42 shown in Figure 4.

[0169] If three processes—biometric information matching, physical appearance matching, and liveness determination—are possible, a highly secure system can be constructed by continuously performing all three processes and determining whether authentication is successful or not. On the other hand, continuously executing all three processes increases the processing load.

[0170] In authentication system 1, the administrator can set processing instruction information 42 indicating whether or not to perform biometric information verification, bodily expression verification, and liveness determination processing. Therefore, by inputting the setting information, the administrator can set the number of times and frames for each of the biometric information verification, bodily expression verification, and liveness determination processing to be executed.

[0171] Therefore, as shown in Figure 21, by configuring the system to not execute any of the following processes—biometric information matching, bodily expression matching, and liveness determination—the administrator can reduce the processing load while ensuring security.

[0172] Furthermore, administrators should set which processes to be prevented from being executed and when, based on the content of the instruction, they should do so. For example, biometric information matching may be set to be performed before the instruction is completed. Also, bodily expression matching may be set to be performed when the response to the expression instruction is completed (for example, in the latter half of the shooting time).

[0173] (Second Embodiment) Next, the authentication system 1 according to the second embodiment of this disclosure will be described. With respect to the authentication system 1 according to the second embodiment, components similar to those in the first embodiment will be denoted by the same reference numerals and detailed descriptions will be omitted. Furthermore, with respect to the authentication system 1 according to the second embodiment, matters not specifically mentioned below will be the same as in the first embodiment.

[0174] The authentication system 1 according to the second embodiment differs in that the authentication server 2 does not set the processing instruction information 42 based on the configuration information entered at the administrator terminal 5, but rather sets it based on information indicating the status of the authentication target (hereinafter referred to as status information) obtained from the user terminal 3.

[0175] Furthermore, the authentication system 1 according to the second embodiment differs from the first embodiment in that, after the authentication server 2 receives a start request signal from the user terminal 3 requesting the start of the authentication process, the authentication server 2 performs the setting process for processing instruction information 42, and then performs the authentication process.

[0176] Furthermore, since the setting process for processing instruction information 42 is always performed before the authentication process, the setting process and the authentication process together can be referred to as the authentication process, in which case the setting process will be included in the authentication process.

[0177] The following describes in detail the processing performed by the authentication server 2 according to the second embodiment after it receives a start request signal, with reference to the drawings.

[0178] Figure 22 is a sequence diagram showing the operation of the authentication server 2 and the user terminal 3 after the authentication server 2 receives a start request signal in the second embodiment, and Figure 23 is a flowchart showing the flow of processing performed by the authentication server 2 after it receives a start request signal.

[0179] As shown in Figures 22 and 23, when the authentication server 2 receives a start request signal, it executes the setting process for the processing instruction information 42 (ST201). As shown in Figures 22 and 23, the processing performed by the authentication server 2 and the user terminal 3 after the setting process for the processing instruction information 42 (ST201) is the same as the authentication process in the first embodiment. Therefore, the processing after the setting process for the processing instruction information 42 (ST201) will not be explained.

[0180] As shown in Figure 22, in the setting process of the processing instruction information 42 (ST201), the authentication server 2 requests the user terminal 3 to send information indicating the status of the device to be authenticated (hereinafter referred to as status information) (ST201A). In response to this request from the authentication server 2, the user terminal 3 sends the status information to the authentication server 2 (ST201B). Based on the acquired status information, the authentication server 2 sets the processing instruction information 42 (ST201C).

[0181] The status information may include, for example, an image of the object to be authenticated acquired by the imaging device 4. Based on the acquired image, the authentication server 2 can detect the object to be authenticated (e.g., a mask or sunglasses). Based on the content of the detected object, the authentication server 2 can select the facial range and instruction expression to be used for face matching (biometric information matching), and set the processing instruction information 42.

[0182] Figure 24 shows an example of setting processing instruction information 42 based on an captured image. In the example shown in Figure 24, multiple processing instruction tables 43 (the first to third processing instruction tables 43A to 43C in the example shown in Figure 24) are stored in the storage device 12 (storage unit 22) in advance to correspond to the state of the authentication target.

[0183] In the example shown in Figure 24, if neither a mask nor sunglasses are detected as the wearable items to be authenticated, the first processing instruction table 43A is used to set the processing instruction information 42. In this case, the entire face is selected as the area to be used for biometric information matching, and opening the mouth is selected as the instruction expression to be matched in the body expression matching process. Furthermore, it is set that the biometric information matching process must be executed in one frame (frame 2).

[0184] Furthermore, if a mask is detected as the item to be authenticated and sunglasses are not detected, the second processing instruction table 43B is used to set the processing instruction information 42. In this case, the upper half of the face (upper face) is selected as the area to be used for biometric information matching, and looking downwards is selected as the instruction expression to be matched in the body expression matching process. In addition, the biometric information matching process is set to be performed in two frames (frames 1 and 2).

[0185] Furthermore, if a mask is not detected as the item to be authenticated, but sunglasses are detected, the third processing instruction table 43C is used to set the processing instruction information 42. In this case, the lower half of the face (lower face) is selected as the area to be used for biometric information matching, and turning to the left is selected as the instruction expression to be matched in the body expression matching process. In addition, the biometric information matching process is set to be performed in three frames (frames 1 to 3).

[0186] Furthermore, if both a mask and sunglasses are detected as items to be authenticated, the authentication server 2 should display a notification to the user terminal 3 indicating that authentication is not possible.

[0187] Thus, according to the authentication system 1 of the second embodiment, processing instruction information 42 is set based on state information indicating the state of the authentication target, such as an image captured by the imaging device 4. Therefore, appropriate processing instruction information 42 can be set according to the state of the authentication target.

[0188] Furthermore, the authentication server 2 can be configured to determine the imaging state based on the status information and autonomously increase or decrease the number of biometric information matching processes. In this case, compared to the first embodiment, the effort required for administrators to input settings can be reduced, thereby lowering the administrator's workload. The authentication server 2 may also set the processing instruction information 42 based on the surrounding environment of the authentication target (for example, ambient illumination) estimated from the captured image.

[0189] Furthermore, in the above description, the authentication server 2 set the processing instruction information 42 based on the device to be authenticated. However, the system is not limited to this configuration. For example, the authentication server 2 (processor 11) may set the processing instruction information 42 based on at least one of the date and time and day of the week on which authentication is performed. For example, the processor 11 of the authentication server 2 may select one of several processing instruction tables 43 based on the day of the week on which authentication is performed, and set the processing instruction information 42 based on the selected processing instruction table 43. This makes it possible to set the processing instruction information 42 based on at least one of the date and time and day of the week, and to automatically change the processing instruction information 42 in the authentication server 2.

[0190] Furthermore, the processor 11 of the authentication server 2 may be equipped with an imaging device 4 for imaging the object to be authenticated, and may set processing instruction information 42 based on the terminal ID (identification information) of the user terminal 3 operated by the object to be authenticated. This allows the processor 11 to set processing instruction information 42 for each user terminal 3. Thus, the processor 11 can set processing instruction information 42 according to the attributes of the user who owns the user terminal 3 (department or company to which the user belongs, employment category, job description, etc.). When authentication starts, the authentication server 2 obtains a start request signal, so it is advisable to identify the source of the start request signal and obtain the terminal ID included in the start request signal to identify the terminal ID of the user terminal 3 to be authenticated.

[0191] In that case, the administrator must set up a processing instruction table 43 containing processing instruction information 42 for each user terminal 3 (or user). However, the administrator may group the user terminals 3 and set up a processing instruction table 43 for each group.

[0192] Figure 25 shows an example of a group settings screen 92 for grouping user terminals 3. The group settings screen 92 shown in Figure 25 is displayed on the administrator terminal 5 when the administrator groups user terminals 3. The administrator can group user terminals 3 by selecting the user terminals 3 to belong to each group on the group settings screen 92 shown in Figure 25 and clicking the register button 93. As shown in Figure 25, it is preferable that the name of the user who owns (uses) user terminal 3 is displayed on the group settings screen 92 instead of the terminal ID of user terminal 3.

[0193] Figure 26 shows an example of a processing instruction setting screen 94 for inputting the association between each group and the processing instruction table 43. The processing instruction setting screen 94 shown in Figure 26 is displayed on the administrator terminal 5 when the administrator inputs the association between each group and the processing instruction table 43. On the processing instruction setting screen 94 shown in Figure 26, the administrator can set the corresponding processing instruction table 43 for each group by selecting the processing instruction table 43 for each group (labeled as setting 1, 2, etc. in Figure 26) and clicking the register button 95.

[0194] Furthermore, user terminals 3 may be grouped based on attributes such as the affiliation, employment category, and job description of the user who possesses (owns) user terminal 3. This makes it possible to set processing instruction information 42 according to the attributes of the user to be authenticated.

[0195] As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments.

[0196] Some of the functions of the authentication server 2 may be provided on the user terminal 3 or the administrator terminal 5. For example, as shown in Figure 27, in the first and second embodiments, the user terminal 3 may be provided with an image acquisition unit 33, and the user terminal 3 may transmit images to the authentication server 2 frame by frame. In addition, some of the registration processing unit 32 may be provided on the user terminal 3, and the extraction of facial features during registration may be performed on the user terminal 3.

[0197] Alternatively, the user terminal 3 may perform at least some of the functions of the authentication server 2 and the administrator terminal 5, and may function, for example, as an authentication device (referred to as an authentication equipment).

[0198] Furthermore, although the above description illustrates an example where one expression instruction is given for each authentication, this is merely an example. In the first and second embodiments, two or more expression instructions may be given for each authentication, and bodily expression matching may be performed in accordance with each expression instruction.

[0199] In the above description, an example was given in which the biometric information matching process, the bodily expression matching process, and the liveness determination process are each performed on the authentication server 2. However, in the first and second embodiments, some of the processes of the biometric information matching process, the bodily expression matching process, and the liveness determination process (for example, all of the liveness determination process) may be performed on the user terminal 3.

[0200] In the above embodiment, the biometric information matching process, the bodily expression matching process, and the liveness determination process may be configured such that the matching result or determination result is obtained only after they have been executed for multiple frames, respectively. For example, if the processing instruction table 43 sets the bodily expression matching process for the 4th and 6th frames, the processor 11 may be configured to perform the bodily expression matching process in the 6th frame using the captured image from the 4th frame and the captured image from the 6th frame, and to obtain the matching result.

[0201] Figure 21 shows an example where the biometric information matching process is set to be performed before the instruction expression is completed. However, if the biometric information matching process can be performed after the instruction expression is completed, it may be set to be performed after the instruction expression is completed. For example, if the instruction expression is the action of "closing the eyes" and the biometric information matching process is face matching, it is possible to perform the biometric information matching process after the instruction expression is completed.

[0202] Figure 28 shows a modified version of Figure 21, illustrating the relationship between the body expression performed by the authentication target and the timing of the processing when the instruction is "close your eyes" and the biometric information matching process is set after the instruction is completed. In this case, the instruction is set to "close your eyes," and the biometric information matching is performed by matching it with a registered face image (a face image with eyes closed).

[0203] Furthermore, if the instruction is to "close your eyes," the subject to authentication can perform the instructed action immediately after the instruction. Therefore, unlike in Figure 21, it is possible to set the body expression matching process in the first half of the recording time.

[0204] Furthermore, the instruction "close your eyes" is a more natural action than instructions that require facial movement, such as "turn your head up, down, left, or right," making it suitable for authentication in places like cafes where there are other people around. [Industrial applicability]

[0205] The authentication server, authentication system, authentication device, authentication method, and authentication program related to this disclosure are useful as authentication servers, authentication systems, authentication devices, authentication methods, and authentication programs that can reduce processing load while ensuring security. [Explanation of Symbols]

[0206] 1: Authentication System 2: Authentication Server 3: User terminal (an example of an authentication terminal or authentication device) 4: Imaging device 5: Administrator terminal 6: Communication Networks 11: Processor 12:Storage device 13: Communication device 14: Memory 15: Storage 21: Communications Department 22: Storage section 23: Control Unit 31: GUI Control Unit 32: Registration Processing Unit 33: Image acquisition unit 34: Processing instruction unit 35: Biometric Information Verification Unit 36: Body expression matching unit 37: Liveness determination unit 38: Authentication Determination Unit 39:Result notification section 40: Instruction Information Setting Unit 42: Processing instruction information 43: Processing instruction table 43A: First processing instruction table 43B: Second processing instruction table 43C: Third Processing Instruction Table 44: Face detection unit 45: Face Recognition Unit 46: Settings screen 47: Input field 47A: Checkbox 47B: Pull-down menu 48: Register button 49: Warning screen 50: Pop-up 52: Processor 53: Storage device 54: Communication equipment 55: Input device 56:Display device 57: Memory 58: Storage 61: Communications Department 62: Imaging Unit 63:Operation section 64:Display section 65: Control Unit 66: Storage section 71: Processor 72: Storage device 73: Communication equipment 74: Input device 75:Display device 76: Memory 77: Storage 79: Communications Department 80: Operation section 81:Display section 82: Storage section 83: Control Unit 84: Error display screen 85: Icon 86: Pop-up 87: Confirmation button 88: History display screen 89: Table 90: Pop-up 92: Group settings screen 93: Register button 94: Processing Instruction Settings Screen 95: Register button

Claims

1. It is an authentication server, The system includes a processor that acquires captured images, including the object to be authenticated, captured by an imaging device, and performs authentication-related processing. The aforementioned processor, Based on predetermined processing instruction information, for each frame constituting the captured image, A biometric information matching process that acquires the biometric information of the person to be authenticated, compares the acquired biometric information with pre-registered registered biometric information, and determines whether or not the person to be authenticated is a registered person. A body representation matching process that obtains the body representation to be authenticated and determines whether the obtained body representation satisfies predetermined conditions, The system selectively performs a liveness determination process to confirm whether the subject to be authenticated is a living organism, and authenticates the subject to be authenticated based on the results of these processes. An authentication server wherein the processing instruction information indicates whether or not the biometric information matching process, the bodily expression matching process, and the liveness determination process should be performed for each of the frames, and each process is configured to be performed in at least one of the frames.

2. It further includes a communication device for communicating with an administrator terminal used by the administrator, The authentication server according to claim 1, wherein the processor obtains configuration information entered at the administrator terminal via the communication device and sets the processing instruction information corresponding to the configuration information.

3. The authentication server according to claim 2, wherein, in the processing instruction information corresponding to the setting information, when the number of executions of at least one of the biometric information matching process, the bodily expression matching process, and the liveness determination process is less than a predetermined threshold, the processor transmits warning information to the administrator terminal via the communication device.

4. The authentication server according to claim 2, wherein, in the processing instruction information corresponding to the setting information, if the number of executions of the biometric information matching process is greater than the number of executions of the bodily expression matching process or the number of executions of the liveness determination process, the processor transmits warning information to the administrator terminal via the communication device.

5. The authentication server according to claim 1, wherein the processor sets the processing instruction information based on at least one of the date and time of authentication and the day of the week of authentication.

6. The authentication server according to claim 1, wherein the processor sets the processing instruction information based on the captured image acquired by the imaging device.

7. The device further includes a communication device for communicating with the authentication terminal equipped with the aforementioned imaging device. The authentication server according to claim 1, wherein the processor sets the processing instruction information based on the identification information of the authentication terminal obtained via the communication device.

8. It is an authentication system, An authentication terminal equipped with an imaging device, which acquires an image captured by the imaging device that includes the object to be authenticated, The system includes an authentication server that acquires the captured image from the authentication terminal and performs authentication-related processing, The aforementioned authentication server, Based on predetermined processing instruction information, for each frame constituting the captured image, A biometric information matching process that acquires the biometric information of the person to be authenticated, compares the acquired biometric information with pre-registered registered biometric information, and determines whether or not the person to be authenticated is a registered person. A body representation matching process that obtains the body representation to be authenticated and determines whether the obtained body representation satisfies predetermined conditions, The system selectively performs a liveness determination process to confirm whether the subject to be authenticated is a living organism, and authenticates the subject to be authenticated based on the results of these processes. An authentication system in which the processing instruction information indicates whether or not the biometric information matching process, the bodily expression matching process, and the liveness determination process should be performed for each of the frames, and each of the processes is configured to be performed in at least one of the frames.

9. Authentication device, The system includes a processor that acquires captured images, including the object to be authenticated, captured by an imaging device, and performs authentication-related processing. The aforementioned processor, Based on predetermined processing instruction information, for each frame constituting the captured image, A biometric information matching process that acquires the biometric information of the person to be authenticated, compares the acquired biometric information with pre-registered registered biometric information, and determines whether or not the person to be authenticated is a registered person. A body representation matching process that obtains the body representation to be authenticated and determines whether the obtained body representation satisfies predetermined conditions, The system selectively performs a liveness determination process to confirm whether the subject to be authenticated is a living organism, and authenticates the subject to be authenticated based on the results of these processes. Authentication device wherein the processing instruction information indicates whether or not the biometric information matching process, the bodily expression matching process, and the liveness determination process should be performed for each of the frames, and each process is configured to be performed in at least one of the frames.

10. Authentication method, The image captured by the imaging device, including the object to be authenticated, is acquired and processed by a processor that performs authentication-related processing. The aforementioned processor, Based on predetermined processing instruction information, for each frame constituting the captured image, A biometric information matching process that acquires the biometric information of the person to be authenticated, compares the acquired biometric information with pre-registered registered biometric information, and determines whether or not the person to be authenticated is a registered person. A body representation matching process that obtains the body representation to be authenticated and determines whether the obtained body representation satisfies predetermined conditions, The system selectively performs a liveness determination process to confirm whether the subject to be authenticated is a living organism, and authenticates the subject to be authenticated based on the results of these processes. Authentication method wherein the processing instruction information indicates whether or not the biometric information matching process, the bodily expression matching process, and the liveness determination process should be performed for each of the frames, and each process is configured to be performed in at least one of the frames.

11. An authentication program that acquires an image containing the object to be authenticated, captured by an imaging device, and causes a computer to perform information processing related to authentication, The aforementioned information processing includes, Based on predetermined processing instruction information, for each frame constituting the captured image, A biometric information matching process that acquires the biometric information of the person to be authenticated, compares the acquired biometric information with pre-registered registered biometric information, and determines whether or not the person to be authenticated is a registered person. A body representation matching process that obtains the body representation to be authenticated and determines whether the obtained body representation satisfies predetermined conditions, The process includes a liveness determination process to confirm whether the subject to be authenticated is a living organism, and a process to selectively execute this process, and to authenticate the subject to be authenticated based on the results of these processes. An authentication program in which the processing instruction information indicates whether or not the biometric information matching process, the bodily expression matching process, and the liveness determination process should be performed for each of the frames, and each of the processes is configured to be performed in at least one of the frames.

Images