User authentication apparatus and operating method thereof

Abstract

According to a first aspect of the present disclosure, there is provided a user authentication apparatus comprising: a face recognition unit configured to receive a visual input and extract a biometric marker from the visual input, the biometric marker being a face; a user identification unit configured to identify a user based on the biometric marker extracted by the face recognition unit; and an activity detection unit configured to detect whether the user identified by the user identification unit is a living person by extracting one or more additional biometric markers from the visual input received by the face recognition unit. According to other aspects of the present disclosure, a corresponding method of operation is conceived and a computer program for performing the method is provided.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a user authentication apparatus. Furthermore, the present disclosure relates to a corresponding method of operating a user authentication apparatus, and a computer program for executing said method. BACKGROUND

[0002] There are multiple ways to identify and authenticate a person. One of the ways is to use biometrics, in which case physical characteristics of a person are used for identification and authentication of said person. Commonly used physical characteristics are fingerprints, iris patterns, and faces. However, in case of using face recognition, security risks arise. SUMMARY

[0003] According to a first aspect of the present disclosure, there is provided a user authentication apparatus comprising: a face recognition unit configured to receive a visual input and to extract a biometric marker from the visual input, the biometric marker being a face; a user identification unit configured to identify a user based on the biometric marker extracted by the face recognition unit; a liveliness detection unit configured to detect whether the user identified by the user identification unit is a live person by extracting one or more additional biometric markers from the visual input received by the face recognition unit.

[0004] In one or more embodiments, the additional biometric markers comprise a heart rate.

[0005] In one or more embodiments, the liveliness detection unit is configured to extract the heart rate from the visual input by applying color amplification and / or motion amplification to the visual input.

[0006] In one or more embodiments, the additional biometric markers comprise facial characteristics and / or predefined movement patterns of the head of the user using multiple cameras or multiple views analysis.

[0007] In one or more embodiments, the additional biometric markers comprise a reconstructed three-dimensional model of the face obtained by illuminating the face from different angles with respect to a reference point on the face and by analyzing the effects of the illumination.

[0008] In one or more embodiments, the additional biometric markers comprise predefined behavior of the pupils after the face is illuminated.

[0009] In one or more embodiments, the additional biometric markers comprise one or more body movements performed in response to a challenge provided by the liveliness detection unit.

[0010] In one or more embodiments, the liveliness detection unit comprises a security element configured to provide the challenge.

[0011] In one or more embodiments, the secure element is further configured to verify whether the body movement performed in response to the challenge corresponds to an expected response.

[0012] In one or more embodiments, the body movement performed in response to the challenge and the expected response is encoded as a checksum.

[0013] According to a second aspect of the disclosure, a method of operating a user authentication device is conceived, comprising: receiving, by a face recognition unit comprised in the user authentication device, a visual input and extracting a biometric marker from the visual input, the biometric marker being a face; identifying, by a user identification unit comprised in the user authentication device, a user based on the biometric marker extracted by the face recognition unit; detecting, by a liveness detection unit comprised in the user authentication device, whether the user identified by the user identification unit is a live person by extracting one or more additional biometric markers from the visual input received by the face recognition unit.

[0014] In one or more embodiments, the additional biometric marker comprises a heart rate.

[0015] In one or more embodiments, the liveness detection unit extracts the heart rate from the visual input by applying color amplification or motion amplification to the visual input.

[0016] In one or more embodiments, the additional biometric marker comprises a facial characteristic and / or a predefined movement pattern of the user’s head using multiple cameras or multiple views analysis.

[0017] According to a third aspect of the disclosure, a computer program is provided, the computer program comprising executable instructions which, when executed by a user authentication device of the kind specified, cause the user authentication device to perform a method of the kind specified. BRIEF DESCRIPTION OF DRAWINGS

[0018] Embodiments will be described in more detail with reference to the drawings.

[0019] Figure 1 An illustrative embodiment of a user authentication device is shown.

[0020] Figure 2 An illustrative embodiment of a method of operating a user authentication device is shown.

[0021] Figure 3 An illustrative embodiment of a user authentication system is shown.

[0022] Figure 4 An illustrative embodiment of a user authentication process is shown.

[0023] Figure 5 Different camera views are shown.

[0024] Figure 6 A movement trajectory of a smartphone is shown.

[0025] Figure 7 A movement trajectory of a head is shown.

[0026] Figure 8A And Figure 8B Another illustrative embodiment of a user authentication process is shown.

[0027] Figure 9 An illustrative embodiment of an enrollment process for a user authentication system is shown. DETAILED DESCRIPTION

[0028] As mentioned above, there are a variety of ways to identify and authenticate a person. One of the ways is to use a biometric, in which case a physical characteristic of a person is used for the identification and authentication of the person. Commonly used physical characteristics are fingerprints, iris patterns, and faces. However, in the case of using face recognition, security risks arise. Specifically, there are several attacks that attempt to circumvent authentication through face recognition. The simplest attack would use a photo or video of a person that an attacker attempts to impersonate. These attacks can be referred to as "liveness attacks" because these attacks aim to trick a biometric authentication system purely by providing a specific, maliciously crafted input. The existence of this type of attack increases the need for countermeasures. Typically, countermeasures against this type of attack in biometrics are referred to as liveness detection. However, cameras used for face recognition can have different qualities, while videos of people and screens used for attacks can have high quality. This makes some liveness detection techniques less effective. Some other types of attacks against face recognition systems can rely on the fact that the hardware used in a face recognition system (e.g., a smartphone) is not very secure and can be attacked by hackers using malicious software.

[0029] A user authentication apparatus that uses face recognition while maintaining a relatively high level of security and a corresponding method of operating the user authentication apparatus are now discussed. Specifically, the presently disclosed user authentication apparatus and corresponding method of operation help to increase resistance to the above-mentioned types of attacks.

[0030] Figure 1An illustrative embodiment of a user authentication apparatus 100 is shown. The user authentication apparatus 100 comprises a face recognition unit 102, a user identification unit 104, and a liveness detection unit 106. Although these units 102, 104, 106 have been shown as separate components, some or all of them can be integrated into the same physical component. Moreover, the units 102, 104, 106 can be implemented as hardware, software, or a combination of hardware and software. For example, some or all of the units 102, 104, 106 can be implemented as a data processor. The face recognition unit 102 is configured to receive a visual input and to extract a biometric signature from the visual input, the biometric signature being a face. Moreover, the user identification unit 104 is configured to identify a user based on the biometric signature extracted by the face recognition unit 102. Finally, the liveness detection unit 106 is configured to detect whether the user identified by the user identification unit 104 is a live person by extracting one or more additional biometric signatures from the visual input received by the face recognition unit 102. By extracting at least one additional biometric signature from the visual input, the liveness of the identified user can be effectively detected, thereby increasing the resistance against attacks on face recognition based authentication.

[0031] In one or more embodiments, the additional biometric signature comprises a heart rate. A heart rate represents a biometric signature that is particularly suitable for verifying the liveness of a user identified by means of face recognition. Thus, in this way, the resistance against attacks on face recognition based authentication is further increased. In one or more embodiments, the liveness detection unit is configured to extract the heart rate from the visual input by applying color amplification and / or motion amplification to the visual input. Applying color amplification or motion amplification to a visual input, e.g. a video input, results in a reliable way to extract a heart rate from the visual input.

[0032] In one or more embodiments, the additional biometric features comprise facial characteristics and / or predefined movement patterns of the head of the user analyzed using multiple cameras or multiple views. These biometric features are particularly suitable to verify the liveliness of the user identified by means of face recognition. Thus, in this way, the resistance against attacks on the authentication based on face recognition is further increased. The facial characteristics can comprise, for example, characteristics of the topology of the face. In one or more embodiments, the additional biometric features comprise a reconstructed three-dimensional model of the face obtained by illuminating the face from different angles with respect to a reference point on the face and by analyzing the effects of the illumination. Such reconstructed model represents a biometric feature particularly suitable to verify the liveliness of the user identified by means of face recognition. Thus, in this way, the resistance against attacks on the authentication based on face recognition is further increased. Moreover, in one or more embodiments, the additional biometric features comprise a predefined behavior of the pupils after the face has been illuminated. The behavior of the pupils of the identified user after the illumination represents a biometric feature particularly suitable to verify the liveliness of the user. Thus, in this way, the resistance against attacks on the authentication based on face recognition is further increased.

[0033] In one or more embodiments, the additional biometric features comprise one or more body movements performed in response to a challenge provided by the liveliness detection unit. By providing such a challenge, i.e. a request to perform a predefined body movement, to the identified user, and analyzing whether the movement is performed correctly, the liveliness of the identified user can be easily verified. Thus, in this way, the resistance against attacks on the authentication based on face recognition is further increased. In one or more embodiments, the liveliness detection unit comprises a secure element configured to provide the challenge. If the challenge is provided by a secure element instead of, for example, a general purpose processor, it will be more difficult for an attacker to gain knowledge about the challenge and manipulate the challenge. Thus, in this way, the resistance against attacks on the authentication based on face recognition is further increased. More specifically, the likelihood of success of the above-mentioned malware-based hacking attacks can be significantly reduced. It should be noted that a secure element can be defined as a tamper-resistant integrated circuit with installed or pre-installed applications, which has a defined functionality as well as a defined security level. Moreover, a secure element can implement security functions, for example, cryptographic functions and authentication functions. In one or more embodiments, the secure element is additionally configured to verify whether the body movement performed in response to the challenge corresponds to an expected response. If the response to the challenge is also verified by a secure element instead of a general purpose processor, the security level is further increased. In a practical implementation, the body movement performed in response to the challenge and the expected response is encoded as a checksum.

[0034] Figure 2An illustrative embodiment of a method 200 of operating a user authentication device is shown. The method 200 comprises the following steps. At 202, a visual input is received by a face recognition unit comprised in the user authentication device and a biometric marker is extracted from the visual input, the biometric marker being a face. At 204, a user is identified by a user identification unit comprised in the user authentication device based on the biometric marker extracted by the face recognition unit. Furthermore, at 206, it is detected by a liveness detection unit comprised in the user authentication device whether the user identified by the user identification unit is a live person by extracting one or more additional biometric markers from the visual input received by the face recognition unit. As mentioned above, by extracting at least one additional biometric marker from the visual input, the liveness of the identified user can effectively be detected, thereby increasing the resistance against attacks on the face recognition based authentication.

[0035] Figure 3 An illustrative embodiment of a user authentication system 300 is shown. The user authentication system 300 comprises a smartphone 302 which can be used to authenticate a user 310. The smartphone 302 comprises a central processing unit (CPU) 304, a graphics processing unit (GPU) 306, a secure element (SE) 308 and a random access memory (RAM) 310. Furthermore, the smartphone 302 comprises a light 312, a camera 314 and a screen 316. The smartphone 302 represents a non-limiting example of a user authentication device of the kind set out. By means of the camera 314, a visual input can be collected, a face can be extracted from the visual input and a liveness check can be performed using an additional biometric marker of the kind set out above. Furthermore, the secure element 308 can be used to implement a challenge-response scheme of the kind set out above. Another non-limiting example of a user authentication device of the kind set out would be a device integrated into a vehicle which makes use of face recognition technology for driver authentication.

[0036] Figure 4 An illustrative embodiment of a user authentication process 400 is shown. The process 400 comprises the following steps. At 402, data is obtained from a camera 402. Next, face recognition is performed 404 using these data. In case of a match 406, i.e. in case of a positive identification of the user, the process 400 continues with trying 410 to extract a heart rate from the data obtained from the camera. Otherwise, the authentication fails 408. If at 412, a heart rate is detectable in the data obtained from the camera, the authentication succeeds 414.

[0037] The presence of heart rate is a good indication that the object in front of the camera is a living person. Heart rate can be extracted from video of a person. At least two techniques can be used to extract this information. The first technique is based on color magnification; for example, this technique is described in the paper “Eulerian video magnification for revealing subtle changes in the world” by Hao-Yu Wu et al., published in ACM Transactions on Graphics (TOG), Volume 31, Issue 4 (DOI: 10.1145 / 2185520.21855). The second technique is based on motion magnification; for example, this technique is described in the paper "Detecting Pulse from Head Motions in Video" by Guha Balakrishnan et al., presented at the IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2013 (DOI: 10.1109 / CVPR.2013.440). It should be noted that although both techniques utilize magnification of small changes in the video, they use different information: the first technique is based on color for its analysis, while the second is based on motion for its analysis.

[0038] Therefore, additional biometric markers (i.e., heart rate) can be extracted from the visual input already available in the system. That is, the camera has already photographed the person for facial recognition purposes, so all the hardware is in place. The presence of heart rate can be used for activity detection by extracting it in one or even two different ways. Furthermore, extracting heart rate in two different ways allows for double verification of the heart rate to counter noise and any attacks that might be able to forge one method but not the other, as such forgery would result in a mismatch between the outputs of the two techniques. It should be noted that two techniques for heart rate detection can be used simultaneously, or only one of the techniques can be used. If the heart rate is detectable but too irregular, this could also indicate an attack. Similarly, if the heart rate is too regular, it may be generated by an algorithm or machine attempting to deceive the system. In other words, some noise is expected in a person's heart rate.

[0039] It should be noted that if a person is holding a user authentication device with a camera, there may be additional movement caused by small hand movements. This effect can be compensated for using image stabilization; in this case, the entire image will be more stable, while the head will still move as assumed by motion magnification techniques. Alternatively, data from the gyroscope can be used to eliminate the effects of movement from the smartphone itself.

[0040] Figure 5 Different camera views 500 are shown. Multiple cameras or multiple views of the same person using a single camera can be used to enhance activity detection and face recognition. In a second scenario, the person may need to turn their head or move the camera to the side (e.g., if the camera is in a smartphone). Figure 5 A system is shown in which two cameras 506, 508 are used to perform dual verification for face recognition, by means of which a person 504 is authenticated. By having two (or more) cameras, the system can perform the dual verification using, for example, a front view 510 and a side view 512. In addition to this dual verification, the system can also verify whether the same points of the face viewed from the side and from the front move as expected and are located in the correct coordinates relative to each other. For example, if the head moves toward the front camera 506 (which can be detected because the head will be larger in the front view 510), it is expected that the head will simultaneously move toward the side in the side view 512 at the same rate. It should be noted that more of these movements can be defined and used during the authentication process.

[0041] Figure 6 and Figure 7 Different movement trajectories are shown, specifically the movement trajectory 600 of the smartphone 602 carried by person 604 and the movement trajectory 700 of the head of person 704 observed by smartphone 702. Specifically, Figure 6 and Figure 7 This illustrates how a system with only a single camera available can detect whether a person 604, 704 identified by facial recognition is alive. With only one camera available, the system can instruct the user to slowly move the camera, such as... Figure 6 As shown, or by turning the user's head left or right, such as Figure 7 As shown. In this way, the system can verify that the person in front of the camera is a real three-dimensional person, not a flat image of a person on a screen or printed photograph. If the user turns their head, the orientation can be randomly selected to avoid using the recorded video. Camera movement has the advantage that movement of the background behind the person indicates that the camera is actually moving; this fact can also be verified using accelerometer readings.

[0042] As mentioned above, the additional biometric markers may include a reconstructed 3D model of the face obtained by illuminating the face from different angles relative to a reference point on the face and analyzing the effects of the illumination. More specifically, light and shadow can be used to verify the shape of the face. If the face is illuminated from different angles, shadows can be observed on different parts of the face. For example, if the light source is on the right side of the person, the shadow of the person's nose will be on the left side of the face. Other features on the face will also be illuminated differently depending on the angle at which the light falls on the face. By analyzing the light and dark spots on the face under several different lighting conditions, a 3D model of the face can be reconstructed and verified against a known model captured during the registration phase. It should be noted that if a screen with a video or photograph of a person is used to attempt to deceive the authentication system, the shadows will not change position in the same way as on a person's real face. Therefore, certain types of attacks against facial recognition can be detected.

[0043] Furthermore, separate models can be used for normal face recognition as well as face recognition using 3D reconstruction based on lighting from different sources. This can be used as an additional verification for authentication. Most devices already have different light sources that can be used to illuminate a person's face during authentication. Smartphones can even use the screen as a light source by increasing screen brightness; for example, by turning all pixels white, the face of a person looking at their phone can be illuminated. Additionally, cars typically have lights directly above the driver, and some modern cars also have light-emitting diode (LED) screens that can be used in the same way as smartphone screens.

[0044] In some cases, different colors of light can be used to verify how shadows and highlights change color when a face is illuminated. This technique can be further enhanced by using two different light sources of different colors simultaneously. If the light sources are located relatively far apart, color adjustments in highlights and shadows can be observed. If these effects are not observed, the system can determine that an attack is likely underway. In addition to examining shadows and highlights when a face is illuminated, pupil behavior can also be analyzed. After being illuminated, the pupil should constrict and then dilate when the additional light source is turned off. Analysis of pupil behavior provides additional validation for activity detection.

[0045] Figure 8A and Figure 8BAnother illustrative embodiment of a user authentication process 800 is shown. Process 800 includes the following steps. At 802, the process begins (e.g., via a general-purpose processor) by capturing data from a camera in an insecure environment. Then, at 804, the data is processed to extract a checksum for face recognition. This checksum (referred to as "checksum C") is provided to a secure element, which compares this checksum with a stored checksum 806. If there is no match 808 between checksum C and the stored checksum, a failure 810 is reported. However, if a match 808 exists, the secure element generates 812 a random challenge R and provides the random challenge to the insecure environment. The environment requests 814 that the user perform challenge R, and the user subsequently performs 816 the challenge. Furthermore, while performing challenge R, the insecure environment analyzes 818 the data from the camera and calculates a checksum corresponding to the challenge (referred to as "checksum CC"), which is provided to the secure element. Next, the secure element compares the checksum CC of challenge R with a stored checksum corresponding to R 820. If there is no match between the checksum CC and the stored checksum (822), a failure (810) is reported. However, if a match (822) is found, a success (824) is reported.

[0046] Figure 8A and Figure 8B The process 800 shown illustrates a non-limiting example of how an activity detection unit can challenge a user to perform one or more body movements and analyze these movements as they are performed by the user to enhance face recognition using challenge-based activity detection. For this purpose, the activity detection unit includes insecure components (e.g., a general-purpose processor) and secure elements. The idea behind this enhancement is to make the authentication process more secure by making it interactive rather than relying on a completely static view of the person during face recognition. This introduces some randomness into the authentication process and, for example, will prevent attacks such as video recordings.

[0047] More specifically, users can be asked to perform actions during facial recognition. The request to perform this action can originate from a secure element, and the final verification (i.e., verification of the response to a challenge) can also be performed by this secure element. In this way, even if the main general-purpose processor is compromised, the secure element will be able to detect the attack. Users can be asked to perform many different actions, such as tilting their head left or right, closing their eyes, opening their mouth, smiling, turning their head left or right, showing their tongue, and looking up, down, left, or right. Furthermore, in some scenarios, these actions can be extended to specific gestures, such as "touch your nose with your right index finger" or "cover your left eye with your left hand."

[0048] Once a user executes a given challenge, the device authenticates the user's gesture or action by verifying whether it is the correct gesture or action (i.e., the gesture or action requested from the user) and by verifying that the user is still the same user. If both are correct, authentication is considered successful. It should be noted that feature extraction can be performed in an insecure environment, while the final verification can be performed by a secure element. The secure element can detect if the user does not perform the required action or if the main system is compromised. If the user is not a legitimate user (e.g., if video recording is used), the user will likely be unable to correctly guess in advance which random action will be requested, thus authentication will fail.

[0049] Figure 9 An illustrative embodiment of a registration process 900 for a user authentication system of the type described is shown. Face recognition is typically performed using certain types of machine learning algorithms. The output of such algorithms for face recognition tasks resembles a large number and is sometimes referred to as a fingerprint, checksum, or feature vector. The algorithm's output is typically stored during the registration phase and compared with the output stored during authentication. The calculation of the checksum can be done in an insecure environment, such as in the CPU or GPU of a smartphone. Conversely, the comparison between the stored checksum and the newly calculated checksum will be performed in a secure environment (i.e., a secure element). Checksums should not be stored in an insecure environment. Furthermore, it is assumed that the system is undamaged during registration.

[0050] Alternatively, a different security system (in a secure physical environment, such as a government building) can be used to extract the initial checksum for registration. In this case, the security system can send the checksum in encrypted form to the secure element of the smartphone. Figure 9 A non-limiting example of such a process 900 is shown. In a secure environment, an image is captured from the camera 902, a checksum is extracted from the data 904, a connection is established 906 with the secure element of the smartphone, and the checksum is sent 908 to the secure element. The secure element establishes a connection with the secure environment 910, obtains the checksum 912, and saves the checksum 914 in its secure memory. To facilitate this registration, the user uses a passport or ID card to prove the user's identity 916, unlocks the phone 918 to prove ownership, takes a photo 920, and launches an application 922 to pair with the secure element to store the checksum.

[0051] The systems and methods described herein may be embodied, at least in part, by computer programs or multiple computer programs, which may exist in various forms, both in use and in development, either on a single computer system or across multiple computer systems. For example, these computer programs may exist as software programs in the form of source code, object code, executable code, or other formats comprising program instructions for performing certain steps. Any of these formats may be embodied, in compressed or uncompressed form, on a computer-readable medium that may include storage devices and signals.

[0052] As used herein, the term "computer" refers to any electronic device that includes processors such as general-purpose central processing units (CPUs), dedicated processors, or microcontrollers. A computer is capable of receiving data (input), performing a series of predetermined operations on the data, and producing results (output) in the form of information or signals. Depending on the context, the term "computer" will particularly mean a processor or more generally a processor associated with a collection of related elements housed within a single housing or enclosure.

[0053] The terms "processor," "data processor," or "processing unit" refer to data processing circuitry, which can be a microprocessor, coprocessor, microcontroller, microcomputer, central processing unit, field-programmable gate array (FPGA), programmable logic circuitry, and / or any circuitry that manipulates signals (analog or digital signals) based on operation instructions stored in memory. The term "memory" refers to storage circuitry or multiple storage circuitry, such as read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and / or any circuitry that stores digital information.

[0054] As used herein, "computer-readable medium" or "storage medium" can be any component capable of containing, storing, transmitting, propagating, or transmitting a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, apparatuses, or propagation media.

[0055] It should be noted that the above embodiments have been described with reference to different subjects. Specifically, some embodiments may be described with reference to claims of the method class, while other embodiments may be described with reference to claims of the device class. However, those skilled in the art will understand from the foregoing that, unless otherwise indicated, any combination of features related to different subjects, specifically, a combination of features of the method class claims and features of the device class claims, is also considered to be disclosed with this document, except for any combination of features belonging to one type of subject matter.

[0056] Additionally, it should be noted that the accompanying drawings are schematic. Similar or identical elements are indicated by the same reference numerals in different drawings. Furthermore, it should be noted that, in an effort to provide a concise description of illustrative embodiments, implementation details that are customary practices to those skilled in the art may not be described. It should be understood that in the development of any such implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's specific objectives, such as complying with system-related and business-related constraints, which may differ between different implementations. Furthermore, it should be understood that such development work can be complex and time-consuming, but is merely a routine task for those skilled in the art in designing, manufacturing, and producing.

[0057] Finally, it should be noted that those skilled in the art should be able to devise numerous alternative embodiments without departing from the scope of the appended claims. Any reference numerals placed in parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of other elements or steps besides those listed in the claims. The word "a" preceding an element does not exclude the presence of a plurality of such elements. The measures described in the claims can be implemented by means of hardware comprising several different elements and / or by means of a suitably programmed processor. In a device claim listing several components, several of these components may be embodied by the same item of hardware. The fact that certain measures are recited in different appendix claims alone does not imply that combinations of these measures cannot be used to gain an advantage.

[0058] List of reference numerals in the attached diagram:

[0059] 100 User Authentication Device

[0060] 102 facial recognition units

[0061] 104 User Identification Unit

[0062] 106 Activity Detection Units

[0063] 200 Method for operating a user authentication device

[0064] 202 receives visual input and extracts biometric markers from the visual input by a face recognition unit included in the user authentication device, the biometric markers being a face.

[0065] 204. The user is identified by a user identification unit included in the user authentication device based on biometric tags extracted by the facial recognition unit.

[0066] 206 The activity detection unit included in the user authentication device detects whether the user identified by the user identification unit is a living person by extracting one or more additional biometric markers from the visual input received by the face recognition unit.

[0067] 300 User Authentication System

[0068] 302 Smartphone

[0069] 304 Central Processing Unit (CPU)

[0070] 306 Graphics Processing Units (GPUs)

[0071] 308 Safety Components (SE)

[0072] 310 Random Access Memory (RAM)

[0073] 310 Users requiring authentication

[0074] 312 lamps

[0075] 314 camera

[0076] 316 screen

[0077] 400 User Authentication Process

[0078] 402 Data Acquisition from Camera

[0079] 404 Performing facial recognition

[0080] 406 match?

[0081] 408 Authentication Failed

[0082] 410 Attempting to extract heart rate

[0083] Can a heart rate of 412 be detected?

[0084] 414 Authentication Successful

[0085] 500 camera views

[0086] 502 Top View

[0087] 504 people

[0088] 506 Front Camera

[0089] 508 Side Camera

[0090] 510 Front camera view

[0091] 512 Side camera view

[0092] The movement trajectory of 600 smartphones

[0093] 602 Smartphone

[0094] 604 people

[0095] 700 Head movement trajectory

[0096] 702 Smartphone

[0097] 704 people

[0098] 800 User Authentication Process

[0099] 802 begins capturing data from the camera.

[0100] 804 processes the data to extract a checksum for face recognition.

[0101] The 806 compares the checksum C with the stored checksum.

[0102] 808 match?

[0103] 810 report failed

[0104] 812 Generate random challenges R

[0105] 814 requires the user to perform a challenge R

[0106] 816 Execution Inquiry

[0107] 818 analyzes data from the camera during challenge execution and calculates the checksum corresponding to the challenge.

[0108] 820 Compares the checksum CC of the challenged R with the checksum stored in R. 822 Match?

[0109] 824 Report Successful

[0110] 900 is used for the registration process of the user authentication system.

[0111] 902 Capture images from the camera

[0112] 904 Extract Checksum

[0113] 906 establishes a connection with the SE of the user's smartphone.

[0114] 908 sends the checksum to SE

[0115] 910 Establishing connections with a secure environment

[0116] 912 Obtain checksum

[0117] 914 stores the checksum in secure memory.

[0118] 916 Use passport or ID card to prove identity

[0119] 918 Unlock your phone to prove ownership

[0120] Take photos at 920.

[0121] The 922 is used for applications that pair SEs to store checksums.

Claims

1. A user authentication device, characterized in that, include: A face recognition unit is configured to receive visual input and extract biometric markers from the visual input, the biometric markers being a face; A user identification unit is configured to identify a user based on the biometric markers extracted by the face recognition unit; An activity detection unit is configured to detect whether the user identified by the user identification unit is a living person by extracting one or more additional biometric markers from the visual input received by the face recognition unit.

2. The user authentication device according to claim 1, characterized in that, The additional biometric markers include heart rate.

3. The user authentication device according to claim 2, characterized in that, The activity detection unit is configured to extract the heart rate from the visual input by applying color amplification and / or motion amplification to the visual input.

4. The user authentication device according to any one of the preceding claims, characterized in that, The additional biometric markers include facial features analyzed using multiple cameras or multiple views and / or predefined movement patterns of the user's head.

5. The user authentication device according to any one of the preceding claims, characterized in that, The additional biometric markers include a reconstructed three-dimensional model of the face obtained by illuminating the face from different angles relative to a reference point on the face and by analyzing the effect of the illumination.

6. The user authentication device according to any one of the preceding claims, characterized in that, The additional biometric markers include predefined behavior of the pupils after the face is illuminated.

7. The user authentication device according to any one of the preceding claims, characterized in that, The additional biometric markers include one or more body movements performed in response to a challenge provided by the activity detection unit.

8. The user authentication device according to claim 7, characterized in that, The activity detection unit includes a security element configured to provide the challenge.

9. A method for operating a user authentication device, characterized in that, include: The user authentication device includes a face recognition unit that receives visual input and extracts biometric markers from the visual input, wherein the biometric markers are faces. The user is identified by a user identification unit included in the user authentication device based on the biometric markers extracted by the face recognition unit; The liveness detection unit, included in the user authentication device, detects whether the user identified by the user identification unit is a living person by extracting one or more additional biometric markers from the visual input received by the face recognition unit.

10. A computer program comprising executable instructions, characterized in that, When executed by the user authentication device according to any one of claims 1 to 8, the executable instructions cause the user authentication device to perform the method according to claim 9.

Images