Facial recognition authentication system including path parameters

Abstract

The present invention relates to a facial recognition authentication system including path parameters. The system and method for registering and authenticating a user in an authentication system via a computing device of the user with a camera include: capturing and storing registration biometric information from at least one first image of the user taken via the camera of the computing device; capturing authentication biometric information from at least one second image of the user; capturing path parameters indicating authentication movement of the mobile device via at least one movement detection sensor during imaging of the at least one second image; comparing the authentication biometric information with the stored registration biometric information; and comparing the authentication movement of the mobile device with an expected movement of the mobile device to determine whether the authentication movement sufficiently corresponds to the expected movement.

Description

[0001] This application is a divisional application of Chinese patent application CN201510544125.5. The application date of the original application is August 28, 2015, the priority date is August 28, 2014, and the name of the invention is “Facial recognition authentication system including path parameters”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 043,224 filed on August 28, 2014, U.S. Provisional Application No. 62 / 054,847 filed on September 24, 2014, U.S. Provisional Application No. 62 / 064,415 filed on October 15, 2014, U.S. Provisional Application No. 62 / 085,963 filed on December 1, 2014, U.S. Provisional Application No. 62 / 101,317 filed on January 8, 2015, U.S. Provisional Application No. 62 / 139,558 filed on March 27, 2015, and U.S. Provisional Application No. 62 / 188,584 filed on July 3, 2015. The contents of each of the above applications are incorporated by reference. Technical Field

[0004] The disclosed embodiments relate to biometric security. More particularly, the disclosed embodiments relate to facial recognition authentication systems. Background Art

[0005] With the growth of personal electronic devices that can be used to access several user accounts and the growing threats of identity theft and other security issues, there is a growing demand for ways to safely access user accounts via electronic devices. Account owners therefore often need to have longer passwords that meet various standards such as using a mixture of uppercase and lowercase letters, numbers and other symbols. With smaller electronic devices such as smart phones, smart watches, "Internet of Things" ("Internet of Things", "IoT"), etc., it becomes annoying to try to tap the long password required for each access to an account into the device. In some cases, because of the inconvenience on their devices, users even decide to disable this troublesome security measure. Therefore, users of such devices may prefer other methods of safely accessing their user accounts.

[0006] One other such method may be through the use of biometrics. For example, an electronic device may have an optical reader that can scan a user's fingerprint to determine that the person requesting access to the device or account is authenticated. However, such fingerprint systems are often too expensive to use on small electronic devices or are generally considered unreliable and unsafe.

[0007] In addition, facial recognition is generally well known and can be used in a variety of environments. Two-dimensional facial recognition is generally used to tag people in social network images or photo editing software. However, facial recognition software has not been widely used alone to securely authenticate users attempting to gain account access because it is not considered secure enough. For example, two-dimensional facial recognition is considered unsafe because the face can be photographed or recorded, and the resulting photo or video display showing the user's image can be used to deceive the system. Therefore, there is a need for a reliable, cost-effective and convenient method to authenticate users attempting to log into, for example, a user account. Summary of the invention

[0008] In view of the foregoing, the disclosed embodiments have been improved, and aspects of the invention may include a method for registering and authenticating a user in an authentication system via a user's mobile computing device. The user's device includes a camera and at least one motion detection sensor such as an accelerometer, a magnetometer, and a gyroscope.

[0009] In one embodiment, a user can register with the system by providing a registration image of the user's face. The registration image is captured by the camera of the mobile device as the user moves the mobile device to different positions relative to the user's head. The user can thus obtain registration images showing the user's face from different angles and distances. The system can also use one or more motion sensors of the mobile device to determine the registration movement path captured by the phone during imaging. At least one image is processed to detect the user's face within the image, and biometric information is obtained from the user's face in the image. Image processing can be completed on the user's mobile device or at a remote device such as an authentication server or user account server. Registration information (registration biometrics, movement and other information) can be stored on the mobile device or the remote device.

[0010] The system can then authenticate the user by the user providing at least one authentication image via a camera of the mobile device while the user moves the mobile device to different positions relative to the user's head. The authentication image is processed for facial detection and facial biometric information. Path parameters are also acquired during imaging of the authentication image (authentication movement). The authentication information (authentication biometric, movement, and other information) is then compared to the enrollment information to determine whether the user should be authenticated or rejected. The image processing and comparison can be performed on the user's mobile device or remotely.

[0011] In some embodiments, multiple registration profiles may be generated by the user to provide further security. For example, a user may generate a registration wearing an accessory such as a hat or glasses, or while making a smiling face. In further embodiments, the user's registration information may be linked to the user's email address, phone number, or other identifier.

[0012] The authentication system may include feedback displayed on the mobile device to assist the user in learning and authenticating with the system. For example, an accuracy meter may provide feedback on the matching rate of the authentication biometric or movement. A movement meter may provide feedback on the movement detected by the mobile device.

[0013] In some embodiments, the system may provide rewards to users who successfully use the authentication system or otherwise employ fraud avoidance measures. Such rewards may include leaderboards, status levels, reward points, coupons or other offers, etc. In some embodiments, the authentication system may be used to log into multiple accounts.

[0014] In addition to biometrics and mobile matching, some embodiments may also use strip detection, glare detection, and screen edge detection to further protect the system. In other embodiments, other user attributes including the user's gender, age, race, etc. may be detected and matched.

[0015] The system can also provide progressive access to user accounts when a user first sets up the authentication system. Authorization can be extended as the user successfully implements the system. For example, lower transaction limits can be applied during a period when the user becomes familiar with the authentication system.

[0016] In some embodiments, the mobile device may show video feedback of an object that the user is imaging to assist the user in imaging his or her face during registration or authentication. The video feedback may be displayed on only a portion of the display screen of the mobile device. For example, the video feedback may be displayed on an upper portion of the display screen. The video feedback display may be positioned on a portion of the display screen corresponding to the position of the front-facing camera of the mobile device.

[0017] To aid in low light imaging, screen areas outside of the video feed may be displayed in a bright color such as white. In some embodiments, LED or infrared light may be used, and near infrared thermal imaging may be accomplished with an infrared camera. A mobile device for imaging may therefore have multiple cameras to capture both visible light and infrared images. A mobile device may also have multiple cameras that image in a single spectrum to provide stereoscopic three-dimensional images.

[0018] In some embodiments, to provide additional security, the mobile device can output the object, color, or pattern to be detected on the display screen during imaging. The predetermined object or pattern can be a unique one-dimensional or two-dimensional barcode. For example, a QR code (two-dimensional barcode) can be displayed on the screen and reflected by the user's eyes. If the QR code is detected in the image, the person can be authenticated. In other embodiments, the object can be moved on the screen and the system can detect whether the user's eyes follow the movement.

[0019] In some embodiments, the system may provide a prompt on the video feedback display to assist the user in moving the device about the user's head during registration and / or authentication. The prompt may include an oval or frame displayed on the display screen in which the user must place his or her face by moving the mobile device until his or her face is within the oval or frame. The prompt may preferably have different sizes and may also be centered on different positions of the screen. When the real three-dimensional image of the person is close to or far away from him or herself, it has been found that the biometric results are different due to the fisheye effect of the lens. Therefore, the three-dimensional person can be authenticated when the biometric results in the close or far away images are different. This also allows the user to have multiple biometric profiles for each distance.

[0020] In other embodiments, for incrementally different biometric results, biometrics from images acquired between the close and far images may be analyzed. In this way, the facial deformation from the far face to the distorted close face is captured and tracked. The incremental frames during authentication can then be matched with frames captured at similar locations during enrollment along the motion path and compared to ensure that the desired similarities and differences are found. The result is a motion path and captured images and biometric data that can prove the three-dimensional person is currently being imaged. Thus, not only are the close and far biometrics compared, but biometric data is acquired between the close and far. The biometric data acquired in between must also correspond to the correct deformation speed along the motion path, greatly enhancing the security of the system.

[0021] A touch screen may be utilized in some embodiments. For example, a user may need to enter a slide specific code or pattern outside of the authentication system described herein. The touch screen may also detect the size and orientation of the user's fingers, and whether the left or right hand is being used on the touch screen. Voice parameters may also be used as an additional layer of security. The system may detect edge sharpness or other indicators to ensure that the image obtained is of sufficient quality for the authentication system.

[0022] When the camera has auto focus, the auto focus can be controlled by the system to verify the presence of a real three-dimensional person. The auto focus can check different features of the user or focus on the environment at different focal lengths. In other embodiments, the authentication image can be saved to review the person trying to authenticate with the system.

[0023] In some embodiments, the required match threshold may be adjusted over time. The system may thus account for changes in biometrics due to age, weight gain / loss, environment, user experience, security level, or other factors. In further embodiments, the system may utilize image distortion prior to acquiring biometric information to further prevent fraudulent access.

[0024] As described herein, the system may utilize any number of security features or combinations of security features as security layers. When authentication fails, the system may be configured so that it is unclear which security layer triggered the failure to maintain the integrity of the security system.

[0025] Other systems, methods, features and advantages of the present invention will be or become apparent to those skilled in the art based on a review of the following drawings and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, within the scope of the present invention and protected by the accompanying claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The components in the drawings are not necessarily to scale, emphasis instead being placed on illustrating the principles of the present invention. In the drawings, like reference numerals refer to corresponding components throughout the different views, and these drawings are:

[0027] Figure 1 illustrates an example environment for use of a facial recognition authentication system according to an exemplary embodiment;

[0028] Figure 2 An example embodiment of a mobile device is illustrated;

[0029] Figure 3 Illustrate exemplary software modules of a mobile device and a portion of a server;

[0030] Figure 4 A method for implementing facial recognition authentication according to one embodiment is shown;

[0031] Figure 5 A method for registering a user in a facial recognition authentication system according to an exemplary embodiment is shown;

[0032] Fig. 6A and 6B illustrates an example of movement of a mobile device around a user's face according to an exemplary embodiment;

[0033] Fig. 7A and 7B illustrates an example of movement of a mobile device around a user's face according to an exemplary embodiment;

[0034] Figure 8 A method for providing authentication information in a facial recognition authentication system according to an exemplary embodiment is shown;

[0035] Fig. 9 A method of verifying an authentication certificate in a facial recognition authentication system according to an exemplary embodiment is shown;

[0036] Fig.10Figure illustrates an exemplary display showing image and digital feedback in a facial recognition authentication system;

[0037] Fig.11A , 11B and 11C illustrate an exemplary video feedback display corresponding to a front-facing camera position in a facial recognition authentication system;

[0038] Fig.12 An exemplary video display feedback of a facial recognition authentication system showing horizontal stretching of edge pixels on the sides of the display;

[0039] Fig.13A and 13B illustrates an exemplary screen display with a facial alignment indicator illustrated as an oval to serve as a guide as a user moves the mobile device toward or away from their face;

[0040] Fig.14 Figure illustrates an exemplary mobile device display showing a graphical code entry interface with an imaging area;

[0041] Fig.15 Figure illustrates an example mobile device display showing a numeric and graphical code entry interface with an imaging area. DETAILED DESCRIPTION

[0042] The following describes systems and methods for providing secure and convenient facial recognition authentication. The systems and methods can be implemented without the need for additional expensive biometric readers or systems while providing enhanced security over traditional facial recognition systems.

[0043] Facial recognition authentication environment

[0044] Figure 1 The figure shows an example environment for the use of the facial recognition authentication system described in this article. However, this is one possible use environment and system. It is expected that after reading the description provided below in conjunction with the accompanying drawings, a person of ordinary skill in the art can obtain different use environments and configurations.

[0045] In this environment, a user 108 may have a mobile device 112 that can be used to access one or more of the user accounts via an authentication system. The user 108 may have a mobile device 112 that is capable of capturing a picture of the user 108, such as an image of the user's face. The user may capture an image or images of himself or herself using a camera 114 on or connected to the mobile device 112. The mobile device 112 may include any type of mobile device capable of capturing still or video images and performing image processing or communication over a network.

[0046] In this embodiment, a user 108 may carry and hold a mobile device 112 to capture images. The user may also wear or hold any number of other devices. For example, the user may wear a watch 130 that includes one or more cameras 134 or biosensors located on the watch. The camera 134 may be configured to generate images from visible light as well as infrared light. The camera 134 may additionally or alternatively employ image intensification, active lighting, or thermal imaging to acquire images in dark environments.

[0047] When pointed toward user 108, camera 134 may capture an image of the user's face. Camera 134 may include, for example, NFC is part of a module that has the ability to communicate with the mobile device 112 or other formats, or directly communicate with the network 116 over a wired or wireless link 154. The watch 130 may include a screen on its face to allow the user to browse information. If the camera module 134 is in communication with the mobile device 112, the mobile device 134 may relay the communication to the network 116. The mobile device 134 may be configured with more than one front-facing camera 114 to provide a 3D or stereoscopic view, or to obtain images in different spectral ranges such as near infrared and visible light.

[0048] The mobile device 112 is configured to wirelessly communicate with a remote server 120 over a network 116. The server 120 may communicate with one or more databases 124. The network 116 may be any type of network capable of communicating to or from the mobile device, including but not limited to a local area network, a wide area network, a personal area network (PAN), or the Internet. The mobile device 112 may communicate with the network via a wired or wireless connection such as Ethernet, WiFi, NFC, etc. The server 120 may include any type of computing device capable of communicating with the mobile device 112. The server 120 and the mobile device 112 are configured with a processor and a memory, and are configured to execute machine-readable code or machine instructions stored in the memory.

[0049] The database 124 stored on the illustrated mobile device or remote location may contain facial biometric information and authentication information of the user 108 in order to identify the user 108 to allow access to associated user data based on one or more images or biometric information received from the mobile device 112 or watch 134. For example, the data may be information about a user account or instructions to allow access to a separate account information server 120B. The term biometric data may include information other than biometric information and path parameters related to facial features. Examples of path parameters may include the acceleration and speed of the mobile device, the angle of the mobile device during image capture, the distance of the mobile device from the user, the path direction associated with the user regarding the user's facial location, or any other type of parameter associated with the movement of the mobile device or the user's face with respect to the camera. Other data such as GPS data, device identification information, etc. may also be included.

[0050] In this embodiment, the server 120 processes the identification request from the mobile device 112 or the user 108. In one configuration, the images captured by the mobile device 112 using facial detection include one or more images of the user's face 108 during movement of the mobile device, such as in an edge-to-edge or horizontal arc or line, a vertical arc or line, forward or backward from the user's face, or any other direction of motion. In another configuration, the mobile device 112 calculates biometric information from the acquired images and sends the biometric information to the server 120. In yet another embodiment, the mobile device 112 compares the biometric information to stored biometric information on the mobile device 112 and sends the authentication result from the comparison to the server 120.

[0051] The data includes images, biometric information, or both, which are sent to the server 120 via the network 116. Using image processing and image recognition algorithms, the server 120 processes the individual's biometric information, such as facial data, and compares the biometric information to biometric data stored in the database 124 to determine the likelihood of a match. In other embodiments, the image processing and comparison are completed on the mobile device 112, and the data sent to the server indicates the results of the comparison. In further embodiments, the image processing and comparison are completed on the mobile device 112 without accessing the server, for example, to gain access to the mobile device 112 itself.

[0052] By using facial recognition processing, an accurate identity match can be established. Based on this and optionally one or more other factors, access can be granted or unauthorized users can be denied. Facial recognition processing is known in the art (or is an established process) and therefore will not be described in detail here.

[0053] Also shown is a second server 120B with an associated second database 124B, and a third server 120C with an associated third database 124C. The second and third databases may be provided to contain additional information that is not available on the server 120 and database 124. For example, only one of the additional servers may be accessed based on authentication of the user 108 performed by the server 120.

[0054] Executing on the mobile device 112 are one or more software applications. Here, this software is defined as an identification application (ID App). The ID App can be configured using one or both of facial detection and facial recognition, and one or more software modules, wherein the software modules monitor path parameters and / or biometric data. Facial detection as used herein is referred to as the process of detecting a face in an image. Facial recognition as used herein refers to a process that can use algorithms to analyze a face, map it to facial features, and convert them into biometric data, such as numerical data. The biometric data can be compared with data derived from one or more different images for similarity or dissimilarity. If a high percentage of similarity is found in the biometric data, the individual shown in the image will be considered a match.

[0055] For the ultimate purpose of authenticating a user by comparing the user's face to identities or images stored in database 124, the ID App may first process the image captured by cameras 114, 134 to identify and locate the face in the image. Figure 1 As shown, this may be the face 108. Authentication may be used to log into an online account or for other access control functions.

[0056] The portion of the photo containing the detected face can then be cropped, cut, and stored for processing by one or more facial recognition algorithms. By first detecting the face in the image and cropping only a portion of the face, the facial recognition algorithm does not need to process the entire image. In addition, in embodiments where facial recognition processing occurs remotely from the mobile device 112, such as at the server 120, less image data needs to be sent over the network to the remote location. It is contemplated that the entire image, the cropped face, or only the biometric data can be sent to the remote server 120 for processing.

[0057] Facial detection software is capable of detecting faces from multiple angles. However, facial recognition algorithms are more accurate for straight lines on an image in adequate lighting conditions. In one embodiment, the highest quality facial images captured for facial recognition are processed first, followed by facial images of lower quality or at different angles than directly toward the face. Processing can occur at a mobile device or remote server that has access to a large database of image data or facial recognition data.

[0058] Facial detection preferably occurs on the mobile device and is performed by mobile device software such as the ID App. This reduces the number and size of images (data) to be sent to the server for processing and minimizes the overall amount of data that must be sent over the network where no face is found in the image. This reduces bandwidth requirements and reduces network speed requirements.

[0059] In another preferred embodiment, facial detection, facial recognition and biometric comparison all occur on the mobile device. However, it is contemplated that facial recognition processing may occur on the mobile device, the remote server, or both.

[0060] Figure 2 An exemplary embodiment of a mobile device is shown. This is only one possible mobile device configuration, and as such it is contemplated that one skilled in the art may configure the mobile device differently. Mobile device 200 may be any type of mobile communication device capable of performing as described below. The mobile device may include a PDA, a cellular phone, a smart phone, a tablet PC, a wireless electronic board, an IoT device, a "wearable" electronic device, or any other computing device.

[0061] In this exemplary embodiment, the mobile device 200 is configured with an external housing 204, which is configured to protect and contain the components described below. Inside the housing 204 is a processor 208, and first and second buses 212A, 212B (collectively referred to as 212). The processor 208 communicates with other components in the mobile device 200 via the bus 212. The processor 208 may include any type of processor or controller that can operate as described herein. The processor 208 may include a general-purpose processor, ASIC, ARM, DSP, controller, or any other type of processing device. The processor 208 and other elements of the mobile device 200 receive power from a battery 220 or other power source. The electrical interface 224 provides one or more electrical ports to electrically dock with the mobile device, such as with a second electronic device, a computer, a medical device, or a power / charging device. The interface 224 may include any type of electronic interface or connector format.

[0062] One or more memories 210 are part of the mobile device 200 for storing machine-readable code for execution on the processor 208, and for storing data, such as image data, audio data, user data, medical data, location data, accelerometer data, or any other type of data. The memory 210 may include RAM, ROM, flash memory, optical memory, or micro-drive memory. The machine-readable code described herein is non-transitory.

[0063] As part of this embodiment, the processor 208 is connected to a user interface 216. The user interface 216 may include any system or device configured to accept user input to control the mobile device. The user interface 216 may include one or more of the following: a keyboard, a trackball, a button, a gear, a fixed-point key, a touch pad, and a touch screen. A touch screen controller 230 is also provided, which interacts through the bus 212 and is connected to the display 228.

[0064] The display includes any type of display screen that is configured to display visual information to a user. The screen may include an LED, LCD, thin film transistor screen, OEL CSTN (color super twisted nematic phase), TFT (thin film transistor), TFD (thin film diode), OLED (organic light emitting diode), AMOLED display (active matrix organic light emitting diode), capacitive touch screen, resistive touch screen, or any combination of these technologies. The display 228 receives signals from the processor 208 and the display converts these signals into text and images that can be understood by a person skilled in the art. The display 228 may further include a display processor (not shown) or a controller that interfaces with the processor 208. The touch screen controller 230 may include a module configured to receive signals from a touch screen, wherein the touch screen is covered on the display 228.

[0065] Also part of the exemplary mobile device are a speaker 234 and a microphone 238. The speaker 234 and the microphone 238 may be controlled by the processor 208. The microphone 238 is configured to receive and convert audio signals into electrical signals based on the control of the processor 208. Similarly, the processor 208 may activate the microphone 234 to generate an audio signal. These devices operate in a manner understood by those skilled in the art and are therefore not described in detail herein.

[0066] Additionally, connected to one or more of the buses 212 are a first wireless transceiver 240 and a second wireless transceiver 244, wherein each of the first and second wireless transceivers is connected to a respective antenna 248, 252. The first and second transceivers 240, 244 are configured to receive incoming signals from a remote transmitter and perform analog front-end processing on the signals to generate analog baseband signals. The incoming signals may be further processed by conversion to a data format for later processing by the processor 208, such as by emulating a data converter. Similarly, the first and second transceivers 240, 244 are configured to receive outgoing signals from the processor 208 or other components of the mobile device 208 and up-convert these signals from baseband to RF frequencies for transmission through respective antennas 248, 252. Although demonstrated using a first wireless transceiver 240 and a second wireless transceiver 244, it is contemplated that the mobile device 200 may have only one such system or two or more transceivers. For example, some devices are devices capable of tri-band or quad-band, or have NFC, or other communication-capable devices.

[0067] It is contemplated that the mobile device, and therefore the first wireless transceiver 240 and the second wireless transceiver 244, can be configured to operate according to any existing or future developed wireless standard, including but not limited to Bluetooth, WI-FI, such as IEEE 802.11a, b, g, n, Wireless LAN, WMAN, Broadband Fixed Access, WiMAX, any cellular technology including CDMA, GSM, EDGE, 3G, 4G, 5G, TDMA, AMPS, FRS, GMRS, Citizens Band Radio, VHF, AM, FM, and Wireless USB.

[0068] In addition, a part of the mobile device is one or more systems connected to the second bus 212B, wherein the bus is also docked with the processor 208. These devices include a global positioning system (GPS) module 260 with an associated antenna 262. The GPS module 260 can receive and process signals from satellites or other transponders to generate location data related to the speed of the position, direction of travel and GPS module 260. GPS is a technology commonly understood by those skilled in the art and therefore will not be described in detail here. A gyroscope 264 is connected to the bus 212B to generate and provide orientation data related to the orientation of the mobile device 204. A magnetometer 268 is set to provide direction information to the mobile device 204. An accelerometer 272 is connected to the bus 212B to provide information or data related to the vibration or force experienced by the mobile device. In one configuration, the accelerometer 272 and the gyroscope 264 generate and provide data to the processor 208 to indicate the moving path and orientation of the mobile device.

[0069] One or more cameras (still, video, or both) 276 are provided to capture image data for storage in memory 210 and / or for possible transmission via a wireless or wired link or for viewing at a later time. One or more cameras 276 may be configured to detect images using visible light and / or near-field co-linear light. Cameras 276 may also be configured to use image enhancement, active lighting, or thermal vision to obtain images in dark environments. Processor 208 may process image data to perform image recognition, such as in the case of facial detection, item detection, facial recognition, item recognition, or bar / box code reading.

[0070] A flasher and / or flashlight 280, such as an LED light, is provided and is processor controllable. The flasher and / or flashlight 280 may be used as a strobe or a conventional flashlight. The flasher or flashlight 280 may also be configured to emit an approach infrared light. A power management module 284 interfaces with the battery 220 and monitors the battery 220 to manage power consumption, control battery charging, and provide supply voltage to various devices that may require different power requirements.

[0071] Figure 3 Exemplary software modules as part of a mobile device and a server are shown. Other software modules may be provided to provide the functionality described below. Provision is made for the functionality described herein to be the presence of matching software (non-transient machine readable code, machine executable instructions or code) configured to perform the functionality. The software will be stored on a memory and executable by a processor.

[0072] In this example configuration, mobile device 304 includes a receiving module 320 and a transmitting module 322. These software modules are configured to receive and send data to a remote device, such as a camera, glasses, server, cellular tower, or WIFI system, such as a router or access point.

[0073] Also part of the mobile device 304 is a location detection module 324 that is configured to determine the location of the mobile device, for example using triangulation or GPS. An account settings module 326 is provided to establish, store, and allow a user to adjust account settings. A log in module 328 is also provided to allow a user to log into the mobile device 304, for example using password protection. A facial detection module 308 is provided to execute a facial detection algorithm, and a facial recognition module 321 includes software code that recognizes a user's face or facial features, for example, to create a numerical value representing one or more unique facial features of the user (facial biometric information).

[0074] The information display module 314 controls the display of information to the user of the mobile device. The display may be present on the screen of the mobile device or a watch. The user input / output module 316 is configured to receive data from the user and display data to the user. The local interface 318 is configured to interface with other local devices, such as using Or other short range communications, or wired links using connectors to connected cameras, batteries, data storage elements. All software (with associated hardware) shown in mobile device 304 is used to provide the functionality described herein.

[0075] Figure 3 Also shown in the server software module 350. These modules are located remotely from the mobile device, but can be located at any server or remote processing element. As is known to those skilled in the art, networks and network data use a distributed processing approach to utilize multiple servers and databases working together to provide a unified server. As a result, it is contemplated that the modules shown in the server block 350 may not all be located on the same server or in the same physical location.

[0076] like Figure 3 As shown, server 350 includes a receiving module 352 and a transmitting module 354. These software modules are configured to receive and send data to a remote device, such as a camera, watch, glasses, server, cellular tower, or WIFI system, such as a router or access point.

[0077] The information display module 356 controls the display of information at the server 350. The user input / output module 358 controls the user interface in conjunction with the local interface module 360. Also located on the server side of the system is a facial recognition module 366, which is configured to process image data from the mobile device. The facial recognition module 366 can process the image data to generate facial data (biometric information) and perform comparison functions with other facial data to determine a facial match as part of a recognition determination.

[0078] The database interface 368 enables communication with one or more databases containing information used by the server modules. The location detection module 370 can use the location data from the mobile device 304 for processing and to increase accuracy. Similarly, the account setup module 372 controls user accounts and can interface with the account setup module 326 of the mobile device 304. A second server interface 374 is provided to interface and communicate with one or more other servers.

[0079] One or more databases or database interfaces are provided to facilitate communication with and searching of databases. In this exemplary embodiment, the system includes an image database that includes images or image data for one or more persons. Such a database interface 362 can be used to access image data users as part of the identification and matching process. These two modules 376, 364 are used to establish privacy settings for individuals and to access databases that may contain privacy settings.

[0080] Authentication system

[0081] Now combine Figure 4 An authentication system with path parameters that can operate in the environment and system described above is introduced. Figure 4 An authentication method for performing facial recognition using path parameters according to one embodiment of the present invention is shown. As will be described in more detail below, the system uses the features of the mobile device 112 and server 120 defined above to generate a secure and convenient login system, such as an example of an authentication system. This reduces the burden on the user of having to type complex passwords into the small screen of the mobile device, prevents fraud through means such as password registration or screenshot capture, and increases security by combining multiple path parameters and / or device parameters that must be met before the user is authenticated.

[0082] At step 410, the system registers the user in the facial recognition authentication system. In one embodiment, an authentication server, such as server 120 ( Figure 1 ), may be configured to authenticate a user to allow access to a user account, such as a bank or other account, via mobile device 112. Authentication server 120 may be included as part of a server of an institution or entity that provides a user account (hereinafter referred to as an "account server"), or an authentication server may be provided separately. Figure 1 In the environment shown in , servers 120B and 120C can represent account servers. In other embodiments, the account server and the authentication service are the same server. In one embodiment, the authentication server 102 can provide an authentication application to the user to be installed on the mobile device 112.

[0083] Will refer to Figure 510B . The registration process according to one embodiment is described below. In this embodiment, at step 510, a user via mobile device 112 establishes a connection between mobile device 112 and account server 120B. As just one example, the user may establish a connection to a server of a financial institution (e.g., a bank), or may establish this connection later in a post-authentication process. Then, at step 512, the user provides the usual login information to authenticate the user, such as a username or password for a financial account. At step 514, the user may then receive a prompt on mobile device 112 to register in a facial recognition authentication system. The user then indicates via the user interface that he or she is willing to establish the authentication system in response to the prompt.

[0084] Next, at step 516, the mobile device 112 may send device information to the authentication server 120. Among the various information, the device information may include a device identifier that uniquely identifies the user's mobile device. Such information may include the device manufacturer, model, serial number, and mobile network information. At step 518, when the authentication server 120 is combined with the account server 120B, the authentication server 120 stores the user account information and associates the user account information with the device information. When the authentication server 120 is independent of the account server 120B, the account server 120B may generate a unique identifier associated with the account information and send the unique identifier to the authentication server 120. The authentication server 120 may associate the device information and the unique identifier with each other and may store the information in the database 124.

[0085] Next, in step 510, the user is prompted to use the camera 114 on the mobile device 112 to provide multiple images of his or her face (hereinafter, "enrollment images"). As the user holds the mobile device and moves the mobile device to different orientations relative to his or her head and face, enrollment images of the user's face are captured. Thus, enrollment images of the user's face are captured from many different angles or orientations. In addition, in step 522, path parameters of the mobile device are monitored and recorded for future comparison. Figures 6A-7B Some non-limiting examples of how a user may hold a mobile device and capture multiple images of her face are shown in FIG.

[0086] exist Fig. 6A and Figure 6B In , the user holds the mobile device 112 on one side of his or her face and moves the mobile device 112 horizontally in an arc path around his or her face until the mobile device 112 is on the other side of his or her face. Fig. 7A and Figure 7BIn the example of FIG. 1 , the user holds the mobile device 112 away from his or her face, and then moves the mobile device 112 closer to his or her face. Figures 6A-7B Many other paths may be used in addition to those shown in . Additionally, the user may move his or her head while the camera is held stationary. The user may also hold the camera steady and move their head relative to the camera. Thus, this method may be implemented in conjunction with a webcam on a laptop or desktop computer, or on any other device such as an IoT device, where the camera is mounted on a similar fixed location or object.

[0087] The enrollment image may be obtained as follows. The user holds and orients the mobile device 112 having the camera 114 so that the camera 114 is positioned to image the user's face. For example, the user may use a front-facing camera 114 on a mobile device 112 having a display screen and may confirm on the display screen that his or her face is in a suitable position to be imaged by the camera 114.

[0088] Once the user has oriented the device, the device can begin to obtain the user's enrollment image. In one embodiment, the user can press a button on the device 112 (such as on a touch screen) or other buttons on the device to start the acquisition of the enrollment image. Then, as described above, when the device images the user's face from multiple angles or orientations, the user moves the mobile device to different orientations relative to his or her head. When using the above-mentioned front camera, the user can continuously confirm that his or her face is being imaged by watching the imaging on the display screen. The user can press the button again to indicate that imaging is complete. Alternatively, the user can hold the button during imaging, and then release the button to indicate that imaging is complete.

[0089] As described above, mobile device 112 may include facial detection. In this embodiment, in step 524, the mobile device may detect the face of the user in each of the enrollment images, crop the image to include only the face of the user, and send the image to authentication server 120 via the network. In step 526, upon receiving the enrollment image, authentication server 120 performs facial recognition on the image to determine biometric information for the user ("enrollment biometric"). In step 528, authentication server 120 may then associate the enrollment biometric with device information and a unique identifier (or account information), and store the biometric information in database 124. For added security, in step 530, mobile device 112 and authentication server 120 may be configured to delete the enrollment image after obtaining the enrollment biometric of the user.

[0090] In another embodiment, the mobile device 112 may send the image to the authentication server 120 without performing facial detection. The authentication server 120 may then perform facial detection, facial recognition, and biometric information processing. In another embodiment, the mobile device 112 may be configured to perform facial detection, facial recognition, and biometric processing, and then send the results or data generated from the processing to the authentication server 120 to be associated with a unique identifier or user account. This prevents sensitive personal data (images) from leaving the user's device. In yet another embodiment, the mobile device 112 may perform each of the above-mentioned steps, and the mobile device 112 may store the enrollment information without sending any of the enrollment biometrics or images to the server.

[0091] In one embodiment, the gyroscope, magnetometer and accelerometer of the mobile device are configured to generate and store data when the user moves the mobile device around his or her head to obtain a registration image (path parameters). The mobile device can process this data in step 532 to determine the path or arc in which the mobile device moves when the user images his or her face. By using data from the accelerometer, magnetometer and gyroscope, the system can check when the user is ready to start scanning himself / herself, and determine the scanning path. Therefore, this data can be used to determine when to start and stop the scanning time interval. In addition, the data can include the time elapsed during the scan. This time can be measured from the user pressing a button to start and stop imaging, or can be measured from the duration of the button being kept pressed while imaging, or during more movement or to complete the swing.

[0092] The registered movement of the mobile device 112, which is data defining the movement of the mobile device during image capture, can be sent to the authentication server 120. The authentication server 120 associates and stores the registered movement, the registered biometrics, the device information, and the unique identifier or account information. Alternatively, data generated by a gyroscope, a magnetometer, and an accelerometer can be sent to the server 120, and the server 120 can process the data to determine the registered movement.

[0093] Thus, in the above described embodiments, the registration information may thus include device information, registration biometrics, and registration movement (based on the movement of the mobile device 112).

[0094] Reference Figure 4Once registration is complete, authentication server 120 may then receive credentials from a user attempting to authenticate with the system, as shown in step 420. For example, a user may attempt to log into a user account. When a user attempts to log in, instead of or in addition to providing typical account credentials such as a username and password, the user may log in in the same manner as done during registration (such as in Figures 6A-7B ), multiple images or videos of his or her face are again captured (“authentication images”) while the mobile device 112 is held in the hand and moved to different orientations relative to the head. In this way, the user can provide the necessary images (the word image encompasses video, as video is a continuous image) from many different angles and / or orientations, as well as path parameters of the device when the image was obtained (“authentication movement”) to confirm both the user's identity and the liveness and authenticity of the individual to ensure it is not a video, screenshot, or other representation of a person.

[0095] exist Figure 8 In one embodiment as outlined in FIG. 8 , in step 810, a user obtains multiple authentication images via a mobile device 112 while moving the mobile device 112 to different orientations relative to the user's head. Using facial detection in step 812, the mobile device 112 detects the user's face in each of the authentication images, crops the image, and sends the image to the authentication server 120. In another embodiment, the mobile device 112 sends the image to the server 124, and the server 124 performs facial detection. In step 814, the authentication server 120 may perform facial recognition on the authentication images to obtain biometric information (“authentication biometrics”). In another embodiment, the mobile device 112 performs facial recognition to obtain authentication biometrics and sends the authentication biometrics to the server 120.

[0096] In step 816, mobile device 112 sends device information identifying the device and path parameters such as gyroscope, magnetometer, and accelerometer information defining the path of the mobile device during imaging, as well as the time elapsed during imaging ("authentication movement") to server 120. Thus, the credentials received by authentication server 120 for logging into a facial recognition system may include device information, authentication images or authentication biometrics, and authentication movement (path parameters).

[0097] Reference Figure 4 In step 430, authentication server 120 verifies that the certificate received from mobile device 112 sufficiently corresponds to the information obtained during registration. Fig. 9By using an algorithm to process features of the face and the light shining on the face between different images, the authentication server 120 can determine that the face in the authentication image is three-dimensional, i.e., not a representation in a printed image or video screen. In the case where the mobile device 112 sends only the authentication biometric to the server 120, the server 120 can verify the authenticity or three-dimensional aspect of the imaged user by comparing the results of the biometrics of the different images.

[0098] In step 920, the authentication server 120 may then compare the login credentials to the information stored from the authentication process. In step 920, the server 120 compares the identification of the device obtained during the login process to the identification of the device stored during registration. In step 930, the authentication biometric may be compared to the registration biometric to determine if they sufficiently correspond to the registration biometric. In step 940, the authentication movement may be compared to the registration movement to determine if it sufficiently corresponds to the registration movement.

[0099] In some embodiments, a copy of the registration information may be stored on the mobile device 112, and the mobile device 112 may verify that the certificate received on the mobile device 112 sufficiently corresponds to the registration information. This would allow the user to secure documents, files, or applications on the mobile device 112 itself in addition to the user's account hosted on a remote device such as the authentication server 120, even when the connection to the authentication server 120 may be temporarily unavailable, such as when the user does not have access to the Internet. In addition, this would allow the user to secure access to the mobile device 112 itself. Alternatively, the registration information may be stored on a server.

[0100] Thus, in step 950, if the authentication server 120 or mobile device 112 determines that the registration information sufficiently corresponds to the received credentials, the server or mobile device can verify that the identity of the user attempting to log in corresponds to the account holder. This avoids the cumbersome process of the user having to manually type in a complex password using the small screen of the mobile device. Many passwords now require uppercase letters, non-text characters, lowercase letters, and numbers.

[0101] The required level of correspondence for determining that the enrollment information sufficiently corresponds to the authentication information in a login attempt may be set in advance. For example, the level of correspondence may be a 99.9% match rate between the enrollment biometric and the authentication biometric, and a 90% match rate between the enrollment biometric and the authentication biometric. The required level of correspondence may be static or may be flexible based on established thresholds.

[0102] For example, the required level of correspondence may be based on GPS information from the mobile device 112. In one embodiment, when the GPS information of the mobile device corresponds to the user's residence or other authorized location(s), the authentication server 120 may require a 99.9% match rate as the level of correspondence. In contrast, if the GPS information shows that the device is in a foreign country far from the user's residence, the authentication server may require a 99.99% match rate as the level of correspondence or may reject it altogether. Thus, the required match between previously stored authentication data (registration information) and currently received authentication data (authentication information) is flexible in that the required percentage match between path parameters or images may vary depending on various factors, such as time of day, location, frequency of login attempts, date, or any other factor.

[0103] Additionally, the required level of correspondence may be time dependent. For example, if a second authentication attempt is made shortly after a first authentication attempt in a location that is far away from the first authentication location based on GPS information from mobile device 112, the threshold for the level of correspondence may be set higher. For example, a user may not be able to travel from Seattle to New York in 1 hour. Likewise, for some users, login attempts between midnight and three in the morning may be a sign of fraud based on the user's usage patterns.

[0104] The level of correspondence between the registration information and the authentication information can be the result of combining various parameters of the registration information and the authentication information. For example, when the button holding time in the authentication information is within 5% of the button holding time of the registration information, the correspondence of the button holding time can constitute 20% of the overall match. Similarly, when the motion path trajectory of the authentication information is within 10% of the registration information, the motion path trajectory can constitute 20% of the overall match. When compared with the registration information, the other parameter matching rates such as face size and facial recognition matching in the authentication information can constitute the remaining 10% and 50% of the overall corresponding level. In this way, the total overall corresponding level can be adjusted (for example, the total corresponding level of all parameters exceeds 75%), or the matching rate of individual parameters can be adjusted. For example, when trying to log in for the second time, the threshold matching rate of one parameter can be increased, or the total corresponding level for all parameters can be increased. The threshold matching rate can also be adjusted based on the account being authenticated or other different desired security levels.

[0105] Reference Figure 4, in step 440, authentication server 120 may grant or deny access based on the verification in step 430. For example, if authentication server 120 verifies that the certificate matches the registration information, server 120 may authenticate the user to allow access to the user's account. In the case where authentication server 120 is separate from account server 120B (such as a bank's server), authentication server 120 may transmit the unique identifier to the account server along with an indication that the identity of the user associated with the unique identifier has been verified. Account server 120B may then authorize the user's mobile device 112 to transmit data to and receive data from account server 120B. Of course, all of this may occur at account server 120B alone or on mobile device 112 itself.

[0106] Alternatively, if the credentials provided by the user are not verified, the authentication server may transmit a message indicating that the login attempt failed for display on the screen of the mobile device 112. The authentication server 120 may then allow the user to attempt to log in again via the facial recognition login system, or the authentication server 120 may require the user to enter typical account credentials, such as a username and password.

[0107] In one embodiment, the server 120 may allow three consecutive failed login attempts before requiring a username and password. If in one of the attempts, the required level of correspondence is met, the user may be verified and access may be granted. According to one embodiment, the authentication server 120 may retain information from each consecutive authentication attempt and combine data from multiple authentication attempts to obtain more accurate facial biometric information of the person attempting authentication. In addition, the level of correspondence may be increased with each consecutive authentication attempt. In addition, the login data (registration information) is refined and improved by averaging path data (authentication movement) and / or image data (authentication image / biometric) from several login attempts.

[0108] Thus, the above-described authentication system allows for authentication to a remote server 120 or authentication on the mobile device 112 itself. As described above, this can be accomplished by: the mobile device 112 captures the authentication certificate, and the authentication server 120 processes and analyzes the certificate in comparison to the registration information (cloud processing and analysis); the mobile device 112 captures the authentication certificate and processes the certificate, and the authentication server 120 analyzes the certificate in comparison to the registration information (mobile device processing, cloud analysis); or the mobile device 112 captures the authentication certificate, and processes and analyzes the certificate in comparison to the registration information (mobile device processing and analysis).

[0109] Advantages and features of embodiments

[0110] The above system provides many advantages. As an example, a facial recognition authentication system provides secure login. For example, if during a login attempt, when the phone is not moving, the camera of the mobile device images a digital screen showing a person rotating their head, the accelerometer, magnetometer, and gyroscope data will not detect any movement. Therefore, the registration information and the authentication movement will not correspond, and the login attempt will be rejected.

[0111] Additionally, because multiple images are used as enrollment images and authentication images, histograms or other photo processing techniques can be used to determine if a digital screen appears in the proper location for the face of the person in the image. For example, the system can check for frequency changes in light in the captured image, or banding in the image (which would indicate an electronic display generating an image), backlighting, suspicious changes in lighting, or other analysis on the image to determine that an actual live user is in fact active, present, and requesting authorization for login by comparing the images.

[0112] As yet another advantage, as explained above, not only must the enrollment biometrics fully correspond to the authentication biometrics, but the enrollment movement must match the authentication movement, and the device information must match the enrollment device information. For example, an application can be downloaded to a mobile device with a digital camera. The application can be a login application, or it can be an application from a financial institution or other entity with which the user has an account. The user can then log in to the application using typical login credentials such as a website username and password. In addition, the user may have a device code that logs in on another device, or may use a camera to scan a QR code or other such code to pair the device with their user account.

[0113] Next, the user holds the mobile device to move the mobile device to different orientations relative to his or her head while maintaining his or her face visible to the camera as the mobile device is moved. As the mobile device is moved, the camera captures a registration image of the face. During imaging, an accelerometer, magnetometer, and gyroscope are used to measure the speed and angle of the current user's mobile device movement to generate the registration movement. In addition, throughout this process, continuous imaging and detection of the face has been shown to prevent spoofing. This is because spoofing attempts cannot be made by rotating the image in and out of front of the camera.

[0114] For example, Fig. 6A and 6BAs shown in FIG. 7 , the user can start the movement from right to left and from left to right. As shown in FIG. 7 , the movement can also be in the front-to-back direction. Any other movement can be used, such as starting from the center, then to the right, and then back to the center. Vertical and diagonal movements can also be used to further mix the complexity of the registration movement. When the user attempts to log in later, the user must repeat the movement pattern in the authentication movement so that in addition to matching the biometric data and device information, the registration movement is also matched. Therefore, the security of the system is greatly improved.

[0115] Thus, the system provides security for authenticating a user with a mobile device. As described above, the system can securely authenticate a user using at least any one or more of the following in any combination: physical device verification, mobile network verification, facial recognition including the size of the face in the image, faces detected in each frame during movement, accelerometer information, gyroscope information, magnetometer information, pixels per square inch, color bits per pixel, image type, a code or pattern entered by the user, and GPS information.

[0116] Another advantage is that the facial recognition login system provides a convenient way for users to log into their accounts with mobile devices. For example, once registered, a user does not need to enter a username and password on a small mobile device each time the user wishes to access the account. Instead, the user only needs to image himself or herself while imitating someone else's registration movements with a mobile device. This is particularly advantageous for smaller mobile devices such as mobile phones, smart watches, etc.

[0117] The system can also be configured to allow users to securely log in to multiple devices or to allow users to securely share devices. In one embodiment, the registration information can be stored on an authentication server (e.g., on the "cloud") and is therefore not associated only with the user's original device. This allows the user to use any number of suitable devices to authenticate with the authentication server. In this way, the user can use a friend's phone (third-party device) or other device to access his or her information, such as account information, address book information, emails or other messages, etc., by performing authentication operations on any device.

[0118] For example, a user may provide an email address, username code, or similar identifier on a friend's phone, causing the authentication server to compare the login information with the registration information for the user's account. This will indicate to the authentication server which authentication profile to use, but does not access the user's data, account, or tasks by itself. Once the friend's phone is logged off, access to the user's information on the friend's phone is terminated. This provides the advantage of allowing the user to use any device to securely access an account or other authentication-accessible information or tasks without having to enter the user's password in a third-party device (which may be recorded or copied in the third-party device). In a sense, the user is the password.

[0119] Through cloud-based registration information, a single user can also securely transfer information between authenticated devices. In one embodiment, a user may have a first device, such as a mobile phone, and be authenticated via the authentication system on the first device. The user can then acquire a new device, such as a new phone, tablet, or other device. By using a cloud-based authentication system, the user can authenticate on the new device and transfer data from the first device to the new device. The transfer of data can be completed via the Internet, local network connection, Bluetooth connection, wired connection, near-field communication, etc. The authentication process can also be part of a security check of a retransmission or recovery system after a mobile phone is stolen or lost. Therefore, the authentication system can be used to activate or authenticate a new device, and the authenticated user verifies the user of the new device.

[0120] Similarly, the system can facilitate secure access to a single device shared by multiple people to control content or other features on the device. In many cases, passwords can be viewed, copied, guessed, or otherwise detected, especially when a device is shared by multiple users. Users can be family members including, for example, parents and children, co-workers, or other relationships such as students. The authentication system can allow any family member to log in based on his or her unique registration information associated with the user account.

[0121] Devices can restrict access to specific content or features to one or more specific user accounts, such as a child's user account, while allowing access to content or features for others, such as a parent's account. As described above, by using an authentication system for shared devices, users such as children cannot use a password to try and access restricted content because the authentication system requires the parent to be present for authentication. Devices shared between users with different privileges are thus further protected and improved. Similarly, in a classroom setting, a single device can be securely shared between multiple people for testing, searching, and grade reporting.

[0122] Adaptation and modification

[0123] Various modifications may be made to the above-described apparatus and method without departing from the scope of the present invention. For example, the image may be processed on the device by a facial identification algorithm and may also be converted on the device into biometric data, which is then compared to previously authenticated biometric data of the user. Alternatively, the image from the device may be sent over a wired or wireless network where a facial recognition algorithm running on a separate server can process the image, create biometric data, and compare the data to previously stored data assigned to the device.

[0124] Multiple profiles for a single user

[0125] In addition, the photo registration process can be performed multiple times for a user to create multiple user profiles. For example, a user can register in the following situations: wearing or not wearing glasses, with or without other wearable devices, under different lighting conditions, wearing a hat, with a different hairstyle, with or without facial or ear jewelry, or making different and unique faces (such as closing eyes, blinking or sticking out tongue) to establish another level of uniqueness for each user profile. Such "faces" made by users will not be available on the user's social media page, and therefore cannot be used for copying, manipulation and use in fraudulent behavior. Each set of registration images, registration biometrics or both can be saved with separate registration moves. In one embodiment, when the mobile device completes the path, at least three images are captured. It can be considered that any number of images can be captured.

[0126] Link registration information

[0127] It can also be considered that the registration process can be linked to an email address, a mobile phone number, or other identifiers. For example, a user can sign with an email address, complete one or more registrations as described above, and confirm the registration via the same email address. The email address can then further enhance the security of the system. For example, if a user does not successfully attempt to log in via the authentication system for a predetermined number of times, such as three times, the authentication system locks the account and sends an email to the email address to notify the user of the unsuccessful login attempt. The email can also include one or more pictures of the person who failed to log in and GPS or other data from the login attempt. The user can then confirm whether this is a valid login attempt and reset the system, or the user can report the login attempt as fraud. If there is a reported fraudulent login, or if there are too many locks, the system can delete the account associated with the email address to protect the user's security. Therefore, future fraud attempts are not possible.

[0128] Feedback Meter

[0129] To further facilitate imaging, the mobile device may include various feedback meters, such as Fig.10 1024 indicates the amount of movement made by the user as the mobile device 1012 moves relative to his or her head to different positions. For example, the movement meter 1024 can be represented as a straight line sliding from one side of the screen. In this way, the registration process may require a certain threshold of device movement in order to register the user with the multi-dimensional authentication system. For example, the system may require the mobile device 1012 to move in an arc or straight line and rotate at least 45 degrees in order to create registration information. For another example, the system may require the acceleration of the device to exceed a threshold. The movement meter can also help the user learn how to use the authentication system to image him / herself.

[0130] The mobile device 1012 may also display a correctness meter 1026 or any other visual representation of the authentication frame to assist the user in authenticating himself / herself using the authentication system. The correctness meter 1026 may display to the user the matching rate (graphical, alpha, or numerical) of a predetermined number of images obtained during the authentication process. The correctness meter may be displayed on the display in a variety of ways, including numerical percentages, color representations, graphical representations, etc. A combination of representations may also be used.

[0131] For example, Fig.10 As shown, the matching speed of a predetermined number of images performed during authentication is displayed on a correctness meter. Fig.10 In the illustrated embodiment, each image may be represented by a column in the picture, and the correctness may be displayed in each column for each image. For example, a bar with a longer bar indicates a higher correctness, and a column with a shorter bar indicates a lower correctness. In addition to the matching correctness of the image, the matching correctness of the path parameters may also be displayed. The user may make improvements over time.

[0132] In another embodiment, each image is represented on the table as a color corresponding to the matching rate. The color dark green can represent a very high matching rate, light green can represent a good matching rate, red can represent a medium matching rate, and gray can represent a poor matching rate. Other color schemes can also be used.

[0133] The height or color of the bar used can correspond to a predetermined match rate. For example, a full bar or dark green can be a match rate above 99.9%, three quarters of a bar or light green can be a match rate between 90% and 99.9%, half a bar or yellow can be a match rate of 50%-90%, red can be a match rate of 20%-50%, and a single line equal to a quarter of a bar or gray can be a match rate of 0%-20%. A pie chart, line graph, or any other type of representation or any other numerical or graphical display can also be used. An overall score or a score for each image can be calculated.

[0134] The correctness meter may also include a message 1028 indicating the overall match score. For example, the correctness meter may indicate the average overall match rate or the number of images that achieved a 99.9% match rate and display the message to the user. By using the mobile meter 1024 and correctness meter 1026 as described above, the user can quickly learn to use the authentication system due to the feedback provided by the meters 1024, 1026.

[0135] Gamification and Rewards

[0136] The movement and correctness meters 1024, 1026 can also be configured to incorporate game features, aspects or techniques into the authentication system to encourage users to try and get the best possible matches (e.g., high scores or high percentages of frames), increasing the user's skill in using the authentication system. This also builds user adoption of this technology.

[0137] For example, users can be encouraged or trained to achieve high scores by themselves by emulating or improving past certification scores. Further improvements to the certification meter can also be incorporated, such as the ability to share correctness match results with others to demonstrate one's skill in using the system or against others' accomplishments. In other cases, users can receive rewards, such as gifts or tickets, for high correctness scores. Although this will increase costs slightly, the reduced fraud losses far outweigh the additional costs.

[0138] Further gaming techniques can be incorporated into the authentication system to encourage users to take actions to prevent unauthenticated or fraudulent authentications. In one embodiment, the authentication system can reward users for residual fraud prevention activities. One such activity is the facial recognition authentication system described herein. For example, based on the correctness meter described above, the system can reward users who successfully pass the system above a specific matching rate. The system can reward point rewards, cash, or other prizes based on the successful authentication or a predetermined number of successful authentications. In the case of utilizing point reward points, the points can be cashed in for a predetermined prize.

[0139] Other game features may involve reward levels for users who gain a predetermined amount of experience using the authentication feature. For example, different reward levels may be based on a user successfully authenticating 100 times, 500 times, 1000 times, etc. Because every instance of fraud loss is significant and can damage the goodwill of a business or organization, the benefits of preventing fraud are significant.

[0140] In one embodiment, a user may be notified of the different levels of completion he or she has achieved, such as a "silver level" for reaching 100 successful certifications and a "gold level" for reaching 1000 successful certifications. The number of points awarded for each of the above certifications that exceeds a given match rate may increase based on the user's experience level. Of course, the names of the levels and the number of certifications for each level as described above are merely exemplary and may vary as required.

[0141] In one embodiment, the certification only counts the reward level for doing business on the website, while in another embodiment, repeated attempts can be made, all of which count towards the reward. Another feature can include a leaderboard (where the user can be notified of the user's ranking) to compare his or her ability or willingness to other users when using the certification system.

[0142] Successfully utilizing the authentication system is beneficial to the companies and organizations utilizing the system by reducing the cost of fraudulent activity and the expense of preventing fraudulent activity. These cost savings can be used to fund the gaming features of the authentication system described above.

[0143] Further actions corresponding to the authentication system and helping to reduce fraud can also be incorporated to allow users to earn points or obtain prizes. Such activities can include users creating long enough and powerful passwords that have used a certain number and combination of characters. This encourages and rewards users to set passwords that are not easily compromised. In addition to initial authentication such as mobile phone or email authentication, answering one or more personal questions, or other auxiliary authentication currently known or developed in the future, other examples can include rewarding users to spend time performing authentication steps. This rewards users for the increased time and inconvenience of reducing the risk of fraud in a company or organization.

[0144] As another example, if the authentication service is used to log into a website or application that offers a membership program, then rewards or gifts can be subsidized from an affiliate commission for purchases made on these websites. For example, if a business (product or service) website utilizes the methods and apparatus herein to avoid fraud and thereby increase profits, then a percentage of each purchase made by a user using the authentication service will be provided to the authentication service. By reducing fraud, consumers are more likely to purchase, and additional users will be willing to enter financial and personal information. An affiliated link, code, or referral source or identifier can be used to direct consumers to the authentication system of the business (product or service) website.

[0145] Multiple account login

[0146] It is also contemplated that the authentication system may be configured to allow a user to access multiple different websites as a result of a single authentication. Since the authentication process and results are unique to a user, the user may first specify the participating websites that the user chooses to log into, and then select one or more websites to log into, and the user performs the authentication described herein. If the security authentication is successful, the user is logged into the selected website. In this way, the authentication process is subject to common access control for multiple different websites and prevents users from having to remember multiple different usernames and passwords, while also reducing fraud and password overhead for each user.

[0147] Automatic start / stop imaging

[0148] It is also contemplated that the authentication system can be configured so that the video camera runs on the phone. The mobile device will capture frames and path parameter data (using gyroscopes, magnetometers, and accelerometers) as the phone moves, but will only process biometric data on the device or send frames to the server if they have faces in them. In this embodiment, once the phone moves, the application executing on the mobile device can trigger the software application to start saving frames, and then if the phone continues to move along the correct path (e.g., a semicircle) and the system detects a face in the frame, the mobile device will start sending images, portions of images, or biometric data to the server for processing. When the system senses movement, it triggers the capture of images at specific intervals. The application can then process the frames to determine if the image contains a face. If the image includes a face, the application will crop it out and then verify if the mobile device's path of movement is similar to the path of movement used during enrollment. If the path of movement is similar enough, the application can send the frames to be scanned or processed one at a time to the server as described above.

[0149] Banding and edge detection

[0150] When a fraud attempt is made using a display screen (such as an LED, LCD or other screen), the system can detect fraudulent login attempts based on the expected properties of the screen. In one embodiment, the authentication system will run checks for bands generated by the digital screen. When a band is detected, the system can identify a fraud attempt at login. In another embodiment, the system will run checks for edge detection of the digital screen. As the mobile device is moved during login to obtain authentication movement, the system checks the captured image for the edges of the screen to identify fraudulent login attempts. The system can also check for image artifacts obtained from the screen, such as glare detection. Any algorithm for band and screen edge detection known now or developed later can be used. Once fraud is detected, authentication will be blocked and transactions or account access will be blocked for access to the website.

[0151] Other property estimates

[0152] The authentication system may further perform analysis on the enrollment image to estimate gender, approximate age, and race. In an alternative embodiment, the user may manually enter one or more of their gender, approximate age, and race, or the information may be retrieved or obtained from existing records known to be correct. The authentication system may then further store the user's estimated gender, age, and race as enrollment credentials or user data. Thus when the user later attempts to authenticate using the system, the system will compare the obtained gender, age, and race obtained from the authentication image (based on processing using biometric analysis to determine such data or an estimate thereof) with the stored gender, age, and race to determine whether to authenticate the user. For example, if the obtained data fields for gender, age, and race match the stored enrollment credentials, then the authentication is successful or this aspect of the authentication is successful.

[0153] The authentication system can make gender, age, and race estimates based on a single image or based on multiple images during the authentication process. For example, the authentication system can use one of the multiple images that has the best view of the user's face for analysis. In another embodiment, different images can be used for analysis of each of age, gender, and race when the different images present the best data for analysis. The authentication system can also estimate gender, age, and race in multiple images and average the results to obtain an overall score for gender, age, and race.

[0154] As an alternative to obtaining gender, age, and race as registration information, the estimated gender, age, and race estimate as an authentication credential may be set over a series of repeated uses of the authentication system. For example, if a previous successful authentication used biometrics and movement information, and the authentication system always estimated the user's age to be between 40 and 50 years old, then the authentication may be set for subsequent credentials required for that user, and the login information to include a facial image estimated to be between 40 and 50 years old. Alternatively, the gender, age, and race estimate may be implemented as one of multiple factors contributing to an overall authentication score to determine whether to authenticate a user.

[0155] For example, if the authentication process has a gender estimate of 1.9 (plus or minus 0.2) for a male rating, then if the actual result does not fall within that range, then the system can deny access to that user. Similarly, if the user's age range in previous authentication attempts or registrations always falls between 40-50 years old, and the authentication attempt falls outside of that range, then the system can deny access or use that result as a compounding factor to deny access.

[0156] In a further embodiment, when a bracelet or watch capable of obtaining an EKG signature is used, a specific EKG signature may be required at login. The EKG signature may also be paired with facial recognition rotation to provide multiple stages of sign-on for critical security and identification applications. Further, the above-mentioned certificate may also include GPS information that only allows logins in specific geographic locations as defined in the registration. In one configuration, the GPS coordinates of the mobile device are recorded and logged for login attempts or actual logins. This is additional information about the user's location. For example, if the GPS coordinates are located in a foreign country where fraud is known, then the attempt is likely to be fraudulent, and if the GPS coordinates indicate that the attempt or login was made at the user's home, then it is unlikely to be fraudulent. In addition, some applications may only allow users to log in at specific locations such as secure government facilities or when they are in a hospital.

[0157] The registration information may further include distance information. Since the motion arc (speed, angle, duration) is unique for each user, the facial detection software on the device can process the image and determine whether the device is too close or too far from the subject. Or in other words, the registration information can take into account the size of the face in the image. Therefore, the possible registration information can also change based on the length of the user's arms, the head and face size, and the optical system in the user's specific mobile device. The user can also be located at a fixed computer or camera, such as a laptop, desktop, or ATM. The user can then move the face (or a combination of these actions) forward or backward, from side to side, or up and down to create an image. Therefore, the method of this operation is not limited to mobile devices. In one embodiment, the camera is located in a car, such as in a mirror, and people move their heads or faces for authentication.

[0158] Step-by-step authentication access

[0159] In one embodiment, the system is set up to limit the operations that a user can perform when initially registering and authenticating. Subsequently, after further authentication or after a predetermined time period and number of authentications, additional capabilities may be granted. For example, during the first 20 authentications within the first 3 months, a maximum of $100 transactions may be allowed. This builds a database of known authentication data in conjunction with transactions that the user does not object to. Subsequently, during the next 20 authentications, a transaction limit of $3000 may be established. When the authentication data is limited and the user is new to the system, this limits the overall loss in the event of fraud, for example if an unauthorized user is able to fraudulently register in the authentication system.

[0160] Video playback for imaging

[0161] When a user uses the front camera to image themselves, as described above, the user can confirm that his / her face has been imaged by viewing the image on the display. The image displayed on the display is configured so as to be smaller than the entire display in area, and can be located in the upper part of the display towards the top of the device. When the user's image is displayed only in the upper part of the user's display screen, the user's eyes tend to look closer at the front camera. When the user's eyes are tracked, the accuracy of facial recognition can be improved. Further, tracking the movement of the eyes from frame to frame allows the system to verify that the image is a live person and is not from a photo or video recording of the person.

[0162] The image shown on the display can also be positioned to match the camera on the user's device, such as Figures 11A-11CAs shown. Mobile devices available today include front-facing cameras configured in a variety of different locations. For example, a mobile device 1112a, 1112b may have a front-facing camera 1114a, 1114b that is configured above the display and off-center toward one side or the other, such as Fig.11A and 11B Therefore, the images 1116a and 1116b of the user feedback displayed on the display may be located at positions corresponding to the positions of the cameras 1114a and 1114b as shown. Fig.11A In the example, camera 1114a is above the display and offset to the left of the center, then image 1116a can be displayed in the upper left corner of the display. Fig. 11B In the example, camera 1114b is above the display and offset to the right of the center, then image 1116b may be displayed in the upper right corner of the display. Fig. 11C As shown, mobile device 1112c may have camera 1114c configured in the center just above the display. Then image 1116c can be displayed in the middle of the upper part of the display. In this way, the user's eyes are guided as close to and / or tracked as close to the camera as possible, which helps eye tracking and movement verification. When the user moves the mobile device, the user can also better see the feedback image and other feedback or information on the screen.

[0163] The image viewed by the user on the display may be further modified so that edge pixels on the sides of the display are stretched horizontally, such as Fig.12 As shown. That is, the predetermined areas 1206, 1208 on the left and right sides are distorted to stretch toward the left and right edges of the screen, respectively. This allows a larger vertical portion of the displayed image to be displayed on the display. At the same time, this trains the user to use the system correctly by keeping his or her face in the middle of the screen, because if it deviates from the middle and part of the face enters one of the distorted areas, then his or her face will be distorted on the screen.

[0164] Authentication in low-light environments

[0165] To aid in imaging, the screen on the mobile device may additionally be displayed with a white background, and the brightness of the screen may be increased to illuminate the user's face in a dark environment. For example, a portion of the display may provide video feedback for the user to confirm that he or she is imaging themselves, while the remainder of the screen is configured to display a bright white color. Fig. 11C As shown, this can be achieved by displaying the video feedback 1116c in the middle of the display and displaying a bright white bar in the peripheral area around the video feedback 1116c. In very dark conditions, the LED flash and rear camera on the back side of the mobile device can be used. Alternatively, the camera can be configured to create an image using infrared or other night vision technology.

[0166] When infrared images as thermal imaging are used, it is possible to further improve security. In detail, thermal imaging can be analyzed to indicate whether the image obtained is from an actual user or a fraudulent image from a screen or other device. When a person is in front of an infrared thermal imaging camera, the detected thermal radiation should be a relatively elliptical shape indicating a person's head. In contrast, the thermal radiation from the screen is typically rectangular. Further, the thermal pattern in the actual human face detected and the movement of the thermal pattern in the image can be compared with the thermal pattern of the expected human face to distinguish images from fraudulent authorization attempts using the screen.

[0167] Detecting output from mobile devices

[0168] The display or other light source on the mobile device can be further utilized to provide additional security measures. During the above-mentioned authentication process, light from the display or other light source is projected onto the user's face and eyes. This projection can then be detected by the camera of the mobile device during imaging. For example, the tones detected on the skin, or the light reflected from the user's cornea can be captured by the camera on the mobile phone. Because of this, random light patterns, colors, and patterns can be utilized to provide further security and ensure that it is a live person who is attempting to authenticate and not just an image or video of a person programmed by a fraudster.

[0169] As an example, when a user begins authentication, the authentication server can generate and send instructions to the user's device to display a random sequence of colors at random intervals. The authentication server stores the randomly generated sequence for the user to compare with the authentication information received from the mobile device later. During authentication imaging, the color displayed by the device is projected on the user's face and reflected by the user's eyes (cornea) or any other surface that receives and reflects light from the screen. The camera on the user's mobile device detects the color reflected by the user's skin or eyes (or other surfaces) and generates color data indicating the color detected based on the screen projection. This data can be returned to the server to determine whether the color sequence or pattern sent to the mobile device matches the known sequence or pattern projected by the screen of the user's device. At the authentication server, based on the comparison, the authentication is successful or rejected. The comparison with the random sequence of colors in the instruction can alternatively occur only in the user's device to determine that the user being authenticated is a live user.

[0170] As another example, when a user begins authentication, the authentication server may send an instruction to the user's device to display a randomly generated pattern, which is then stored on the authentication server. The pattern may include graphics, text, lines or bars, flashing patterns, colors, QR codes, etc. The randomly uploaded pattern is displayed during authentication imaging, and the pattern is reflected by the user's eyes (cornea). The camera of the user's device detects the pattern reflected by the user's eyes and processes the mirror image of the reflected, displayed pattern. The processed pattern (e.g., converted into a numerical value) is transmitted to the authentication server and compared with the randomly generated and stored pattern on the authentication server to verify whether the pattern displayed on the screen and imaged after reflection on the user's face establishes a match.

[0171] If a match occurs, this establishes or increases the likelihood that the device is imaging a live person. If the pattern does not match, or the match threshold is not met, then the authentication process may be video (access denied) or account access or transaction quantity may be limited. Note that this example may also be included on a desktop computer with a webcam, which does not include the above-mentioned enrollment and authentication moves. Further, this example may include not only facial recognition, but also iris recognition or other types of eye blood vessel recognition as an additional security layer, or any facial feature unique to the user.

[0172] When the above example is implemented on a desktop computer, eye tracking can also be used to further prove the presence of a live user. For example, the screen can show a ball or other random object or symbol moving in a random pattern, and the user looks at the ball with his or her eyes. The movement of the eyes relative to the pattern is then tracked by the reflection of the screen of the eyes. Eye tracking can also be established by establishing an anchor point, such as by clicking a mouse at a certain location on the screen (assuming that the user is looking at the location where the mouse click occurs), and then estimating the location of the user looking at the screen relative to the anchor point.

[0173] Intuitive user training and enhanced security through "zooming"

[0174] In one embodiment, the system is configured to help users easily learn to use the system authentication. Fig.13AAs shown, once registration or authentication begins as previously described, the system causes the user's mobile device 1310 to display a small oval 1320 on screen 1315 while the mobile device 1310 is imaged to the user. Instructions 1325 are displayed on screen 1315, instructing the user to hold the mobile device 1310 so that his or her face or head appears within the oval 1320. Since the oval 1320 is small, the user needs to keep the mobile device 1310 away from his or her body, such as straightening his or her arms while holding the mobile device 1310. The maximum arm length and face size are unique to the user. In another embodiment, the arm may not be completely straight to adjust the operation when space is not available, such as in a car or in a crowded location. Note that when the small oval 1320 is displayed in the middle of the display, it can be located anywhere on the screen 1315.

[0175] Then, if Fig. 13B As shown, the system causes the user's mobile device 1310 to display a larger oval 1330 on the display 1315. The display 1315 may also display corresponding instructions 1335 directing the user to "zoom in" on his or her face so as to fill the oval 1330 with his or her face. The user may select a substantially straight line to the user's face (e.g., Fig. 7A and 7B 1300) by bringing the mobile device 1310 close to his or her face until the user's face fills or exceeds the oval 1330.

[0176] In this way, the system provides and teaches the user a simple method to provide enrollment and authentication images and enrollment and authentication movements. The system can also teach different enrollment and authentication movements through different positions of the small ellipse 1320 on the screen 1315 and by changing the order and size of the displayed ellipses. For example, the user can zoom to 1 / 2 size, then zoom out by moving the mobile device, and then zoom in to full screen. The system can be configured to detect whether the zoom function of the camera (when equipped) is used, which usually requires the user to touch the screen.

[0177] In one embodiment, the registration movement can be omitted and the authentication movement can be compared to the movement expected based on the prompt on the screen. For example, the device or authentication server generates a series of ellipses of different sizes, in which the user must place his or her face by moving the mobile device held in the user's hand. In this way, the authentication movement can be different during each login, depending on the order, size and position of the ellipses displayed on the screen.

[0178] When using Fig.13A and 13BThe system may also include other safety features when the "zoom in" movement is shown. A typical camera on a mobile device or any other device includes a lens. This causes a "fish-eye" effect in the final image captured by the camera. In some instances, the curvature is not obvious to the human eye, or may only be noticeable at a certain focal length. The curvature or fish-eye effect may vary with focal length or the distance between the user and the lens. The extent of the fish-eye effect depends on the type of optical system used by the camera lens and other factors.

[0179] The fisheye effect in an image of a person's face is more pronounced when a person is close to the camera to image his or her face. This effect causes the relative size of the person's face to appear different than when the person is farther from the camera. For example, a person's nose may be as much as 30% wider and 15% taller relative to the person's face when the image is captured very close than when the image is captured at a distance. The relatively large difference in focal distances of the different facial features causes the relative sizes to differ when the person is imaged close to the camera compared to when the person is imaged at a greater distance from the camera - when the different facial features have relatively the same focal distance.

[0180] This difference has been found to be important to many facial video algorithms. That is, a facial recognition algorithm may not consider a live person imaged very close to be the same person as a live person imaged at a greater distance. In contrast, if a 2D photo of a person is imaged by a camera at a close distance and at a greater distance, the change in relative focal length between the lens and the 2D image is less noticeable. Therefore, a facial recognition algorithm may consider the 2D photo to be the same person when imaged at a close distance from the lens and at a greater distance.

[0181] This effect can be used to improve the security of cognitive systems. For example, during enrollment, the user can provide enrollment images at close and far distances from the camera, in addition to other positions provided by movement. Later, during authentication, authentication images can be obtained at distances close to the camera and at far distances to determine whether they match the enrollment information obtained from the enrollment images. Further, due to the fisheye effect expected when a real, three-dimensional person is present, the absence of relative changes in the size of facial features will alert the system to fraudulent attempts in authentication. This effect is difficult to recreate using two-dimensional pictures (printed photos or screens) and therefore, this step can act as a security test to prevent the use of two-dimensional pictures (instead of a living person's face) for authentication.

[0182] In other words, using this "zooming in" and "zooming out" motion on the user's face, two or more biometric profiles can be created for the same person. One of the multiple profiles of the person may be imaged farther from the camera, and one of the multiple profiles of the person may be imaged closer to the camera. In order for the system to authenticate a person, the authentication image and biometrics must match two or more of the profiles in the enrollment image and biometrics.

[0183] In addition, by comparing the background of images obtained at closer or farther distances, the system can detect the presence of a real person as compared to an unreal photo of a person. When the mobile device 1310 is held so that the person's face fits within the oval 1320, objects in the background almost directly behind the person may be visible. However, when the mobile device 1310 is held so that the person's face fits within the larger oval 1330, the person's face obstructs the camera's ability to see the same objects almost directly behind the person. Therefore, the system can compare the background of images obtained at closer or farther distances to determine whether a real person is attempting to authenticate with the system.

[0184] Of course, in Fig.13A and 13B In the example, shapes or guides other than ellipses 1320 and 1330 may be used to guide the user to hold the mobile device 1310 at an appropriate distance from his or her face. For example, the mobile device 1310 may display a full or partial square or rectangular frame. Further, the system may vary the size and position of frames such as ellipses 1320, 1330 to further increase security. For example, the system may require a medium-sized frame, a small frame, and then a large frame. As another example, the system may require a small frame at a first position and a second position, and then a large frame. This may be done randomly in order to teach the user different registration and authentication movements.

[0185] For a single user, the number of frame sizes presented to the user may also vary based on the results of other security features described herein. For example, if the GPS coordinates of the mobile device show that the device is in an unexpected location, more frames at different distances may be required for authentication. One or more indicators such as lights, words, or symbols may be presented on the screen so as to be visible to the user, thereby directing the user to the desired distance that the mobile device should be from the user.

[0186] exist Fig.13A and 13B, the system can predict the expected fisheye distortion of the image based on the mobile device used for enrollment and authentication, and based on known and trusted enrollment data. Additionally or alternatively, known specifications of a given model of mobile phone camera can be utilized to predict the expected distortion of a person's facial features at different distances from the lens. Therefore, authentication must be device-dependent. Further, enrollment information from the user at every possible distance from the camera is not required.

[0187] For example, as described above, an enrollment image and biometrics of a user may be acquired at two distances from the user. During authentication, multiple images are captured in addition to the images corresponding to the closer and farther distances of the enrollment image and biometrics. Even if only two enrollment profiles are acquired, based on the expected distortion of these intermediate images based on the distances the device has traveled, the system can verify that changes in distortion of the images are occurring at the correct rate.

[0188] The capture of these images may be still images or video, such that frames or images are extracted from a video captured during the movement from a first position away from the user to a second position closer to the user. Thus, it is contemplated that the operation captures a large number of frames during the zoom movement and, based on data from accelerometers, magnetometers, etc., ensures that distortion occurs at the correct rate due to the size of the head and the distance the mobile device is moved.

[0189] Over time, based on accumulated data or data calculated during the design phase, the system will have data that indicates if the phone is moved a certain distance towards a person's face, the distortion effect should fall within a known percentage of the final distortion level or initial distortion level. Thus, in order to fool or spoof the authentication system disclosed herein, an attempt at spoofing would require not only distorting the spoofed two-dimensional picture image, but also cropping the background and then making a video of the face, distortion, and background all gradually implemented at the correct rate, but all without any banding from the video screen, and without having any screen edges visible, which is highly unlikely.

[0190] Many currently known face detection and face recognition algorithms are configured to look for small faces within an image. Therefore, in order to ensure that the face detection and face recognition algorithms detect and recognize faces in the enlarged image ( Fig. 13B ), the system adds a large buffer around images taken at a close distance. This creates a larger overall image and allows current facial detection and recognition algorithms to detect and recognize faces even if the user's face is larger in the original image.

[0191] When using Fig.13A and 13BThe eye tracking security features described above may also be enhanced when the registration and authentication moves produced by the described process. For example, a QR code, random shape, barcode, color, text, number, or any other visible indicator may be displayed on the screen when the user is instructed to bring the mobile device 1310 close to his or her face to fill in the ellipse 1330. At this close distance, the reflection of the displayed indicator from the user's eyes or face may be more easily imaged by the camera. Moreover, eye movements, blinking, etc. that determine that the person being imaged is "alive" may also be more easily obtained at close range.

[0192] In one embodiment, at least one blink is required to prove aliveness in order to authenticate. In another embodiment, blinks may be counted over time during authentication and the number of blinks may be averaged. This allows an additional factor in authentication to be the number of blinks observed during the action. By observing the way the user blinks during the action, the system can verify in future authentication attempts that the user blinks at the expected time and device position during the action.

[0193] In other embodiments, the size or position of the ellipse or frame may be changed to be different from Fig.13A , 13B The size or position shown so that the user must adjust the position and / or angle of the phone to place his or her face within the oval. This creates another way to ensure that the user is alive.

[0194] In an exemplary method, a mobile device is located at a first distance from a user and a first image is captured for processing. The distance can be linear away from the user, and in this embodiment is not an arc or track. This can be achieved by the user moving the mobile device (either by hand, or by the mobile device being on a movable device or upper track system). Alternatively, the lens system is adjusted in a fixed system to change the size of the user's face relative to the frame size. Alternatively, the user can remain still, multiple cameras can be used, or the camera can be moved without the user moving. Once some form of movement (from the device, camera, lens, or user) has been achieved to place the camera at a second distance, a second image is captured for processing. The movement from the first position to the second position can be directly toward the user. Both images are processed.

[0195] The processing may include calculations to verify differences between the two images, or differences in biometric measurements obtained from the two images, which indicate that a real person is being imaged. Processing may be performed to compare the first authentication image to the first enrollment image (corresponding to the first distance) to determine if there is a match, and then compare the second authentication image to the second enrollment image (corresponding to the second distance) to determine if there is a match. If there is a match, authentication may continue.

[0196] Variations of these methods may also be made possible by a system that requires a match at a first distance, but does not match at a second distance, thereby indicating that the second image is not a two-dimensional picture. The processing that results in a match or a mismatch may be any type of image or facial recognition processing algorithm. With other processing described herein, the processing may be performed on a mobile device, one or more remote servers, or any combination of such devices.

[0197] All processing described herein can be performed on the mobile device only, the remote server only, or a combination thereof. The biometric data can be stored on the mobile device or the server, or split between the two for security purposes. For example, an image can be processed on the mobile device but compared to enrollment data in the cloud or remote server. Alternatively, the image can be sent to the cloud (remote server) for processing and comparison.

[0198] Touch screen enhancement

[0199] Additional added security modifications may include information about the user's fingers. Many mobile devices with touch screens can detect the location and approximate size of a user's touch on the screen. Therefore, the approximate size of the user's finger or thumb can be measured by the system. In addition to the size of the finger, the orientation angle of the finger or whether the finger or thumb of the right hand or the finger or thumb of the left hand is used can be detected.

[0200] In one embodiment, the user selects an account to open, starts enrollment imaging, or starts authentication imaging by touching the touch screen of the user device. The authentication system can thus detect whether the touch made by the user during authentication corresponds to previously stored enrollment information including the size of the user's finger or thumb, the amount of pressure applied to the screen, and whether the user is using the right or left hand. This adds an extra layer of security to the authentication system.

[0201] Furthermore, the authentication system may require the user to initiate authentication by touching the fingerprint reader or touch screen in one or more predetermined ways. Fig.14 As shown, the touch screen 1410 may be divided into predetermined areas 1420. For example, nine equal circular, square or other shaped areas 1420 may be provided on the touch screen 1410 of the mobile device. During registration, the user selects one of the areas 1420 of the screen 1410 to touch to initiate authentication. During authentication, if the preselected area 1420 is not touched to start authentication or during the entire authentication process, authentication is rejected. This is just one possible design possibility, and other design options may be conceived.

[0202] Area 1420 on the touch screen may be visually represented by a grid, or may not be displayed at all on the touch screen 1410. Fig.15As shown, in addition to area 1420 or in place of area 1420, buttons 1520 may also be displayed on touch screen 1510. Here, the user may initiate authentication by pressing one or more of buttons 1520 in a predetermined manner. The user may also initiate authentication by a predetermined card swipe method. The location to be touched by the user may change with each authentication attempt and may be communicated to the user by any indication from the authentication server, such as a code, number, letter, color, verification code, or other indicator.

[0203] Voice parameters

[0204] It is also contemplated that the user could record their voice by speaking a phrase while recording their image during the enrollment process when first using the system. Then, for authentication, the user also speaks the phrase while moving the mobile device to capture an image of their face. Thus, an additional path parameter could be the voice uttered by the user, with the use of voice recognition serving as another layer or element of the authentication process.

[0205] Image quality assurance

[0206] The authentication system may also process the image received from the mobile device to determine whether the image is of sufficient quality. For example, the system may check for blurry images due to the image being out of focus or the camera lens being blurred by fingerprints, oil, etc. The system may warn the user that the image is of insufficient quality (or too bright or too dark) and instruct the user to adjust the focus, exposure, or other parameters, or clean the camera lens.

[0207] Autofocus

[0208] The authentication system may also utilize the autofocus feature of the mobile device camera when so equipped. For example, when a real, three-dimensional person is imaged, the system checks to ensure that the clarity of the image changes throughout the process of the camera performing autofocus. In another embodiment, the system may control the autofocus so that the camera focuses at a first position or distance to check the clarity (focus) of a portion of the image containing the face. The system then controls the camera to focus at a second position or distance where the presence of the face is not detected, and checks the clarity (focus) of a portion of the image. If a three-dimensional person in a real environment is being imaged, it is expected that the focus setting should be different at the first and second positions, suggesting that a real person is currently being imaged. However, if the focus is the same at both positions, this indicates that a two-dimensional photo or screen is being imaged, indicating a fraudulent login attempt.

[0209] The system may also control the device's autofocus to detect different focal distances of different specific features in the image. For example, when a person's face is imaged from the front, the person's ears are expected to be at a different focal distance (further away) than the tip of the person's nose.

[0210] Image of login attempt

[0211] The authentication server may be configured to store the authentication image for a predetermined length of time. The image may provide an additional security advantage as evidence of a person attempting to log into a user's account. For example, the system may store a predetermined number of previous attempted logins, such as twenty login attempts, or store images from login attempts within a predetermined time period, such as during the past seven days or weeks. Any spoofing or attempted spoofing will result in a picture of the person attempting to log in being stored or sent to the authentication server of the account server.

[0212] Knowing that a photo was taken and sent is a great deterrent to anyone who might be dishonest, as they know their picture is being taken and stored, which ensures the user's security. Likewise, any attempts or failed attempts can result in a photo being stored and indicating who tried to access the account. It is also conceivable that an email or text message could be sent to the authorized user along with a picture of the person who attempted a failed login so that they know who is trying to access their account. With the photo or image also being processed by the authentication server, this establishes a first line of security for the user's account.

[0213] Adaptive matching threshold

[0214] Further, the level or percentage of correspondence between the registration information and the authentication information of the authenticated user may change over time. In other words, the system may include an adaptive threshold.

[0215] After a user has used the above authentication system regularly, the user will have logged into the system a large number of times by moving the mobile device in a predetermined path relative to his or her head. Therefore, it can be expected that the user will gain experience in using the authentication system and that the user will gradually become accustomed to a comfortable and standardized movement path. In contrast, the user's initial registration movement will most likely be the most awkward and clumsy movement, as the user has little experience with the authentication system.

[0216] In order to make the authentication system more convenient for the user without sacrificing security, an adaptive threshold system allows the registration movement to adjust so that the user is not stuck with awkward and clumsy initial movements as registration movements. To make it more convenient, based on each successful authentication, the successful authentication movement is stored and the motion path is added to a list of acceptable motion paths. The list of acceptable motion paths may be limited to a predetermined number of paths. When a new successful authentication is completed and the list of acceptable motion paths is full, the older registered motion path is deleted and the latest registered motion path is stored in its place. Alternatively, the motion path stored in the list that is least similar to the other motion paths may be deleted. Thus, by storing the most similar or latest motion path, the registration movement can slowly adjust over time as the user becomes familiar with the system and gets used to a comfortable motion path for authentication.

[0217] In addition, other registration information can be adaptively changed in a similar manner as the user information. For example, successful authentication photos or biometric information can be stored as part of the registration information, and old registration information can be discarded over time. In this way, the authentication system can be convenient for users even over long periods of time as the user experiences aging, facial hair growth, different makeup styles, new glasses, or other subtle facial changes.

[0218] Determining how much change is allowed over time in the motion path or biometric information, or both, can be set by the entity requiring authentication in order to meet the security needs of the entity. The time or number of scans after the initial registration can be used to adjust the adaptive threshold. For example, in the first few days after registration, the threshold may be lower while the security threat is low and the difference in the path is likely to be higher. After several authentications or several days, the threshold may be increased. The threshold can be further set based on trend data of the motion path or biometric information. For example, the threshold can be looser in the direction that the data is trending, while having a tighter data tolerance relative to the trend.

[0219] The time aspect may also be added with the location information. For example, if a user manages and authenticates a transaction near his home, then attempts another transaction an hour later in a foreign country, the transaction may be rejected. Or the transaction may be rejected if the distance between the previous authentication location and the next authentication location is not travelable or impossible to travel within the amount of time between login or authentication attempts. For example, if a user authenticates in Denver, but attempts an hour later in New York, Russia, or Africa, then the first or second attempt is likely fraudulent because the user is likely unable to travel between the two locations within an hour.

[0220] Further, if the next transaction attempt is attempted at a more reasonable time and distance from the first transaction, the level of the corresponding threshold may be raised to provide increased security without automatically rejecting the transaction. Similarly, an altimeter may be used such that if the altitude determined by the mobile device is different from the altitude of the city in which the user reports being located, this may indicate a fraud attempt. Thus, an altitude or barometric pressure reading from a mobile device may be used to verify location and may be cross-referenced against GPS data, IP address or router location data or a user identification location.

[0221] Random image distortion

[0222] To provide an additional layer of security for the facial recognition authentication system, the system may use random image distortions. For example, upon registering with the system, a user may be assigned a random distortion algorithm. The distortion algorithm may include, for example, the following distortions to the image: widening or narrowing the face by a predetermined amount, adding or applying a predetermined shape to the user's face at a predetermined location. For example, the distortion may be a circle applied 100 pixels above the user's left eye.

[0223] By using a uniquely assigned distortion to an image from a user, the biometric data for that user will be unique to the account or device used by the user. That is, the registered biometric stored on the authentication server or mobile device will not only reflect the user's facial features, but also the uniquely assigned image distortion. Therefore, even if a correct, fraudulent representation of a person is used on a different device or via a different account, the authentication biometric provided will not fully correspond due to the difference or absence of the unique distortion. Therefore, overall security can be improved.

[0224] Security Layer

[0225] Note that each of the above embodiments, modifications and improvements can be used in any combination as needed to create multiple security layers for authentication. In addition, when more than one of the above improvements or modifications is combined, the authentication system can be configured not to provide any feedback or indication about which layer authentication failed.

[0226] For example, when a predetermined touch pattern to initiate authentication is combined with authentication movement and face authentication, the system does not indicate whether the touch pattern is incorrect, or the authentication movement or authentication image fails to correspond to the authentication information. Instead, the system provides the same authentication rejection regardless of what failure occurred. This is in the case where any number of the above security features are combined. In this way, it is difficult for a fraudster to detect which aspect of the fraudulent certificate must be corrected, further improving the security of the system.

[0227] Example Applications

[0228] Similarly, while financial account authentication is described herein, authentication using path parameters and image data may be implemented in any environment where verification of a user's identity is required before access is permitted (e.g., automated access, indoor access, computer access, website or data access, cell phone use, computer use, package receipt, event access, ticketing, court access, airport access, retail transactions, IoT access, or any other type of situation).

[0229] For example, an embodiment will be described below in which the above authentication system is used to securely perform retail transactions. In this embodiment, a user is registered with an authentication server as described above or an authentication application on a mobile device, and has generated registration information, including a registration image and / or biometrics, and a registration movement. In this example, the user initiates or attempts to complete a transaction at a retail store using a credit card, smart card, or using a smartphone with NFC capabilities.

[0230] The user begins paying for goods or services by tapping a credit card, smart card to make a transaction, or using an app on a smartphone with NFC capabilities. The retail store will then authorize the credit card or account to the relevant network of financial institutions ("gateways"). For example, the retail store, through a gateway (such as a gateway operated by VISA or AMERICAN EXPRESS), will determine whether the account is available and has sufficient available funds.

[0231] The gateway then communicates with the authorization server by authenticating the user identity to authorize the transaction. For example, the gateway may send an authorization request to the authentication server, and the authentication server then sends a notification, such as a push notification, to the user's mobile device to request the user to authenticate the transaction.

[0232] Upon receiving notification from the authentication server, such as by a vibration, beep, or other sound on the mobile device, the user can then authenticate his or her identity with the mobile device. The authentication server can also send information about the transaction to the user for verification by the user. For example, the authentication server can send information that causes the mobile device to display the vendor for the transaction, the vendor's location, and the purchase total.

[0233] Then, as before, when the user moves the mobile device to different orientations relative to the user's head, the user can hold the mobile device and obtain multiple authentication images. When moving the mobile device to obtain the authentication image, the mobile device also tracks the path parameters (authentication movement) of the mobile device via the gyroscope, magnetometer, and accelerometer to obtain the authentication movement of the device. The mobile device can then send device information, authentication images, and authentication movements to the authentication server. In other embodiments, the mobile device can process the image to obtain biometric data and send the biometric data to the server. In yet other embodiments, the mobile device can process the image, obtain authentication information, compare the authentication information with the registration information stored on the mobile device, and send the pass / fail result of the comparison to the authentication server.

[0234] Next, if the device information, authentication image and / or biometrics, and authentication movement correspond to the registered device information, registered image and / or biometrics, and registered movement, the authentication server can authenticate the user's identity and confirm that the user wishes to authorize transactions on his or her account. Next, the authentication server transmits an authorization message to the gateway. Once the gateway has received confirmation of authorization, the gateway can communicate with the retail store to allow the retail transaction.

[0235] When using the above system and method to authorize retail transactions, several advantages can be obtained. Because the user's identity verification and confirmation of the transaction are completed via the authentication system and mobile device, the user is no longer required to provide his or her credit card or signature, or enter a personal password (pin number) into the retailer's point of sale system. In addition, the retail store does not need to check the user's photo identification. The above method and system also has the following advantages: It provides a secure transaction that can work with mobile and online transactions without cameras such as security cameras in the store.

[0236] In the secure retail transaction described above, the user obtains the total amount due on his or her mobile device via the gateway and authentication server. However, in one embodiment, the mobile phone can use a camera such as a barcode, QR code or similar scanner to identify the items and the price of the items being purchased. The mobile device can then total the total amount due and act as a checkout station to complete the transaction with the retail store.

[0237] In another embodiment, a user of the application may wish to pay an individual or vendor anonymously. In this case, the user will specify the total amount to be paid into the application, and the application will create a unique identification transaction number. This number can then be shown to the second user, so the second user can enter the identification transaction number on the application on a separate device. The unique identification transaction number can also be sent from the user to the second user via NFC, Bluetooth, QR code or other suitable method. The second user can also enter the total amount and request payment.

[0238] Upon receiving the payment request and uniquely identifying transaction number, the authentication server may send a notification to the first user's mobile device to authenticate the transaction. The user will then verify his or her identity using the facial recognition authentication system described above. Alternatively or in addition, the user may verify his or her identity using other biometric data such as fingerprint or retinal scans, path-based motion and imaging, or the user may enter a password. After authentication, the user's device will send a request to the user's payment provider to request and authorize payment to the second user. In this way, payments can be completed securely while the users in the transaction are anonymous.

[0239] According to one embodiment, as an additional security measure, GPS information from the mobile device may also be sent to the authentication server to authenticate and allow the retail transaction. For example, the GPS coordinates from the mobile device may be compared to the coordinates of a retail store to confirm that the user is actually present at the retail store. In this way, a criminal who has stolen a credit card and is attempting to use the card from a remote location (compared to the retail location) cannot complete the transaction because the user's phone is not at the location of the retail store. The IP address may also be used to determine location.

[0240] As explained above, the level or percentage of correspondence between the registration information and the authentication information used to authenticate the user can also be adjusted based on the coordinates of the mobile device's GPS. For example, if the retail store and the coordinates of the mobile device's GPS are near the user's residence, the level of correspondence can be set at a lower threshold, such as at a 99% match rate. Alternatively, if, for example, the location is very far from the user's residence and is abroad, the level of correspondence can be set at a higher threshold, such as at a 99.999% match rate.

[0241] Although various embodiments of the present invention have been described, it will be apparent to those skilled in the art that more embodiments and implementations are possible within the scope of the present invention. In addition, the various features, elements and embodiments described herein may be claimed or combined in any combination or arrangement.

Claims

1. A method for authenticating a user via a computing device of the user having a camera, the method comprising during an authentication session: capturing at least one first image of the user taken using the camera of the computing device at a first distance from the user, and processing the at least one first image to obtain first biometric data from the at least one first image; Moving the camera from a first position to a second position or the user from the first position to the second position to change the distance between the user and the camera from the first position to the second position; capturing at least one second image of the user taken using the camera of the computing device at a second distance from the user, the second distance being different than the first distance, and processing the at least one second image to obtain second biometric data from the at least one second image; comparing the first biometric data to previously captured and stored first stored biometric data of the user, comparing the second biometric data to previously captured and stored second stored biometric data of the user, or comparing both the first biometric data and the second biometric data to the first stored biometric data and the second stored biometric data, respectively, to determine whether the first biometric data matches the first stored biometric data, whether the second biometric data matches the second stored biometric data, or whether both the first biometric data and the second biometric data match the first stored biometric data and the second stored biometric data, respectively; comparing the first biometric data to the second biometric data to determine whether the first biometric data matches the second biometric data; as well as Authenticate the user when: the first biometric data does not match the second biometric data due to a difference between the at least one first image and the at least one second image resulting from a change in distance between the user and the camera in three dimensions when the at least one first image and the at least one second image were captured; The first biometric data matches the first stored biometric data, the second biometric data matches the second stored biometric data, or both the first biometric data and the second biometric data match the first stored biometric data and the second stored biometric data, respectively.

2. The method according to claim 1, wherein: The computing device includes one of the following: a desktop computer, a laptop computer, a tablet computer, a smart phone, and a smart watch.

3. The method according to claim 1, wherein: Due to distortion of the user's facial features caused by the different distances between the camera lens and the user's face, the first biometric data should not match the second biometric data.

4. The method according to claim 1, wherein: The computing device is configured to display one or more prompts on a screen of the computing device to guide the user to establish the computing device at the first distance and to guide the user to establish the computing device at the second distance.

5. The method according to claim 4, wherein: The one or more prompts on the screen are at least a first ellipse having a first size and a second ellipse having a second size, the user's face is located within the first ellipse to capture the at least one first image, and the user's face is located within the second ellipse to capture the at least one second image.

6. The method according to claim 1, wherein: Biometric data includes data obtained from facial features of the user.

7. The method according to claim 1, wherein: The previously captured and stored first stored biometric data or the second stored biometric data of the user is biometric data obtained from one or more images of the user captured at the first distance, the second distance, or both during an enrollment session.

8. The method according to claim 1, further comprising: Displaying an image on a screen of the computing device while capturing the at least one first image, the at least one second image, or both, and processing the at least one first image, the at least one second image, or both to detect reflections of the displayed image of one or more eyes of the user.

9. The method according to claim 1, wherein: The comparison and authentication occurs at the remote server.

10. A method for authenticating a user via a computing device of the user having a camera, the method comprising during an authentication session: capturing at least one first image of the user taken using the camera of the computing device at a first distance from the user, and processing the at least one first image to obtain first biometric data based on a face of the user from the at least one first image; capturing at least one second image of the user taken with the camera of the computing device at a second distance from the user, the second distance being different than the first distance, and processing the at least one second image to obtain second biometric data based on the user's face from the at least one second image; comparing the first biometric data to previously captured and stored first stored biometric data of the user to determine whether the first biometric data matches the first stored biometric data; comparing the second biometric data to previously captured and stored second stored biometric data of the user to determine whether the second biometric data matches the second stored biometric data, the second stored biometric data being different than the first stored biometric data; as well as Authenticate the user when: the first biometric data matches the first stored biometric data, and the second biometric data matches the second stored biometric data; as well as Due to a change in the distance between the camera and the user when capturing the at least one first image and the at least one second image, the first biometric data does not match the second biometric data.

11. The method according to claim 10, wherein: The first biometric data does not match the second biometric data due to distortion of the user's facial features caused by the different distances of the camera's lens from the user's face during image capture.

12. The method according to claim 10, wherein: The computing device is configured to display one or more face frame prompts on a screen of the computing device to guide the user to capture the at least one first image at the first distance and to capture the at least one second image at the second distance.

13. The method of claim 10, wherein the previously captured and stored first stored biometric data or the second stored biometric data of the user is biometric data of the user captured during an enrollment session and obtained on one or more previously captured images at the first distance, the second distance, or both.

14. The method according to claim 10, wherein: The comparison and authentication occurs at the remote server.

15. An authentication system comprising: User computing devices, including: a processor configured to execute machine-readable instructions; a camera configured to capture an image; a transceiver configured to send data to and receive data from a remote server; a memory storing non-transitory machine-readable instructions executable by the processor, the machine-readable instructions configured to: capturing at least one first image of a face of the user when the camera is a first distance from the user; prompting the user to change the distance between the user's face and the camera to a second distance; capturing at least one second image of the user's face when the camera is the second distance from the user's face, such that the first distance is different from the second distance; generating first biometric data from the at least one first image; generating second biometric data from the at least one second image; sending the first biometric data and the second biometric data to one or more remote servers; One or more remote servers configured with a processor, a memory, and a transceiver such that the memory stores non-transitory machine-readable instructions executable by the processor for: receiving the first biometric data and the second biometric data from the user computing device; comparing the first biometric data with second biometric data for differences derived from three-dimensional faces in the at least one first image and the at least one second image captured at different distances from the user; determining that the three-dimensional human face is not present when the second biometric data has no difference compared to the first biometric data, the difference comprising at least a difference due to a change in relative distance between a facial feature of the user and the camera when the at least one first image is captured at the first distance and the at least one second image is captured at the second distance; and An authentication message is sent to the computing device with the results of the comparing and determining steps.

16. The authentication system according to claim 15, wherein: The computing device is one of the following: a laptop, a desktop, a smart phone, a smart watch, a tablet, or a computing device connected to a network.

17. The authentication system according to claim 15, wherein: The machine-readable instructions stored on the memory of the computing device are further configured to compare the first biometric data and the second biometric data to determine that a real person is being imaged.

18. The authentication system according to claim 15, wherein: The comparing occurs to determine that the first biometric data does not match the second biometric data due to predicted perspective distortion of facial features of the user caused by distance differences when the image of the user is captured at different distances from the lens of the camera.

19. A method for authenticating a user, comprising: capturing at least one first image of a face of a user when a camera of the computing device is a first distance from the face of the user; prompting the user to change the distance between the user and the camera to a second distance, the second distance being caused by movement of the camera or by the user bringing the camera closer to or farther away from the user's face; capturing at least one second image of the user's face when the camera is at the second distance from the user, such that the first distance is different from the second distance; generating first biometric data from the at least one first image; generating second biometric data from the at least one second image; sending the first biometric data and the second biometric data to one or more remote servers; One or more remote servers configured with a processor, a memory, and a transceiver such that the memory stores non-transitory machine-readable instructions executable by the processor for: receiving the first biometric data and the second biometric data from the user computing device; comparing the first biometric data with second biometric data for differences in three-dimensional faces in the at least one first image and the at least one second image captured when the camera is located at different distances from the user's face; determining that the three-dimensional human face is not present when the second biometric data has no difference compared to the first biometric data, the difference comprising at least a difference due to a change in relative distance between a facial feature of the user and the camera when the at least one first image is captured at the first distance and when the at least one second image is captured at the second distance; as well as An authentication message is sent to the computing device with the results of the comparing and determining steps.

20. The method according to claim 19, wherein: The computing device is one of the following: a laptop, a desktop, a smart phone, a smart watch, a tablet, or a computing device connected to a network.

21. The method of claim 19, further comprising comparing the first biometric data and the second biometric data to determine that a real person is being imaged.

22. The method according to claim 19, wherein: The comparing occurs to determine that the first biometric data does not match the second biometric data due to predicted perspective distortion of the user's facial features caused by distance differences when the image of the user's face is captured at different distances from the lens of the camera.

Images