Facial Recognition in Noisy Environments

By using transient image templates in computing devices, the accuracy of image recognition under adverse lighting conditions is solved, and the usability and security of certification are improved.

CN113039543BActive Publication Date: 2024-11-19MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN201980075238.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-11-14
Filing Date
2019-11-07
Publication Date
2024-11-19
Estimated Expiration
2039-11-07

AI Technical Summary

Technical Problem

Under adverse lighting conditions, it is difficult for computing devices to accurately recognize images, resulting in reduced authentication accuracy and practicality.

Method used

Using a transient image template, after the comparison of the captured image and the baseline image template fails, the image is stored as a transient template and is used for subsequent authentication in the same environment until the template is deleted in a predetermined condition (such as time or position change).

Benefits of technology

Improve image authentication accuracy under adverse lighting conditions, reduce dependence on auxiliary authentication methods, and enhance the security of computing devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113039543B_ABST
    Figure CN113039543B_ABST
Patent Text Reader

Abstract

When image-based authentication of the acquired image fails and the user of the computing device has provided user credentials that authorize the user to access the computing device, the computing device stores the acquired image as a transient image template. The transient image template can be used to further authorize the user to access the computing device for a predetermined time period. When the predetermined time period expires, the computing device deletes the transient image template. The computing device then reverts to using the baseline image template to perform image-based authentication on the user. The transient image template can also be associated with a location so that when the computing device is at or near the location, the computing device uses the transient image template for image-based authentication.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The subject matter disclosed herein relates generally to image-based authentication mechanisms, and more particularly to using transient image templates to perform image-based authentication in situations such as those caused by adverse lighting conditions. Background Art

[0002] Computing devices typically employ one or more mechanisms to authenticate a user to use the computing device. Examples of such mechanisms include personal identification numbers (e.g., PINs), alphanumeric passwords, biometric inputs (e.g., voice, fingerprint, etc.), and image recognition. Image recognition is often preferred over the use of alphanumeric passwords or PINs because the user is not required to remember the alphanumeric password or PIN and can access the computing device simply by having the computing device capture his or her image. Image recognition also provides a hands-free and voice-free method by which a user can authenticate him or herself to a computing device. The recognized image may include a face, iris, cornea, or other body part.

[0003] Typically, a computing device performs image recognition using a baseline image template. The computing device may use one or more image processing techniques to compare the baseline image template to a captured image of a user attempting to authenticate himself or herself with the computing device. If the computing device determines that the captured image is similar to the baseline image template, the computing device may then authorize the user to access the computing device.

[0004] However, in some cases, the computing device may determine that the captured image is not similar to the baseline image template even though the user is in fact authorized to access the computing device. As an example, changing lighting conditions may affect the captured image, making it difficult for the computing device to determine whether the captured image is similar to the baseline image template. Further, changing lighting conditions may persist, making the image recognition mechanism a means of authenticating the user with reduced accuracy or utility. Summary of the invention

[0005] To address these and other problems that arise in the field of image-based authentication, the present disclosure provides one or more embodiments of a computing device that establishes a temporary or transient image template for authenticating a user in situations where the computing device is unable to authenticate the user using a captured image.

[0006] In one embodiment, a disclosed system includes a computer-readable storage device having computer-executable instructions stored thereon, and one or more hardware processors communicatively coupled to the computer-readable storage device. When executing the computer-executable instructions, the one or more hardware processors configure the system to acquire a first image using a computing device, and compare the first image with a first image template to determine whether a person corresponding to the first image is authorized to use the computing device. The one or more hardware processors also configure the system to display a request for user credentials to authenticate the user based on the comparison of the first image and the first image template, and receive the user credentials. The one or more hardware processors also configure the system to authorize the user to use the computing device based on determining the user credentials, store the first image as a second image template, wherein based on a predetermined condition, the second image template is used instead of the first image template to authorize the user, and acquire the second image using the computing device. In addition, the one or more hardware processors configure the system to compare the second image with the second image template based on the predetermined condition to determine whether the user is authorized to use the computing device, and grant access to the computing device based on the comparison of the second image and the second image template.

[0007] In another embodiment of the system, the system is further configured to evaluate a predetermined condition, delete the second image template in response to the evaluation of the predetermined condition, and designate the first image template for authorizing the user after deleting the second image template.

[0008] In another embodiment of the system, evaluating the predetermined condition is performed by establishing a timer indicating a duration that the second image template will be used to authenticate the user and determining whether the timer has expired, wherein deleting the second image template is responsive to determining that the timer has expired.

[0009] In another embodiment of the system, the system is further configured to associate the location with a second image template, and the evaluation of the predetermined condition includes determining whether the computing device is a predetermined distance from the location associated with the second image template, and deleting the second image template includes: deleting the second image template in response to determining that the computing device exceeds the predetermined distance to the location.

[0010] In yet another embodiment of the system, the location is associated with the second image template in response to the user being authenticated via user credentials.

[0011] In another embodiment of the system, the system is further configured to determine an ambient light level associated with the first image, and storing the first image as the second image template includes comparing the ambient light level to an ambient light level threshold and storing the first image as the second image template in response to determining that the ambient light level exceeds the ambient light level threshold.

[0012] In another embodiment of the system, the comparison of the first image with the first image template causes the image-based authentication to fail, and the user credentials authorize the user after the image-based authentication fails. Additionally, the system is further configured to increment a failure count value in response to the image-based authentication failure, determine whether the failure count value exceeds a failure count threshold, and replace the first image template with the second image template so that the first image template is no longer used for image-based authentication.

[0013] The present disclosure further provides a method for accessing a computer device, the method comprising: acquiring a first image using a computing device, comparing the first image with a first image template to determine whether a person corresponding to the first image is authorized to use the computing device, and displaying a request for user credentials to authenticate the user based on the comparison of the first image and the first image template. The method also includes: receiving user credentials, storing the first image as a second image template based on a determination that the user credentials authorize the user to use the computing device, wherein the second image template is used instead of the first image template to authorize the user based on a predetermined condition, and acquiring the second image using the computing device. The method further includes comparing the second image with the second image template to determine whether the user is authorized to use the computing device based on the predetermined condition, and granting access to the computing device based on the comparison of the second image with the second image template.

[0014] In another embodiment of the method, the method includes evaluating the predetermined condition, deleting the second image template in response to the evaluation of the predetermined condition, and after deleting the second image template, designating the first image template for use in authorizing a user.

[0015] In another embodiment of the method, evaluating the predetermined condition includes establishing a timer indicating a duration that the second image template is to be used to authenticate the user, and determining whether the timer has expired, wherein deleting the second image template is responsive to a determination that the timer has expired.

[0016] In another embodiment of the method, the method includes associating a location with a second image template, and evaluating the predetermined condition includes determining whether the computing device is a predetermined distance from the location associated with the second image template, and deleting the second image template includes: deleting the second image template in response to determining that the computing device exceeds the predetermined distance to the location.

[0017] In yet another embodiment of the method, in response to the user authenticating via user credentials, the location is associated with the second image template.

[0018] In another embodiment of the method, the method includes determining an ambient light level associated with the first image, wherein storing the first image as a second image template further includes comparing the ambient light level to an ambient light level threshold, and in response to determining that the ambient light level exceeds the ambient light level threshold, storing the first image as the second image template.

[0019] In another embodiment of the method, the comparison of the first image with the first image template causes the image-based authentication to fail, and the user credentials authorize the user after the image-based authentication fails. In addition, the method includes: in response to the image-based authentication failure, incrementing a failure count value; determining whether the failure count value exceeds a failure count threshold, and replacing the first image template with the second image template so that the first image template is no longer used for image-based authentication.

[0020] The present disclosure further provides a system for accessing a computer device, wherein the system includes a unit for acquiring a first image using a computing device, a unit for comparing the first image with a first image template to determine whether a person corresponding to the first image is authorized to use the computing device, and a unit for displaying a request for user credentials to authenticate a user based on the comparison of the first image and the first image template. The system further includes a unit for receiving the user credentials, a unit for storing the first image as a second image template based on a determination that the user credentials authorize the user to use the computing device, wherein the second image template is used instead of the first image template to authorize the user based on a predetermined condition, and a unit for acquiring the second image using a computing device. The system also includes a unit for comparing the second image with the second image template to determine whether the user is authorized to use the computing device based on a predetermined condition, and a unit for granting access to the computing device based on the comparison of the second image with the second image template.

[0021] In another embodiment of the system, the system further comprises: a unit for evaluating a predetermined condition; a unit for deleting the second image template in response to the evaluation of the predetermined condition; and a unit for designating the first image template for the authorized user after deleting the second image template.

[0022] In another embodiment of the system, the unit for evaluating the predetermined condition performs the evaluation by establishing a timer indicating the duration that the second image template will be used to authenticate the user, and determining whether the timer has expired. In addition, the unit for deleting the second image template deletes the second image template in response to determining that the timer has expired.

[0023] In another embodiment of the system, the system further includes a unit for associating a location with a second image template; and a unit for evaluating a predetermined condition by determining whether the computing device is a predetermined distance from the location associated with the second image template, and a unit for deleting the second image template by deleting the second image template in response to determining that the computing device exceeds the predetermined distance to the location.

[0024] In another embodiment of the system, the system also includes a unit for determining an ambient light level associated with the first image; wherein the unit for storing the first image as a second image template stores the first image as the second image template by comparing the ambient light level to an ambient light level threshold and storing the first image as the second image template in response to determining that the ambient light level exceeds the ambient light level threshold.

[0025] In another embodiment of the system, the comparison of the first image with the first image template causes the image-based authentication to fail, and the user credentials authorize the user after the image-based authentication fails. Additionally, the system includes means for incrementing a failure count value in response to the image-based authentication failure, means for determining whether the failure count value exceeds a failure count threshold, and means for replacing the first image template with the second image template so that the first image template is no longer used for image-based authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Some embodiments are shown by way of example and not limitation in the figures of the accompanying drawings.

[0027] Figure 1 is a block diagram illustrating a computing device in communication with an image processing server for authenticating a user using image recognition technology according to an example embodiment.

[0028] Figure 2 Shows Figure 1 Example embodiments of applications and data for a computing device.

[0029] Figure 3A-3B According to an example embodiment, a method for Figure 1 Computing equipment certification Figure 1 A method for generating a transient image template based on an image acquired by a user.

[0030] Figure 4A-4B A method for using one or more transient image templates to generate a Figure 1 A method for authenticating a user of a computing device.

[0031] Figure 5A-5BA method for generating a transient image template from an acquired image is shown, wherein the transient image template is to be associated with a corresponding authentication location, according to an example embodiment.

[0032] Figure 6A-6B A method for using one or more transient image templates and a certified GPS location to Figure 1 A method for authenticating a user of a computing device.

[0033] Figure 7A-7B A method for modifying one or more baseline image templates when a predetermined number of transient image templates have been generated according to an example embodiment is shown.

[0034] Figure 8 According to an example embodiment, Figure 1 A method for implementing a timer in a transient image template of a computing device.

[0035] Fig. 9 is a block diagram illustrating components of a machine capable of reading instructions from a machine-readable medium (eg, a machine-readable storage medium or a machine-readable storage device) and performing any one or more operations of the methods discussed herein, according to some example embodiments. DETAILED DESCRIPTION

[0036] The following description describes the system, method, technology, instruction sequence and computing machine program product of the example embodiment of the present theme. In the following description, for the purpose of explanation, many specific details are set forth in order to provide an understanding of the various embodiments of the present theme. However, it will be apparent to those skilled in the art that the embodiments of the present theme can be put into practice without some or other of these specific details. Examples represent possible variations only. Unless otherwise clearly stated, structures (e.g., structural components, such as modules) are optional and can be combined or subdivided, and operations (e.g., in processes, algorithms or other functions) can be sequentially varied or combined or subdivided.

[0037] The present disclosure provides systems and methods for performing image-based authentication on a user requesting access to a computing device. In one embodiment, a computing device captures an image of a user to authenticate the user, and compares the captured image to a baseline image template to determine whether the captured image corresponds to the baseline image template. In the event that the computing device determines that the captured image corresponds to the baseline image template, the computing device can authenticate the user. In the event that the computing device determines that the captured image does not correspond to the baseline image template, the computing device can first display a prompt or query to request the user to authenticate himself or herself using a second authentication mechanism (e.g., a PIN or pattern entered by the user).

[0038] In the event that the second authentication mechanism is successful (e.g., the user has authenticated him or herself), the computing device may then determine whether the ambient light in the captured environment of the device and / or the lighting conditions of the captured image are sufficient to capture an adequate image. In this regard, the computing device may compare the measured ambient light (or detected ambient light) to one or more ambient light thresholds, such as a lower ambient light threshold (indicating the likelihood of a darkened condition) or a higher ambient light threshold (indicating the likelihood of a brightened condition).

[0039] In the event that the computing device determines that the lighting conditions caused a failure in the authentication process, the computing device may store the captured image and use the captured image as a transient image template for comparison to future captured images. In this regard, it is assumed that the computing device is used in an environment where suboptimal lighting conditions persist, and the transient image template will facilitate authentication of the user based on images captured in the same environment.

[0040] Thus, the present disclosure provides technical improvements to the field of image-based authentication because the use of transient image templates increases the likelihood that a user will be able to access their computing device when a subsequently captured image exhibits similar qualities (e.g., poor lighting conditions, blurriness, partial obscuration, etc.) as the initial captured image on which the transient image template is based. However, as a security precaution, the computing device may delete the transient image template, remove the transient image template, or otherwise make the transient image template unavailable after a predetermined period of time has passed. Transient image templates may be deleted from a computing device because the overall security of the computing device decreases as the number of transient image templates increases. Thus, in addition to the technical benefits to the field of image-based authentication, the present invention also provides technical benefits to computer security and authentication because the deletion of transient image templates reduces the likelihood that an unauthorized user will use a captured image that exhibits lighting conditions similar to the transient image template to gain access to a computing device.

[0041] In addition, and as referenced below Figures 6A-7B As discussed, the present disclosure further provides variations of the above-described embodiments, such as by extending the embodiments to cover geographic locations and / or modifying existing templates to accommodate assumed ongoing lighting conditions. Thus, further technical improvements to the functionality of computing devices are disclosed.

[0042] The disclosure now turns to various disclosed embodiments implementing aspects of the technology described herein. Figure 11 is a block diagram illustrating a computing device 106 in communication with an authentication server 110 for authenticating a user 104 using image recognition techniques according to an example embodiment. In one embodiment, the computing device 106 is configured to authenticate the user 104 by capturing an image of the user 104 and comparing the captured image to an image template. In another embodiment, the computing device 106 is configured to authenticate the user 104 by capturing an image of the user 104 and transmitting the captured image to the authentication server 110, which then performs a comparison of the captured image to the image template. The authentication server 110 then communicates the comparison result to the computing device 106.

[0043] The authentication server 110 provides server-side functionality to the computing device 106 via a network 108 (e.g., the Internet or a wide area network (WAN)). The computing device 106 may include a client application and / or module for interacting with the authentication server 110 and performing image-based authentication of the user 104. Examples of applications that may be instantiated by the computing device 106 include a network client, a single-purpose application (e.g., an "application"), a multi-purpose application (e.g., a programmatic client), or a combination thereof. The image processing server 106 may also be communicatively coupled to other computing devices (not shown), which may include applications and / or programs similar to the computing device 106.

[0044] The computing device 106 may include, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronic device, or any other communication device that the user 104 can use to perform various computing tasks (e.g., access the Internet, make phone calls, conduct video conferences, etc.). In some embodiments, the computing device 106 may include a display module (not shown) to display information (e.g., in the form of a user interface). In other embodiments, the computing device 106 may include one or more of a touch screen, an accelerometer, a gyroscope, a camera, a microphone, a global positioning system (GPS) device, etc. As described below with reference to Figure 2 As discussed in greater detail, computing device 106 uses its various sensors and hardware components to verify the identity of user 104 and to store transient images of user 104 based on the various lighting conditions in which user 104 has captured his or her image.

[0045] In one embodiment, authentication server 110 is a network-based device that is in bidirectional and / or unidirectional communication with computing device 106. In some cases, multiple users may have corresponding user accounts that may be authenticated by computing device 106 (e.g., where computing device 106 is a terminal of an enterprise network), and multiple users may request access to computing device 106 at different times, or simultaneously if computing device 106 implements a multi-tasking operating system. One or more users may be a person, machine, or other device that interacts with computing device 106. In various embodiments, a user is not part of networked system 102, but may interact with networked system 102 via computing device 106 or other device.

[0046] The computing device 106 may communicate with the authentication server 110 via one or more networks 108. Although the one or more networks 108 are illustrated as a single instance, one of ordinary skill in the art will appreciate that multiple networks and / or multiple types of networks may communicatively couple the computing device 106 with the authentication server 110. For example, one or more portions of the one or more networks 108 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network, a wireless wide area network (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of a public switched telephone network (PSTN), a cellular telephone network, a wireless network, a Wi-Fi network, a WiMAX network, another type of network, or a combination of two or more such networks.

[0047] The computing device 106 may include one or more applications (also referred to as "apps"), such as, but not limited to, a web browser, a messaging application, an electronic mail (email) application, a communication session application, and the like. In some embodiments, if the communication session application is stored as a local application on the computing device 106, the application is configured to locally provide a user interface and at least some functionality for communicating with the authentication server 110 (e.g., authenticating a user, establishing a communication channel between one or more computing devices and / or services, displaying or outputting audio and / or video, capturing audio and / or video using one or more input devices, etc.). In contrast, if the communication session client is not stored on the computing device 106, the computing device 106 may use its web browser to access services provided by the authentication server 110.

[0048] The user 104 may interact with the networked system 102 via the computing device 106 or other means. For example, the user 104 provides input (e.g., touch screen input or alphanumeric input) to the computing device 106, and the input is transmitted to the networked system 102 via the network 108. In this case, the authentication server 110, in response to receiving the input from the user 104 or the instruction from the computing device 106, transmits the information to the computing device 106 through the network 108 for presentation to the user 104. In this way, the computing device 106 facilitates the interaction between the user 104 and the authentication server 110.

[0049] In addition, despite Figure 1 The networked system 102 shown in FIG. 1 employs a client-server architecture, but the subject matter is certainly not limited to such an architecture and may, for example, well find application in distributed or peer-to-peer architecture systems.

[0050] The computing device 106 is configured to authenticate the user 104 using one or more image recognition technologies. In one embodiment, the computing device 106 instantiates a facial recognition application or other module that uses an image of the user 104 to determine whether the user 104 is authorized to access the computing device 106. For example, by using a camera or other image capture device of the computing device 106. The facial recognition application can utilize one or more image recognition libraries to perform facial recognition, such as the Open Source Computer Vision Library (OpenCV), which is an open source computer vision and machine learning software library. OpenCV contains interfaces for a variety of computing programming and scripting languages, such as C++, Python, Java, and MATLAB, and can be instantiated under various operating systems including Microsoft Windows, Linux, Google Android, and Mac OS. Additionally and / or alternatively, the facial recognition application can be implemented using Windows Hello, which is a biometric login system for computing operating systems and can be obtained from Microsoft Corporation located in Redmond, Washington.

[0051] Accordingly, computing device 106 may include one or more libraries from OpenCV, which are then accessed by applications instantiated by computing device 106. Additionally and / or alternatively, authentication server 110 may instantiate one or more libraries of OpenCV, and computing device 106 may transmit one or more images to authentication server 110 for authentication and / or further processing. Further, computing device 106 and / or authentication server 110 may instantiate one or more applications developed using Windows Hello.

[0052] In one embodiment, user 104 interacts with computing device 106 to establish a baseline image template of user 104. As will be understood by one of ordinary skill in the art, a baseline image template is an image template that a computing device uses to compare with other images and determine whether one or more images contain and / or are similar to the baseline image template. In this case, computing device 106 may use the baseline image template to authenticate user 104, wherein user 104 uses computing device 106 to acquire one or more images of user 104 and compare the acquired images to the baseline image template. Additionally and / or alternatively, authentication server 110 may receive one or more acquired images from computing device 106 and communicate instructions to computing device 106 regarding whether user 104 is authenticated. Thus, in this regard, one or more operations described as being performed by computing device 106 may alternatively be performed by authentication server 110.

[0053] The comparisons performed by computing device 106 include, but are not limited to, histogram matching, template matching, and / or feature matching. Where computing device 106 is configured to perform template matching, computing device 106 may be configured to employ various template matching techniques, including, but not limited to, sum of squared differences, normalized sum of squared differences, cross-correlation, normalized cross-correlation, correlation coefficient, and other such template matching techniques or combinations thereof.

[0054] In the case where computing device 106 employs feature matching, computing device 106 may determine various features from a baseline image template and store these determined features as baseline image features. Thus, when computing device 106 acquires an image of user 104 for authentication (e.g., by capturing an image of user 104 using a camera), computing device 106 may determine various features from the captured image (e.g., captured image features) and then compare the captured image features to the baseline image features. Types of feature matching that computing device 106 may perform include, but are not limited to, Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Robust Independent Elementary Features (BRIEF), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB).

[0055] One of the challenges of using image-based authentication is that the captured image of 104 may vary depending on the environment in which computing device 106 is used. For example, user 104 may be in an environment that is dark or has low lighting conditions, or user 104 may be in an environment that is too bright and has bright lighting conditions. In this case, the low light and bright light are relative to the light acquired by the camera of computing device 106. Thus, user 104 may perceive lighting conditions that he or she considers acceptable, but computing device 106 obtains an image of user 104 with measurable dark areas and / or overexposed areas. As another example, user 104 may not be able to properly focus the camera lens of computing device 106, in which case the resulting image may be blurry or out of focus.

[0056] In the event that computing device 106 acquires an image of user 104 that has problematic quality (e.g., underexposed areas, overexposed areas, blur, partial blur, etc.), computing device 106 and / or authentication server 110 may be unable to match the acquired image to a baseline image template (e.g., via one or more template matching techniques and / or one or more image feature matching techniques). Therefore, in the event that computing device 106 and / or authentication server 110 cannot authenticate user 104 using image-based authentication, computing device 106 may use an auxiliary means of authenticating user 104. In one embodiment, the auxiliary means includes displaying a prompt on a display of computing device 106 that requests user 104 to provide specific input. The specific input may include, but is not limited to, a PIN, a pattern, a selection of one or more images, or any other combination of inputs. The specific input may also include a non-tactile input, such as a voice recognition input.

[0057] Computing device 106 may then determine whether the secondary authentication means has authenticated user 106. In the event that user 106 has successfully authenticated him or herself using the secondary authentication means, computing device 106 may store a temporary or transient copy of the captured image of user 106 that was used by computing device 106 but did not successfully authenticate the user. Computing device 106 stores the temporary or transient copy based on the assumption that, because user 104 has authenticated him or herself using the secondary authentication means, the image quality (e.g., lighting conditions) of the captured image originally used to authenticate user 104 may be poor (e.g., the light of the captured image is below or above a predetermined ambient light threshold), which results in computing device 106 being unable to authenticate user 104 through image-based authentication. Additionally and / or alternatively, computing device 106 instructs authentication server 110 to store the captured image as a temporary or transient image.

[0058] The computing device 106 then uses the temporary or transient copy of the captured image to later authenticate the user 104. In one embodiment, the computing device 106 and / or the authentication server 110 establishes the temporary or transient copy of the captured image as a transient image template, through which the user 104 is later authenticated. In this embodiment, the computing device 106 and / or the authentication server 110 uses the transient image template to compare with later acquired images of the user 104, such as by template matching, histogram matching, and / or feature matching. In the case of feature matching, the computing device 106 may determine one or more features from the transient image template and store the determined features as transient features for comparison with the determined features of later acquired images of the user 104.

[0059] The transient image template is temporary or transient because computing device 106 and / or authentication server 110 may be programmed or configured with a predetermined time period that indicates how long computing device 106 and / or authentication server 110 should keep a temporary or transient copy of the captured image. In this regard, in the event that computing device 106 and / or authentication server 110 receives a request to authenticate user 104 using image-based authentication, and the predetermined time period for the transient image template has not expired, computing device 106 and / or authentication server 110 compares the acquired image of user 104 to the transient image template (e.g., via template matching, feature matching, etc.). In some cases, computing device 106 may compare the acquired image to both the baseline image template (e.g., in a first-pass authentication attempt) and the transient image template (e.g., in a second-pass authentication attempt if the first-pass authentication attempt fails).

[0060] There are multiple benefits to configuring the computing device 106 and / or the authentication server 110 to use a transient image template. The first benefit is that the transient image template solves the near-term problem of the user 104 being in an environment with poor lighting conditions. Although the computing device 106 and / or the authentication server 110 can be programmed or configured with a baseline image template pair of the user 104, an image of the user 104 acquired in a poor lighting environment may cause the computing device 106 and / or the authentication server 110 to fail authentication because the baseline image template may have a different light exposure level than the acquired image. Therefore, the transient image template acquired in the poor lighting environment is used as a better basis to compare it with later acquired images of the user 104 in the poor lighting environment. The foregoing features result in improved usability and human-computer interaction because they reduce the number of times the user authenticates himself using a secondary authentication means (e.g., the user's PIN). For example, in the case where the user regularly accesses his phone while in a specific location with specific lighting conditions, the computing device 106 only queries the user's PIN on the first failed attempt, and then "learns" from the failure so as not to query the user again using the PIN for the same lighting conditions.

[0061] Another benefit is that computing device 106 and / or authentication server 110 frees up storage space used by transient image templates. Since user 104 is likely to use computing device 106 in different environments, and the lighting conditions of the different environments are likely to vary, computing device 106 and / or authentication server 110 may store multiple transient image templates. Thus, by continuing to use computing device 106, an increased amount of storage space of computing device 106 and / or authentication storage device 110 becomes dedicated to storing transient image templates. Thus, by deleting transient image templates after a predetermined period of time has passed, computing device 106 and / or authentication server 110 frees up storage space for other purposes or to store additional transient image templates.

[0062] Another benefit of deleting the transient image template is the security it provides. As the number of transient image templates increases, the likelihood that an acquired image of user 104 matches one or more transient image templates also increases. Thus, the inherent security provided by using image-based authentication is reduced. Thus, by deleting the transient image template after the expiration of a predetermined time period, computing device 106 and / or authentication server 110 returns to its secure state of utilizing a baseline image template.

[0063] In addition to, or in lieu of, the aforementioned time-limited transient image templates, computing device 106 and / or authentication server 110 may be configured to assign specific locations to specific transient image templates. In one embodiment, and as described below with reference to Figure 2As discussed, computing device 106 may include a global positioning system (GPS) transceiver that computing device 106 uses to determine its location. When computing device 106 and / or authentication server 110 creates a transient image template from an acquired image of user 104, computing device 106 and / or authentication server 110 may associate a set of GPS coordinates with the transient image template. In the case where computing device 106 and / or authentication server 110 employs feature matching, computing device 106 and / or authentication server 110 may also associate the GPS coordinates with one or more transient image features. In this way, if user 104 moves to various locations, computing device 106 knows which transient image template and / or transient image features to use when comparing acquired images of user 104 based on the location of computing device 106.

[0064] In one embodiment where computing device 106 and / or authentication server 110 associates GPS coordinates with transient image templates and / or transient image features, computing device 106 and / or authentication server 110 may also establish a predetermined range in which the transient image templates and / or transient image features are referenced relative to their associated GPS coordinates. The predetermined range may vary from a few feet to a few hundred yards. In this manner, computing device 106 and / or authentication server 110 references the transient image templates and / or transient image templates associated with the stored set of GPS coordinates when the user is within a predetermined range of the stored set of GPS coordinates (e.g., one or more GPS coordinates associated with one or more transient image templates or transient image features). Thus, computing device 106 does not have to be in exactly the same location as computing device 106 when creating the transient image templates and / or transient image features to perform image-based authentication using the transient image templates and / or transient image features.

[0065] In addition to deleting transient image templates and / or transient image features based on the expiration of an assigned predetermined time period, the computing device 106 and / or the authentication server 110 may also delete transient image templates and / or transient image features based on location. In one example, the computing device 106 and / or the authentication server 110 may be configured to delete the transient image template and / or transient image features when the computing device 106 is outside a predetermined range associated with one or more GPS coordinates associated with the transient image template and / or transient image features. The predetermined range may be measured using one or more distance metrics, including but not limited to feet, yards, meters, kilometers, miles, and other such distance metrics. In one embodiment, the predetermined range may be configured by the user 104, such as by providing an input to indicate the predetermined range when creating a transient image template and / or transient image feature, which input will trigger the deletion of the transient image template and / or transient image feature. In another embodiment, the computing device 106 and / or the authentication server 110 is pre-configured with a predetermined range that causes the deletion of the transient image template and / or transient image feature.

[0066] In yet another embodiment of the disclosed computing device 106 and / or authentication server 110, the computing device 106 and / or authentication server 110 is configured to enhance and / or modify an existing baseline image template using one or more previously captured images. Additionally and / or alternatively, the computing device 106 and / or authentication server 110 establishes a new baseline image template rather than modifying an existing baseline image template. In summary, the process involves establishing a new baseline image template after a predetermined number of image-based authentication failures occur for the baseline image template. The assumption of the process is that the user 104 is in an environment where a captured image of the user 104 is incompatible with an existing baseline image template of the user 104, and the user 104 is unable to authenticate himself or herself using the captured image and the existing baseline image template.

[0067] In one embodiment, computing device 106 and / or authentication server 110 establishes a new baseline image template and / or baseline image features by determining the number of failures that have occurred for image-based authentication using a current baseline image template at a particular location and / or within a predetermined time period. In this regard, computing device 106 and / or authentication server 110 may maintain a global failure count for the current baseline image template and / or a specific failure count for a particular location at which the current baseline image template is used for image-based authentication. In the event that the failure count reaches or exceeds a configurable failure count threshold, the result signals computing device 106 and / or authentication server 110 that the current baseline image template is not applicable for the location at which image-based authentication is being performed. In one embodiment, computing device 106 and / or authentication server 110 may then select the acquired image and / or transient image template as a new baseline image template.

[0068] Thus, the present disclosure describes various embodiments for performing image-based authentication of a user 104 using transient image templates and / or transient image features. The present disclosure also describes embodiments in which a baseline image template and / or baseline image features are replaced and / or modified in response to determining that image-based authentication has failed after a predetermined number of images of the user 104 have been captured. Furthermore, any of the foregoing embodiments may be used in combination with one another; thus, the computing device 106 may be configured according to various combinations of the foregoing embodiments.

[0069] Figure 2 According to an example embodiment, Figure 1 , wherein the computing device 106 is configured according to one or more of the foregoing embodiments. Figure 2 As shown, and in one embodiment, the computing device 106 is a mobile device 106. The computing device 106 is configured with various sensors and / or transceivers 206-212 and includes various applications 240 and data 242 to facilitate image-based authentication of the user 104.

[0070] The various functional sensors, transceivers, and other hardware and / or software of computing device 106 and / or authentication server 110 may reside on a single device, or may be distributed across several computers in various arrangements. The various components of computing device 106 and / or authentication server 110 may access one or more databases, and each of the various components of computing device 106 and / or authentication server 110 may communicate with each other. Furthermore, although discussed in the singular, Figure 2 , but it should be understood that in other embodiments, multiple instances of a component may be employed.

[0071] In one embodiment, the computing device 106 includes one or more processors 202 that execute or implement one or more applications 240. The one or more processors 202 can be any type of commercially available processor, such as a processor available from Intel Corporation, Advanced Micro Devices, Texas Instruments, or other such processors. Further, the one or more processors may include one or more special-purpose processors, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The one or more processors may also include programmable logic or circuits that are temporarily configured by software to perform certain operations. Therefore, once configured by such software, the one or more processors become a specific machine (or a specific component of a machine) that is specially customized to perform the configured functions, rather than a general-purpose processor.

[0072] The computing device 106 may further include various computer-readable storage devices 204 and / or computer-readable media for storing applications 240 and / or data 242. The computer-readable storage device 204 includes one or more devices configured to temporarily or permanently store instructions and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., erasable programmable read-only memory (EEPROM)), and / or any suitable combination thereof. The term "computer-readable storage device" should be taken to include a single device or multiple devices (e.g., a centralized or distributed database, or associated caches and servers) capable of storing applications 240 and data 242. Thus, the computer-readable storage device 204 may be implemented as a single storage device or device, or alternatively and / or additionally as a "cloud-based" storage system or storage network including multiple storage devices or devices.

[0073] The computing device 106 may also include various sensors and / or transceivers to facilitate establishing one or more temporary template images and / or temporary template features. In one embodiment, the computing device 106 includes one or more image sensors 206 for acquiring one or more images of the user 104, and the computing device 106 may use the one or more images as one or more baseline image templates for determining one or more baseline image features, as one or more transient image templates for determining one or more transient image features, and / or for attempting authentication. Examples of image sensors 206 include, but are not limited to, active pixel sensors in semiconductor charge coupled devices (CCDs), complementary metal oxide semiconductors (CMOSs), and / or active pixel sensors in N-type metal-oxide semiconductors (NMOSs). The image sensor 206 may also include an infrared sensor 206, such as a forward-looking infrared (FLIR) sensor.

[0074] Additionally and / or alternatively, the computing device 106 may include a GPS transceiver 208. The GPS transceiver 208 is configured to obtain one or more GPS coordinates corresponding to the location of the computing device 106. As described below, upon successful authentication of the user using one or more secondary authentication means, the obtained GPS coordinates may be associated with one or more transient image templates and / or transient image features.

[0075] In addition, the computing device 106 may include one or more ambient light sensors 210. The one or more ambient light sensors 210 are configured to measure the ambient light of the environment in which the computing device 106 is located. For example, the one or more ambient light sensors 210 can provide a measurement of the ambient light of an image captured by the image sensor 206. Figure 3A-3B As shown, the ambient light measured by ambient light sensor 210 can be used as an indicator of whether the environment in which computing device 106 is located is dark or bright (e.g., whether the measurement value is below or above the corresponding ambient light threshold), which indicates that when comparing the acquired image to the baseline image template, the authentication failure is likely due to poor lighting conditions rather than other defects in the acquired image. In some examples, the ambient light sensor can be a function of image sensor 206.

[0076] Finally, the computing device 106 may include one or more wireless transceivers 212 for wirelessly communicating with one or more other devices and / or systems (e.g., the authentication server 110). In this case, the one or more wireless transceivers 212 include various transceivers that facilitate wireless communication, including but not limited to Wi-Fi transceivers (e.g., transceivers configured to communicate using IEEE 902.11a / b / g / n), The one or more wireless transceivers 212 may include a cellular radio (e.g., a radio configured to communicate using Long Term Evolution (LTE), Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), and other such wireless transceivers), or a combination thereof. In one embodiment, the one or more wireless transceivers 212 are configured to communicate data with the computing device 106, such as one or more acquired images and / or authentication input provided by the user 104.

[0077] refer to Figure 2 , the application 240 of the computing device 106 is configured to perform an image-based authentication of the user 104, and to grant and / or deny access to the computing device 106 based on the results of the image-based authentication. In addition, the application 240 is configured to store a transient or transient image template and / or transient image features in response to the image-based authentication process. In addition, the application 240 can be configured to modify and / or generate a replacement baseline image template and / or baseline image features in response to determining that a threshold number of captured images that are also associated with successfully authenticating the user 104 using the auxiliary authentication means have been obtained. In one embodiment, the application 240 is written in a computer programming and / or scripting language. Examples of such languages ​​include, but are not limited to, C, C++, C#, Java, JavaScript, Perl, Python, or any other computer programming and / or scripting language now known or later developed.

[0078] To perform these and other operations in order to facilitate these results, applications 240 include, but are not limited to, operating system 214, authentication application 216, and facial recognition application 218. Figure 2 Items 214-218 are referred to as "applications", but those of ordinary skill in the art will appreciate that these items 214-218 may also be implemented as sub-modules of independent applications, or may also be implemented as one or more software libraries that may be accessed by another application (e.g., operating system 214). Further, although computing device 106 may include alternative and / or additional modules or applications (e.g., a network application, a printing application, a software-implemented keyboard, etc.), such alternative and / or additional modules and / or applications are not relevant to the present disclosure and are omitted from discussion for the sake of brevity and readability.

[0079] The data 242 used and / or supporting the application 240 includes, but is not limited to, one or more acquired images 222, a current GPS location 224, one or more image templates 226, one or more temporary (or transient) image templates 228, and one or more authenticated GPS locations 230. In addition, the data 242 may include one or more determined image features 232, one or more associations 234 between the transient image templates 228 and the authenticated GPS locations 230, various timing data 236, and user credentials 238 for authenticating the user 104. As with the application 240, the data 242 may include alternative and / or additional data not relevant to the present disclosure, such as user credentials, operating system state, web browser data, etc., and discussion thereof is omitted herein for the sake of brevity and readability.

[0080] The operating system 214 is configured to execute and / or access one or more applications 240 and control the functionality of the various sensors and / or transceivers 206-212. The operating system 214 may be configured to operate the computing device 106 in various states, such as an unlocked state and a locked state, wherein the unlocked state allows the user 104 to access various functions of the operating system 214, and the locked state restricts the user 104 from accessing various functions of the operating system 214 and / or imposes restrictions on the operating mode of the computing device 106. Examples of operating systems that may be executed by the one or more processors 202 include, but are not limited to, Windows 10 Mobile, Google Android, Apple iOS, Blackberry Secure, and other such operating systems. In one embodiment, the operating system 214 utilizes the authentication application 216 to authenticate the user 104 as an authorized user of the computing device 104.

[0081] The authentication application 216 is configured to authenticate the user 104 using one or more authentication means. Examples of these authentication means include non-biometric methods, such as a username / password combination, a PIN, a secret pattern, or other input-based authentication means. The authentication application 216 may also perform biometric-based authentication, including but not limited to fingerprint recognition, image-based authentication, voice analysis recognition, and other means for biometric authentication.

[0082] In one embodiment, authentication application 216 is configured to perform image-based authentication of user 104 in response to a request from operating system 214 to grant user 104 access to computing device 106. For example, computing device 106 may be in a locked state and user 104 has requested to place computing device 106 in an unlocked state. Therefore, in order to unlock computing device 106, operating system 214 may request authentication application 216 to authenticate the user.

[0083] In one embodiment, the authentication application 216 authenticates the user 104 via image-based authentication. In this regard, the data 242 may include one or more image templates 226 corresponding to images of users authorized to access the computing device 106, such as one or more images of the user 104. Those skilled in the art will appreciate that an image template used in facial recognition is an image that a facial recognition program or module attempts to find within another image or a portion of another image. The image template 226 may be stored in one or more image formats, including, but not limited to, portable network graphics (PNG), bitmap (BMP), joint photographic experts group (JPEG / JPG), or other such image formats or combinations thereof.

[0084] Thus, the authentication application 216 may use the one or more image templates 226 to call and / or execute the facial recognition application 218 to perform facial recognition operations on the user 104. In one embodiment, the one or more image templates 226 are established during a registration operation in which one or more acquired images are assigned as one or more baseline image templates by which the user 104 is later authenticated 106 when the user 104 attempts to access the computing device. In this regard, the baseline image templates 226 are retained on the computing device 106 and / or the authentication server 110. However, as described below, there may be situations in which the computing device 106 and / or the authentication server 110 replaces the baseline image template 226.

[0085] To perform facial recognition operations, the facial recognition application 218 may invoke one or more image sensors 206 to acquire one or more images of the user 104, which are then stored as acquired images 222. In this embodiment, the facial recognition application 218 then performs one or more image template matching operations to attempt to match one or more image templates 226 to the acquired images 222. As discussed above, the template matching operations may include, but are not limited to, sum of squared differences, normalized sum of squared differences, cross-correlation, normalized cross-correlation, correlation coefficient, and other such template matching techniques or combinations thereof.

[0086] Additionally and / or alternatively, the facial recognition application 218 can perform feature-based matching using one or more image features 232 extracted and / or determined from the image template 226. The image features 232 to be matched can be stored as image features 232. As described above, the facial recognition application 218 can perform image feature matching operations including, but not limited to, SIFT, SURF, BRIEF, FAST, ORB, or a combination thereof.

[0087] As contemplated by the present disclosure, the disclosed subject matter addresses situations where an initial image-based authentication operation fails due to one or more environmental conditions, such as due to poor lighting conditions of the environment in which the captured image 222 was captured. Although image-based authentication performed by the authentication application 216 may succeed on a first attempt in certain environments, the presently disclosed subject matter is directed to subsequent attempts to unlock the computing device 106 in the event that the image-based authentication fails, particularly in situations where the lighting or other environmental conditions affect the final image of the user 104 captured by the computing device 104.

[0088] In one embodiment, the facial recognition application 218 acquires one or more images 222 of the user 104 in response to a request from the authentication application 216, such as when the user 104 has requested the operating system 214 to unlock the computing device 106. For example, the facial recognition application 218 may instruct the one or more image sensors 206 to acquire one or more images 222 of the user 104. Where the facial recognition application 218 is configured to perform a template matching operation, the facial recognition application 218 may perform one or more template matching operations using the acquired image 222 and the image template 226. Additionally and / or alternatively, where the facial recognition application 218 is configured to perform a feature matching operation, the facial recognition application 218 may determine one or more features from the acquired image 222 and compare the determined features to the image features 232 of the image template 226. The facial recognition application 218 may then notify the authentication application 216 whether a match exists between the acquired image 222 (or acquired image features) and the one or more image templates 226 (or image features 232). As one of ordinary skill in the art will appreciate, a match does not necessarily mean a 100% correspondence between the compared images 222-226 and / or compared features, but rather that the facial recognition application 218 has determined that there is a likelihood (eg, 90% or 95% probability) of a match.

[0089] In the event that the facial recognition application 218 is unable to determine whether a match exists (e.g., the facial recognition application 218 returns an execute call with an exception or error message), or the facial recognition application 218 determines that a match does not exist, the facial recognition application 218 notifies the authentication application 216 accordingly (e.g., by returning an execute call with a specific message, error value, and / or other indicator). The authentication application 216 can then prompt the user 104 via the display of the computing device 106 to authenticate himself or herself using a secondary authentication means (e.g., by providing a PIN, secret pattern, etc.).

[0090] In one embodiment, authentication application 216 references user credentials 238 to authenticate user 104 via a secondary authentication means. User credentials 238 may include, but are not limited to, a PIN, a secret pattern, a username and password, or other set of credentials that authenticate user 104 to computing device 106.

[0091] In the event that the user 104 successfully authenticates himself or herself to the authentication application 216, this signals to the authentication application 216 that the user 104 is authorized to access the computing device 106. In addition, it indicates that the acquired image 222 may be visually flawed (e.g., overexposed, underexposed, blurred, etc.), but the acquired image 222 may have been used to authenticate the user 104.

[0092] Thus, in one embodiment, the facial recognition application 218 is established with various thresholds, error tolerances, and / or tolerances to accommodate situations where the acquired image 222 should have authenticated the user 104, but the quality of the acquired image 222 prevents the facial recognition application 218 from confirming the authentication. In the event that the comparison of the acquired image 222 with the image template 226 indicates a failed match but within the error range, the acquired image 222 is temporarily stored. In the event that the user 104 is unsuccessfully authenticated via the acquired image 222, the user 104 subsequently authenticates himself or herself to the authentication application 216 via the secondary authentication means, the facial recognition application 218 stores one or more of the acquired images 222 as a transient image or transient image template 228. The acquired image 222 is stored as a transient image template 228 because the comparison of the acquired image 222 with the baseline image template 226 has indicated that the acquired image 222 is almost a sufficient match (e.g., because the comparison produces a value equal to or greater than the error range but less than the match threshold), which indicates that one potential factor that may affect the comparison may be the environment in which the acquired image 222 was acquired. Thus, by establishing the acquired image 222 as the transient image template 228, the facial recognition application 218 can reference the transient image template 228 in subsequent attempts to authenticate the user 104, assuming that subsequent images are acquired in the same environment (e.g., under the same environmental conditions) as the transient image template 228.

[0093] Recognizing that the user 104 may move the computing device 106 from one environment to another, the facial recognition application 218 of the present disclosure is configured to associate timing data 236 with one or more transient image templates 228. In one embodiment, the timing data 236 includes a timer with a predetermined time period, wherein the timer is associated with each transient image template 228. The predetermined time period may be measured in seconds, minutes, hours, days, or any other time increment. An example of a predetermined time period is five minutes.

[0094] During the life of the timer, the operating system 214 and / or the facial recognition application 218 decrements the timer. When the timer expires, the facial recognition application 218 deletes the corresponding transient image template 228. By deleting the transient image template 228 associated with the expired timer, the facial recognition application 218 preserves the security of the computing device 106 and reclaims any storage space that the transient image template may have been using.

[0095] In addition to or in lieu of associating timing data 236 with one or more transient image templates 228, computing device 106 may also be configured to associate one or more authenticated GPS locations 230 with one or more transient image templates 228. These associations may be stored as template / GPS associations 234. In some cases, the physical location of computing device 106 may affect the image quality of one or more acquired images 222. Typically, an authenticated GPS location is a location where user 104 has authenticated himself or herself to computing device 106 (e.g., via authentication application 216). Further, authentication application 216 may record GPS location 224 as an authenticated GPS location 230 in which user 104 is required to use a secondary authentication means in response to a failure of an image-based authentication mechanism.

[0096] As described above, the facial recognition application 218 can store one or more acquired images 222 as one or more transient image templates 228. In addition, where the authentication application 216 is configured to communicate with the GPS transceiver 208 and / or access the GPS location 224 stored in the data 242, the authentication application 216 and / or the facial recognition application 218 can establish an association 234 between the one or more transient image templates 228 and the GPS location 224 (which is then stored as the authenticated GPS location 230). In this manner, in the event that the user 104 stays at a particular location or revisits a particular location, the facial recognition application 218 and / or the authentication application 216 queries the authenticated GPS location 230 to determine if there is an existing authenticated GPS location 230 for the particular location (e.g., as indicated by the GPS location 224). In the event that there is an authenticated location 230 at the particular location, the facial recognition application 218 then queries the template / GPS association 234 to obtain the corresponding transient image template 228 associated with the authenticated location 230. The facial recognition application 218 then references the obtained transient image template to perform image-based authentication of the user 104 using one or more of the acquired images 222 .

[0097] Additionally, one or more of the transient image templates 228 may also be associated with the authenticated GPS location 230 and timing data 236 (e.g., an expiration timer). In this regard, in the event that the computing device 106 is located at or within a predetermined distance of the authenticated GPS location 230, the facial recognition application 218 and / or the authentication application 216 determines whether a corresponding transient image template 228 exists, and, if so, whether the timing data 236 indicates that the transient image template 228 has expired. In the event that the transient image template 228 has not expired, the facial recognition application 218 and / or the authentication application 216 subsequently references the transient image template when performing image-based authentication.

[0098] In this manner, computing device 106 is configured to determine which transient image template 228 to use based on the location of computing device 106. Because computing device 106 may include one or more transient image templates 228, the association with authorized GPS location 230 (e.g., via template / GPS association 234) allows computing device 106 to more easily determine which transient image template(s) 228 to use in performing image-based authentication of user 104. Thus, the use of authorized GPS location 230 and template / GPS association 234 represents an improvement over existing techniques for authenticating users.

[0099] Those of ordinary skill in the art will appreciate that GPS location 224 is not limited to only locations determined using GPS. In this regard, GPS location 224 may be any location determined by computing device 106 using one or more combinations of geolocation techniques, including but not limited to cellular triangulation (e.g., via wireless transceiver 212), Wi-Fi-based positioning systems (WiPS / WFPS), Internet Protocol (IP) address geolocation, and other such geolocation techniques. Thus, authenticated GPS location 230 is not limited to those locations determined using GPS, but also includes any location determined using one or more of the aforementioned geolocation techniques.

[0100] In some instances, it may be advisable and / or beneficial to modify and / or add one or more baseline image templates 226. Thus, in one embodiment, the authentication application 216 is configured to develop and / or modify the baseline image template 226 after a predetermined number of repeated failures of image-based authentication, after image-based authentication fails a predetermined number of times within a given time period, after image-based authentication fails a predetermined number of times at or near the same location, and / or after the computing device 106 has acquired a predetermined number of transient image templates 228 for a particular location. In these instances, the authentication application 216 may determine that the failure rate of image-based authentication reaches and / or exceeds an acceptable failure rate. For example, the computing device 106 may be programmed and / or configured with one or more failure rates corresponding to different types of potential failures (e.g., failures based on the number of attempts; failures based on the number of attempts within a predetermined time period, and failures based on multiple attempts at a particular location). These failures may occur in situations where the environment surrounding the computing device 106 makes it difficult for one or more image sensors 206 to capture sufficient and / or acceptable images of the user 104 for authentication.

[0101] To address this shortcoming, the authentication application 216 and / or facial recognition application 218 may record and / or monitor the number of image-based authentication failures that have occurred at a location or within a predetermined time period. In one embodiment, failures are recorded and / or monitored after a transient image template has been deleted or removed. The assumption of this approach is that the environment in which the computing device 106 is located is causing the computing device 106 to repeatedly transfer to one or more transient image templates 228, in which case it is reasonable for the computing device 106 to refer to the transient image template as a baseline image template. Therefore, in one embodiment, for the purpose of modifying or replacing the baseline image template 226, image-based authentication failures are recorded after the transient image template has been used and the transient image template has expired.

[0102] In one embodiment, the authentication application 216 and / or the facial recognition application 218 replaces one or more baseline image templates 226 with one or more transient image templates 228 after the number of image-based authentication failures has reached a predetermined threshold. In one embodiment, the failure threshold is time-based, such as three image-based authentication failures within a 20-minute period (e.g., a predetermined duration). In another embodiment, the failure threshold is location-based, such as five image-based authentication failures at the same location (or the approximate location provided by the GPS transceiver 208). In yet another embodiment, the failure threshold can be based on both time and location, such as five image-based authentication failures at the same location within a 20-minute period.

[0103] In this manner, the computing device 106 is configured to modify and / or replace one or more baseline image templates 226 with their transient counterparts after a failure threshold has been reached or exceeded. The method is technically beneficial because it improves the manner in which the computing device 106 performs facial recognition in environments that are problematic and / or result in failures of image-based authentication using previously stored baseline image templates 226. The method is further technically beneficial because it reduces the amount of storage space occupied by transient image templates and reduces the amount of computing resources required to compare acquired images to the baseline image templates 226 (e.g., the facial recognition application 218 compares the acquired image 222 to the baseline image template 226 without having to compare the acquired image 222 to various transient image templates 228).

[0104] Figure 3A-3B 2 shows a method 302 for generating a transient image template 228 from an acquired image 222 according to an example embodiment. The method 302 may be performed by Figure 2 One or more components shown are used to perform, with reference to Figure 2 Have a discussion.

[0105] First, refer to Figure 2 and Figure 3A , the computing device 106 receives an access request via the operating system 214 (operation 304). The access request may include a request to perform image-based authentication of the user 104. The operating system 214 and / or the authentication application 216 then initializes one or more image sensors 206 to acquire one or more images of the user 104 (operation 306). The image sensor 206 then acquires one or more images of the user 104 and stores the images as acquired images 222 (operation 308). In some embodiments, the authentication application 216 and / or the facial recognition application 218 then determines one or more image features from the acquired images 222 (operation 310). Although shown as an optional operation, it may be performed as Figure 3A Operation 310 may be performed alternatively and / or in addition to one or more operations shown in .

[0106] The facial recognition application 218 then compares the acquired image 222 to one or more baseline image templates 226 (operation 312). As described above, the image template matching operations may include, but are not limited to, sum of squared differences, normalized sum of squared differences, cross-correlation, normalized cross-correlation, correlation coefficient, and other such template matching techniques or combinations thereof. Additionally and / or alternatively, the facial recognition application 218 performs one or more feature matching operations (operation 312) using the baseline image features 232 and one or more features determined from the acquired image 222.

[0107] The facial recognition application 218 then provides an instruction or notification or communication to the authentication application 216 as to whether the one or more acquired images 222 match the baseline image template 226 (operation 314). In the event of a positive determination (e.g., the "yes" branch of operation 314), the method 302 proceeds to operation 316, in which the authentication application 216 confirms that the user 104 is authorized to access the computing device 106. Alternatively, in the event of a negative determination (e.g., the "no" branch of operation 314), the method 302 proceeds to operation 316. Figure 3B Operation 318.

[0108] Reference Figure 3B At operation 318, the computing device 106 displays a prompt to the user 104 via the operating system 214, requesting the user 104 to authenticate himself or herself using the second authentication means. For example, the operating system 214 may display a prompt requesting the user 104 to provide one or more user credentials, such as a PIN, a secret pattern, a user name and password, a fingerprint, or other such user credentials.

[0109] The provided user credentials are then compared to the previously stored user credentials 238 (operation 320). In the event that the authentication application 216 determines that the user 104 is not authenticated (e.g., the "no" branch of operation 320), the method 302 proceeds to operation 322. At operation 322, the authentication application 216 instructs the operating system 214 to deny the user 104 access to the computing device 106. The method 302 then ends, and the user 104 may be prompted to reauthenticate him or herself to the computing device 106.

[0110] Alternatively, in the event that the user 104 is authenticated (e.g., the "yes" branch of operation 320), the computing device 106 then performs several operations in preparation for storing one or more acquired images 222 as one or more transient image templates 228. First, the authentication application 216 and / or the facial recognition application 218 may determine whether one or more of the acquired images 222 have the necessary quality to be a transient image template. Otherwise, if the acquired images 222 do not have the necessary quality, the user is likely to be unable to authenticate himself or herself in a later attempt in which the low-quality acquired images 222 are transient image templates 228.

[0111] As described above, the authentication application 216 and / or the facial recognition application 218 determines various quality metrics of one or more of the acquired images 222 (operation 324). Examples of quality metrics include detected light levels (e.g., underexposure or overexposure), blurriness, whether one or more faces can be detected (e.g., object detection probability), and other such quality metrics or combinations thereof.

[0112] The authentication application 216 and / or the facial recognition application 218 then determines whether the quality metric values ​​determined and / or obtained for one or more of the acquired images are within a predetermined range of acceptable quality metric values ​​(e.g., equal to or greater than a lower threshold and equal to or less than an upper threshold) (operation 326). A range of quality metric values ​​is used because if one or more quality metric values ​​are too high (e.g., exposure values ​​are too high), the resulting image may be overexposed and the image of the user 104 may become blurry; similarly, if one or more quality metric values ​​are too low (e.g., object detection probability, detected ambient light level, etc.), the resulting image may be underexposed and / or darkened, and the image of the user 104 may not be determined by shadows or other dark markings. Therefore, in the event that the determination of operation 326 is made in the negative (e.g., the "no" branch of operation 326), the method 302 proceeds to operation 328. At operation 328, the acquired images with unacceptable ambient light values ​​are deleted or discarded.

[0113] Alternatively, where this determination is made in the affirmative (e.g., the "yes" branch of operation 326), the method 302 proceeds to operation 330, where the authentication application 216 and / or facial recognition application 218 generates a transient image template 228 based on the acquired image. Additionally and / or alternatively, the authentication application 216 and / or facial recognition application 218 may also determine one or more features from the acquired image that will be used in future attempts at image-based authentication of the user 104. The authentication application 216 and / or facial recognition application 218 then associates timing data 236 with the one or more transient image templates 228 and / or determined features, wherein the timing data 236 includes a timer indicating a duration that the transient image template 228 will remain on the computing device 106. As described above, upon expiration of the timer, the transient image template (or determined feature) associated with the timer is deleted (or otherwise removed) from the computing device 106.

[0114] Figure 4A-4B A method for using one or more transient image templates to generate a Figure 1 Method 402 for authenticating a user using a computing device. Method 402 may be performed by Figure 2 One or more components shown in Figure 2 Method 402 illustrates operations performed by computing device 106 step by step.

[0115] First, refer to Figure 2 and Figure 4A, the computing device 106 receives an access request via the operating system 214 (operation 404). The access request may include a request to perform image-based authentication of the user 104. The operating system 214 and / or the authentication application 216 then initializes one or more image sensors 206 to acquire one or more images of the user 104 (operation 406). The image sensor 206 then acquires one or more images of the user 104 and stores such images as acquired images 222 (operation 408). In some embodiments, the authentication application 216 and / or the facial recognition application 218 then determines one or more image features from the acquired images 222 (operation not shown).

[0116] The facial recognition application 218 then determines various quality metric values ​​for the acquired images 222 (operation 410). As previously explained, the method 402 involves a situation where a transient image template 228 already exists. Therefore, the facial recognition application 218 determines the quality metric values ​​for the acquired images 222 to ensure that a corresponding transient image template 228 exists for the quality metric values ​​so determined. In one embodiment, one or more transient image templates 228 are selected to match one or more acquired images 222 in which the quality metric values ​​of the one or more acquired images 222 are equal to the quality metric values ​​of the corresponding transient image templates 222 or are within a predetermined range of the quality metric values ​​of the corresponding transient image templates 222. Additionally and / or alternatively, the facial recognition application 218 may reject those acquired images 222 in which the quality metric values ​​of the acquired images 222 are outside the predetermined range of quality metric values ​​associated with the transient image templates 228.

[0117] In the event that the facial recognition application 218 determines that there is no transient image template 228 having a quality metric value similar to the quality metric value of the acquired image 222 (e.g., the “No” branch of operation 412), the method 402 proceeds to operation 414, in which the computing device 106 authenticates the user 104 using a secondary authentication means (e.g., Figure 3B Additionally and / or alternatively, the method 402 may proceed to operation 308 such that the computing device 106 may perform operations to establish the transient image template 228.

[0118] Alternatively, in the event that the facial recognition application 218 determines that there are corresponding one or more transient image templates 228 (e.g., the "yes" branch of operation 412), the method 402 proceeds to operation 414, in which the facial recognition application 218 performs the template matching and / or feature matching operations described above.

[0119] Reference Figure 4B, the facial recognition application 218 determines whether there is a matching transient image template 228 and / or transient features of the acquired image 222 and / or the determined image features of the acquired image (222). After the facial recognition application 218 determines that there is a match (or a match between the transient features and the determined features) between at least one transient image template 228 and at least one of the acquired images 222 (e.g., the "yes" branch of operation 416), the method 402 proceeds to operation 424, where the facial recognition application 218 indicates or notifies the authentication application 216 that the user 104 has been authenticated. Therefore, at operation 424, the user 104 is granted access to the computing device 106.

[0120] Alternatively, in the event that the facial recognition application 218 determines that at least one acquired image does not sufficiently match at least one of the one or more transient image templates 228 (e.g., the "no" branch of operation 416), the facial recognition application 218 notifies the authentication application 216 of the failure. Accordingly, the authentication application 216 may then instruct the operating system 214 to display a prompt requesting the user 104 to provide one or more user credentials. When the user credentials are provided, the authentication application 216 then determines whether the user 104 is authenticated (e.g., by comparing the provided credentials to the user credentials 238) (operation 420). In the event that the authentication application 216 determines that the user 104 is not authenticated (e.g., the "no" branch of operation 420), the method 402 proceeds to operation 422, in which the authentication application 216 and / or the operating system 214 deny the user 104 access to the computing device 106. In one embodiment, the computing device 106 then restarts the authentication and requests the user 104 to authenticate himself or herself to the computing device 106.

[0121] Alternatively, in the event that the authentication application 216 determines that the user 104 is authenticated (eg, the “yes” branch of operation 420 ), the method 402 proceeds to operation 424 , where the authentication application 216 indicates and / or notifies the operating system 214 that the user 104 is authorized to use the computing device 106 .

[0122] Figure 5A-5B 2 shows a method 502 for generating a transient image template 228 from an acquired image 222, wherein the transient image template 228 is to be associated with a corresponding authentication location, according to an example embodiment. The method 502 may be performed by Figure 2 One or more components shown are used to perform, with reference to Figure 2 Have a discussion.

[0123] First, refer to Figure 2 and Figure 5A, the computing device 106 receives an access request via the operating system 214 (operation 504). The access request may include a request to perform image-based authentication of the user 104. The operating system 214 and / or the authentication application 216 then initializes one or more image sensors 206 to acquire one or more images of the user 104 (operation 506). The image sensor 206 then acquires one or more images of the user 104 and stores such images as acquired images 222 (operation 508). In some embodiments, the authentication application 216 and / or the facial recognition application 218 then determines one or more image features from the acquired images 222 (operation 510). Although shown as an optional operation, it may be performed as an operation. Figure 5A Operation 510 may be performed alternatively and / or in addition to one or more operations shown in .

[0124] The facial recognition application 218 then compares the acquired image 222 to one or more baseline image templates 226 (operation 512). As described above, the image template matching operations may include, but are not limited to, sum of squared differences, normalized sum of squared differences, cross-correlation, normalized cross-correlation, correlation coefficient, and other such template matching techniques or combinations thereof. Additionally and / or alternatively, the facial recognition application 218 performs one or more feature matching operations (operation 512) using the baseline image features 232 and one or more features determined from the acquired image 222.

[0125] The facial recognition application 218 then notifies or communicates to the authentication application 216 an indication of whether the one or more acquired images 222 match the baseline image template 226 (operation 514). In the event that this determination is made in the affirmative (e.g., the "yes" branch of operation 514), the method 502 proceeds to operation 516, in which the authentication application 216 confirms that the user 104 is authorized to access the computing device 106. Alternatively, in the event that this determination is made in the negative (e.g., the "no" branch of operation 514), the method 302 proceeds to operation 516. Figure 5B Operation 518.

[0126] Reference Figure 2 5B, at operation 518, the computing device 106 displays a prompt to the user 104 via the operating system 214, requesting the user 104 to authenticate himself or herself using the second authentication means. For example, the operating system 214 may display a prompt requesting the user 104 to provide one or more user credentials, such as a PIN, a secret pattern, a user name and password, a fingerprint, or other such user credentials.

[0127] The provided user credentials are then compared to the previously stored user credentials 238 (operation 520). In the event that the authentication application 216 determines that the user 104 is not authenticated (e.g., the "no" branch of operation 520), the method 502 proceeds to operation 522. At operation 522, the authentication application 216 instructs the operating system 214 to deny the user 104 access to the computing device 106. The method 502 then ends, and the user 104 may be prompted to reauthenticate him or herself to the computing device 106.

[0128] Alternatively, in the event that the user 104 is authenticated (e.g., the "yes" branch of operation 520), the computing device 106 then performs several operations to prepare for storing the one or more acquired images 222 as one or more transient image templates 228. First, the authentication application 216 and / or the facial recognition application 218 may determine the location of the computing device 106 (operation 524). In one embodiment, the authentication application 216 and / or the facial recognition application 218 queries the GPS transceiver 208 to obtain the GPS location 224 of the computing device 106. However, and as previously described, the location of the computing device 106 may be determined according to other techniques, such as cellular triangulation, IP address geolocation, WiPS / WFPS, and other such techniques or combinations thereof.

[0129] When the user 104 has authenticated himself or herself, this indicates to the authentication application 216 and / or the facial recognition application 218 that the GPS location 224 (or location determined using other geolocation techniques) of the computing device 106 is a trusted location. Therefore, the authentication application 216 and / or the facial recognition application 218 can store the GPS location 224 as the authenticated GPS location 230.

[0130] In addition, and as referenced Figure 3A-3B As discussed, the facial recognition application 218 can determine whether the acquired image is sufficient to perform matching and / or comparison operations, including but not limited to determining the ambient light level in the acquired image, determining the blurriness of the acquired image, determining the color level in the acquired image (e.g., via histogram analysis), and other such determinations. Thus, in this manner, the facial recognition application 218 determines whether the acquired image 222 is of the necessary quality for performing image-based authentication.

[0131] Based on whether one or more of the acquired images 222 are sufficient to perform image-based authentication, the authentication application 216 and / or facial recognition application 218 may then generate a transient image template 228 and / or transient image features from the one or more acquired images 222 (operation 526). Thereafter, the authentication application 216 and / or facial recognition application 218 then establishes an association between the authenticated GPS location 230 and the transient image template 228, such as by storing the association in a two-dimensional table or other relational database (not shown). Additionally, and in a manner similar to method 302, the authentication application 216 and / or facial recognition application 218 then associates timing data 236 with the one or more transient image templates 228 and / or determined features, wherein the timing data 236 includes a timer indicating a duration that the transient image template 228 will remain on the computing device 106. As described above, upon expiration of the timer, the transient image template (or determined feature) associated with the timer is deleted (or otherwise removed) from the computing device 106.

[0132] Thus, in addition to including the timing data 236 in one or more transient image templates 228, the authentication application 216 and / or the facial recognition application 218 also associates the authenticated GPS location 230 with the one or more transient image templates 228. Thus, each transient image template 228 may be associated with an authenticated GPS location 230 and timing data 236. These associations further improve the functionality of the computing device 106 because they allow the computing device 106 to authenticate the user 104 based on time (e.g., whether a timer associated with the transient image template 228 has expired) and location (e.g., whether the computing device 106 is located at a location where the user 104 has previously been authenticated).

[0133] FIG. 6A to FIG. 6B A method 602 for authenticating a user 104 to a computing device 106 using one or more transient image templates 228 and an authenticated GPS location 230 is shown according to an example embodiment. The method 602 may be performed by Figure 2 One or more components shown in Figure 2 Have a discussion.

[0134] First, refer to Figure 2 and Fig. 6A, the computing device 106 receives an access request via the operating system 214 (operation 604). The access request may include a request to perform image-based authentication of the user 104. The operating system 214 and / or the authentication application 216 then initializes one or more image sensors 206 to acquire one or more images of the user 104 (operation 606). The image sensor 206 then acquires one or more images of the user 104 and stores such images as acquired images 222 (operation 608). In some embodiments, the authentication application 216 and / or the facial recognition application 218 then determines one or more image features from the acquired images 222 (operation not shown).

[0135] Then, the authentication application 216 and / or the facial recognition application 218 determines the location of the computing device 106 (operation 610). In one embodiment, the GPS transceiver 208 provides the GPS location 224 of the computing device 106. As previously described, the method 602 involves a situation where one or more transient image templates 228 already exist and are associated with one or more authenticated GPS locations 230; therefore, the authentication application 216 and / or the facial recognition application 218 obtains the location of the computing device 106 to determine whether there is a corresponding transient image template associated with a GPS location that is equal to or approximately equal to the GPS location 224 of the computing device 106.

[0136] Therefore, using the GPS location 224, the authentication application 216, and / or the facial recognition application 218, it is determined whether there is an authenticated GPS location 230 that is close to or near the GPS location 224. In one embodiment, the computing device 106 is programmed with a distance threshold that indicates a threshold distance at which the GPS location 224 should be located at or within the authenticated GPS location 230 to be considered at or near the authenticated GPS location 230. If the authentication application 216 and / or the facial recognition application 218 cannot identify the corresponding authenticated GPS location (e.g., the "No" branch of operation 612), the method 602 proceeds to operation 614. At operation 614, the computing device 106 authenticates the user 104 using an auxiliary authentication means (e.g., Figure 5B Additionally, and / or alternatively, the method 402 may proceed to operation 508 such that the computing device 106 may perform operations that establish the transient image template 228 and the association between the transient image template and the authenticated GPS location 230.

[0137] Alternatively, in the event that the authentication application 216 and / or the facial recognition application 218 determines that there are one or more authenticated GPS locations 230 corresponding to the GPS location 224 (e.g., the "yes" branch of operation 612), the method 602 proceeds to operation 614. At operation 614, the facial recognition application 218 references the template / GPS association 234 using the authenticated GPS location 230 determined at operation 612 to identify the corresponding transient image template 228. The facial recognition application 218 then uses the corresponding one or more transient image templates 228 to compare the corresponding one or more transient image templates to the one or more acquired images 222.

[0138] Reference Figure 6B , the facial recognition application 218 determines whether there is a matching transient image template 228 and / or transient features of the acquired image 222 and / or the determined image features of the acquired image 222 (operation 616). In the event that the facial recognition application 218 determines that there is a match between at least one transient image template 228 and at least one of the acquired images 222 (or a match between a transient feature and the determined features) (e.g., the "yes" branch of operation 616), the method 602 proceeds to operation 624, in which the facial recognition application 218 indicates or notifies the authentication application 216 that the user 104 has been authenticated. Therefore, at operation 624, the user 104 is granted access to the computing device 106.

[0139] Alternatively, in the event that the facial recognition application 218 determines that at least one acquired image does not sufficiently match at least one of the one or more transient image templates 228 (e.g., the “No” branch of operation 616), the facial recognition application 218 notifies the authentication application 216 of the failure. Accordingly, the authentication application 216 may then instruct the operating system 214 to display a prompt requesting the user 104 to provide one or more user credentials. When provided with the user credentials, the authentication application 216 then determines whether the user 104 is authenticated (e.g., by comparing the provided credentials to the user credentials 238) (operation 620). In the event that the authentication application 216 determines that the user 104 is not authenticated (e.g., the “No” branch of operation 620), the method 602 proceeds to operation 622, in which the authentication application 216 and / or the operating system 214 deny the user 104 access to the computing device 106. In one embodiment, the computing device 106 then restarts the authentication and requests the user 104 to authenticate himself or herself to the computing device 106.

[0140] Alternatively, in the event that the authentication application 216 determines that the user 104 is authenticated (e.g., the “yes” branch of operation 620 ), the method 602 proceeds to operation 624 , in which the authentication application 216 indicates and / or notifies the operating system 214 that the user 104 is authorized to use the computing device 106 .

[0141] FIG. 7A to FIG. 7B 2 shows a method 702 for modifying one or more baseline image templates 226 when a predetermined number of transient image templates 228 have been generated, according to an example embodiment. The method 702 may be performed by Figure 2 One or more components are implemented, refer to Figure 2 Have a discussion.

[0142] The focus of method 702 is to determine whether there has been a previous image-based authentication failure within a predetermined time period and / or at a specific location. Each time an image-based authentication failure occurs, a failure count is increased, and when the failure count reaches or exceeds a failure count threshold, the authentication application 216 and / or the facial recognition application 218 then modifies the baseline image template and / or replaces the baseline image template with a transient image template.

[0143] To expedite the description of method 702, method 702 is shown at a point where the initial image-based facial recognition fails (operation 704). For clarity, the operations prior to operation 704 may correspond to Figure 3A-3B Operations 304-320 and / or Figure 5A-5B Operations 504-520 of (eg, user 104 has attempted to authenticate him or herself using the facial recognition methods disclosed herein).

[0144] refer to Fig. 7A and Figure 7B At operation 706, facial recognition application 218 determines whether user 104 is authorized to access computing device 106 by comparing the credentials provided by the user (e.g., provided at or about the time at which image-based facial recognition failed) with previously stored user credentials 238. In the event that the comparison fails and the credentials provided by the user do not match the previously stored user credentials 238 (e.g., the "no" branch of operation 706), facial recognition application 218 notifies and / or indicates the failure to authentication application 216. Accordingly, authentication application 216 may then instruct operation 214 to deny user 104 access to computing device 106 (operation 708).

[0145] Alternatively, in the event that the user 104 is authenticated (e.g., the "yes" branch of operation 706), the method 702 proceeds to operation 710, where the authentication application 216 and / or the facial recognition application 218 generates a transient image template from the captured image that caused the image-based authentication to fail.

[0146] The authentication application 216 and / or the facial recognition application 218 then increments the failure count. The failure count can be associated with a particular location (e.g., one or more authenticated GPS locations 230), or can be associated with a predetermined time period. In various cases, the failure count can be initialized to a zero value when a predetermined condition is met, such as when the computing device 106 arrives at a particular location after a predetermined time away from the particular location, after a second predetermined time period has passed without an image-based authentication failure occurring (e.g., indicating that the environment is favorable for image-based authentication), and / or as described below, when a transient image template is converted to a baseline image template. Depending on the manner in which the computing device 106 implements the failure count, each authenticated GPS location 230 can be associated with a failure count, where the failure count value is either zero or a non-zero value.

[0147] refer to Figure 7B , the authentication application 216 and / or the facial recognition application 218 then determines whether the failure count reaches or exceeds a predetermined failure count threshold (operation 718). In one embodiment, each of the one or more authenticated GPS locations 230 is associated with a corresponding failure count threshold, such that the failure count value of the authenticated GPS location is compared to its corresponding failure count threshold. The failure count threshold may be non-configurable and pre-configured for the computing device 106. Alternatively, the failure count threshold may be configurable by the user 104, such that the user 104 may provide input to establish a value for the failure count threshold, such as when the acquired image 222 is converted to the transient image template 228.

[0148] In the event that the failure count value does not meet or exceed the failure count threshold (e.g., the "no" branch of operation 718), method 702 terminates with respect to determining whether the baseline image template should be replaced or permanently supplemented. Alternatively, in the event that the failure count value does meet or exceed the failure count threshold (e.g., the "yes" branch of operation 718), method 702 proceeds to operation 720, where the baseline image template is replaced with a transient image template used in the image-based authentication process (operation 720). In some cases, replacing the baseline image template with the transient image template includes replacing a global or universal baseline image template with the transient image template. In other cases, replacing the baseline image template includes replacing a baseline image template associated with a specific location with the transient image template. In another example, the computing device 106 may maintain a predetermined number of transient image templates from previous image-based authentication failures, and, when the failure count value meets or exceeds the failure count threshold, replacing the baseline image template may include calculating an average image template from a set of transient image templates, and replacing the baseline image template with the average image template.

[0149] Having replaced the baseline image template with the transient image template (or average transient image template), the authentication application 216 and / or the facial recognition application 218 then resets the failure count (e.g., a global failure count or a failure count associated with a specific location). The method 702 then terminates with respect to determining whether the baseline image template should be replaced or modified.

[0150] Figure 8 A method 802 for updating timing data of one or more transient image templates according to an example embodiment is shown. The method 802 may be performed by Figure 2 One or more components shown in and referring to Figure 2 Have a discussion.

[0151] Initially, the authentication application 216 retrieves the timing data 236 associated with the transient image template 228 or one or more transient image features (operation 804). In one embodiment, the timing data 236 includes a timer that indicates the amount of time remaining before the authentication application 216 and / or the facial recognition application 216 deletes the transient image template 228 or the transient image feature from the computing device 106. Alternatively, the timer may indicate the amount of time that the transient image template 228 or the transient image feature has been resident on the computing device 106.

[0152] The authentication application 216 and / or facial recognition application 218 then determines whether the timing data 236 associated with the transient image template or one or more transient image features has expired (operation 806). In one embodiment, determining whether the timer has expired includes determining whether the timer has reached a zero value. Alternatively, determining whether the timer has expired includes comparing the timer to a predetermined timer threshold; if the timer reaches or exceeds the predetermined timer threshold, the authentication application 216 and / or facial recognition application 218 determines that the timer has expired.

[0153] If the authentication application 216 and / or the facial recognition application 218 determines that the timing data 236 does not indicate that it has expired (e.g., the "No" branch of operation 806), the method 802 proceeds to operation 808. At operation 808, the authentication application 216 and / or the facial recognition application 218 adjusts the timing data 236; Figure 8 , adjusting the timing data 236 may include decreasing the timing data 236 by a predetermined amount (e.g., one second). In another embodiment, adjusting the timing data 236 may include increasing the timing data 236 by a predetermined amount. Then, the timing data 236 associated with the transient image template and / or one or more transient image features is updated with the new decreasing (or increasing) timing value (operation 810). Then, the method 802 returns to operation 804, where the method begins again.

[0154] Returning to operation 806, where the authentication application 216 and / or facial recognition application 218 determines that the timing data 236 indicates that a timer associated with the transient image template and / or the transient image feature has expired (e.g., the "yes" branch of operation 806), the method 802 proceeds to operation 912. At operation 812, the authentication application 216 and / or facial recognition application 218 deletes or otherwise removes the transient image template or one or more transient image features from the computing device 106. The method 802 may then end for that particular timing data.

[0155] Although methods 302-802 are shown separately, those skilled in the art will appreciate that the operations of methods 302-802 may be reordered and / or rearranged in various combinations, with repeated or redundant operations omitted. Figure 8 The specific embodiments thereof are not shown, but may be arranged or combined in any logical or operational manner.

[0156] In this manner, the present disclosure provides various embodiments related to image-based authentication of user 104, and in particular, provides for establishing a transient image template after image-based authentication fails but user 104 has authenticated himself or herself using a second authentication means. By using transient image templates and / or transient image features, computing device 106 can perform image-based authentication in environments where image-based authentication consistently or typically fails due to baseline image templates and / or baseline image features. In this regard, the present disclosure provides a technical improvement over the manner in which image-based authentication is traditionally performed. As a result, the functionality and operation of computing device 106 is improved compared to conventionally implemented computing devices.

[0157] Modules, components, and logic

[0158] Certain embodiments are described herein as including a logic or multiple components, modules, or mechanisms. A module may constitute a software module (e.g., code embodied on a machine-readable medium or a machine-readable storage device) or a hardware module. A "hardware module" is a tangible unit that is capable of performing certain operations and may be configured or arranged in some physical manner. In various example embodiments, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or an application portion) to operate as a hardware module that performs certain operations described herein.

[0159] In some embodiments, the hardware module can be implemented mechanically, electronically, or any suitable combination thereof. For example, the hardware module may include a dedicated circuit or logic that is permanently configured to perform certain operations. For example, the hardware module may be a dedicated processor, such as an FPGA or ASIC. The hardware module may also include programmable logic or circuits that are temporarily configured by software to perform certain operations. For example, the hardware module may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware module will become a specific machine (or a specific component of a machine) tailored specifically for performing the configured functions, rather than a general-purpose processor. It will be appreciated that the decision to mechanically implement the hardware module in a dedicated and permanently configured circuit or in a temporarily configured circuit (e.g., configured by software) may be made from cost and time considerations.

[0160] Therefore, the phrase "hardware module" should be understood to include tangible entities, referring to entities that are physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in a certain manner or perform certain operations described herein. As used herein, a "hardware-implemented module" refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each hardware module does not need to be configured or instantiated at any one time instance. For example, in the case where a hardware module includes a general-purpose processor that is configured by software to be a special-purpose processor, the general-purpose processor can be configured as different special-purpose processors (e.g., including different hardware modules) at different times. Therefore, the software configures one or more specific processors accordingly, for example, to constitute a specific hardware module at one time instance and to constitute different hardware modules at different time instances.

[0161] Hardware modules can provide information to other hardware modules and receive information from other hardware modules. Therefore, the described hardware modules can be considered to be communicatively coupled. In the case of multiple hardware modules being present at the same time, communication can be achieved by signal transmission (e.g., through appropriate circuits and buses) between or among two or more hardware modules. In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, by storing and retrieving information in a memory structure accessible to multiple hardware modules. For example, a hardware module can perform an operation and store the output of the operation in a memory device to which it is communicatively coupled. Then, another hardware module can access the memory device at a later time to retrieve and process the stored output. The hardware module can also initiate communication with an input or output device and can operate on a resource (e.g., a collection of information).

[0162] The various operations of the example methods described herein may be performed, at least in part, by one or more processors that are temporarily configured (e.g., configured by software) or permanently configured to perform the relevant operations. Such processors, whether temporarily or permanently configured, may constitute processor-implemented modules that execute to perform one or more operations or functions described herein. As used herein, a "processor-implemented module" refers to a hardware module implemented using one or more processors.

[0163] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some operations of a method may be performed by one or more processors or processor-implemented modules. In addition, one or more processors may also support the execution of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some operations may be performed by a group of computers (as an example of a machine including a processor), which may be accessed via a network (e.g., the Internet) and one or more appropriate interfaces (e.g., APIs).

[0164] The execution of certain operations may be distributed among processors, not only residing within a single computer, but also deployed across multiple computers. In some example embodiments, the processor or processor-implemented modules may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented modules may be distributed across multiple geographic locations.

[0165] Machine and software architecture

[0166] In some embodiments, the combination is implemented in the context of a machine and associated software architecture. Figures 1 to 8 Described modules, methods, applications, etc. The following sections describe representative architectures suitable for use with the disclosed embodiments.

[0167] Software architectures are used in conjunction with hardware architectures to create devices and machines tailored for specific purposes. For example, a specific hardware architecture combined with a specific software architecture will create a mobile device, such as a mobile phone, a tablet device, etc. Slightly different hardware and software architectures may produce smart devices for the "Internet of Things", while another combination produces a server computer for use in a cloud computing architecture. Not all combinations of such software and hardware architectures are presented here, as those skilled in the art can readily understand how to implement the subject matter of the present invention in contexts different from the disclosure contained herein.

[0168] Machine structure and machine readable medium examples

[0169] Fig. 9 is a block diagram illustrating components of a machine 900 capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium or a machine-readable storage device) and executing any one or more of the methods discussed herein, according to some example embodiments. Specifically, Fig. 9A schematic diagram of a machine 900 is shown in the example form of a computer system in which instructions 916 (e.g., software, programs, applications, applet, applications, or other executable code) may be executed to cause the machine 900 to perform any one or more of the methodologies discussed herein. For example, the instructions 916 may cause the machine 900 to perform Figure 3A-8 Additionally or alternatively, instruction 916 may implement Figure 1 to Figure 2 One or more of the components of the computer system. The instructions 916 convert a general, unprogrammed machine 900 into a specific machine 900 that is programmed to perform the functions described and shown in the described manner. In alternative embodiments, the machine 900 operates as a standalone device, or can be coupled (e.g., networked) to other machines. In a network deployment, the machine 900 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 900 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a PDA, or any machine capable of sequentially or otherwise executing instructions 916 specifying the actions to be taken by the machine 900. In addition, although only a single machine 900 is shown, the term "machine" should also be taken to include a collection of machines 900 that execute instructions 916 individually or collectively to perform any one or more of the methods discussed herein.

[0170] The machine 900 may include a processor 910, memory / storage devices 930, and I / O components 950, which may be configured to communicate with each other, for example, via a bus 902. In an example embodiment, the processor 910 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 912 and a processor 914 that may execute instructions 916. The term "processor" is intended to include a multi-core processor that may include two or more independent processors (sometimes referred to as "cores") that may execute instructions 916 simultaneously. Although Fig. 9 Multiple processors 910 are shown, and machine 900 may include a single processor with a single core, a single processor with multiple cores (eg, a multi-core process), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0171] The memory / storage device 930 may include a memory 932, such as a main memory or other memory storage device, and a storage unit 936, both of which may be accessed by the processor 910 via the bus 902. The storage unit 936 and the memory 932 store instructions 916 that embody any one or more of the methodologies or functions described herein. During execution by the machine 900, the instructions 916 may also reside, in whole or in part, within the memory 932, within the storage unit 936, within at least one of the processors 910 (e.g., within a cache memory of the processor), or any suitable combination thereof. Thus, the memory 932, the storage unit 936, and the memory of the processor 910 are examples of machine-readable media.

[0172] As used herein, "machine-readable medium" includes machine-readable storage devices capable of temporarily or permanently storing instructions 916 and data, and may include, but are not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., erasable programmable read-only memory (EEPROM)), and / or any suitable combination thereof. The term "machine-readable medium" should be considered to include a single medium or multiple media (e.g., centralized or distributed databases or associated caches and servers) capable of storing instructions 916. The term "machine-readable medium" should also be understood to include any medium or combination of multiple media capable of storing instructions (e.g., instructions 916) executed by a machine (e.g., machine 900), such that when the instructions are executed by one or more processors (e.g., processor 910) of the machine 900, the machine 900 performs any one or more of the methods described herein. Therefore, "machine-readable medium" refers to a single storage device or device, as well as a "cloud-based" storage system or storage network including multiple storage devices or devices. The term "machine-readable medium" itself does not include signals.

[0173] Input / output (I / O) components 950 may include a variety of components to receive input, provide output, generate output, send information, exchange information, capture measurements, and the like. The specific I / O components 950 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone will likely include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It will be appreciated that I / O components 950 may be included in Figure 8950 are grouped according to their functions only to simplify the following discussion, and the grouping is by no means limiting. In various example embodiments, the I / O components 950 may include output components 952 and input components 954. The output components 952 may include visual components (e.g., displays such as plasma display panels (PDPs), light emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), other signal generators, and the like. The input components 954 may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric input, optical keyboards, or other alphanumeric input components), pointing-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing tools), tactile input components (e.g., physical buttons, touch screens or other tactile input components that provide the location and / or force of touch or touch gestures), audio input components (e.g., microphones), and the like.

[0174] In further example embodiments, the I / O component 950 may include a biometric component 956, a motion component 958, an environmental component 960, or a position component 962, among a variety of other components. For example, the biometric component 956 may include a component for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 958 may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), and the like. The environment component 960 may include, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor for detecting hazardous gas concentrations for safety or measuring atmospheric pollutants), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The location component 962 may include a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), and the like.

[0175] A variety of technologies may be used to implement communications. I / O components 950 may include a communication component 964 operable to couple machine 900 to network 980 or device 970 via coupling 982 and coupling 972, respectively. For example, communication component 964 may include a network interface component or other suitable device that interfaces with network 980. In further examples, communication component 964 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, Components (e.g. Low Energy), Components and other communication components that provide communication by other means. Device 970 can be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0176] In addition, the communication component 964 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 964 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as universal product codes (UPC) barcodes, multi-dimensional barcodes (e.g., Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF416, Ultra Code, UCCRSS-2D barcodes), and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be derived through the communication component 964, such as positioning via Internet Protocol (IP) geolocation, Signal triangulation for positioning, by detecting NFC beacon signals that may indicate a specific location, and more.

[0177] Transmission medium

[0178] In various example embodiments, one or more portions of network 980 may be an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, the Internet, a portion of the Internet, a portion of the PSTN, a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, 980 or a portion of network 980 may include a wireless or cellular network, and coupling 982 may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, coupling 982 may implement any of a variety of types of data transmission technologies, such as single carrier radio transmission technology (lxRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, including 3G third generation partnership project (3GPP), fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide microwave access interoperability (WiMAX), long term evolution (LTE) standards, and other standards defined by various standards development organizations, other remote protocols, or other data transmission technologies.

[0179] Instructions 916 may be sent or received over network 980 using a transmission medium via a network interface device (e.g., a network interface component included in communication component 964) and may utilize any of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 916 may be sent or received to device 970 using a transmission medium via coupling 972 (e.g., a peer-to-peer coupling). The term "transmission medium" shall be deemed to include any intangible medium capable of storing, encoding, or carrying instructions 916 for execution by machine 900, and includes digital or analog communication signals or other intangible media to facilitate communication of such software.

[0180] language

[0181] Throughout the specification, multiple instances may implement components, operations or structures described as single instances. Although the individual operations of one or more methods are shown and described as individual operations, one or more individual operations may be performed simultaneously, and the operations do not need to be performed in the order shown. The structures and functions represented as individual components in the example configuration may be implemented as combined structures or components. Similarly, the structures and functions presented as individual components may be implemented as individual components. These and other variations, modifications, additions and improvements fall within the scope of this article's subject matter.

[0182] Although an overview of the inventive subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of the embodiments of the present disclosure. For convenience only, the term "invention" may be used herein, individually or collectively, to refer to such embodiments of the inventive subject matter, and if more than one invention is actually disclosed, it is not intended to automatically limit the scope of the present application to any single disclosure or inventive concept.

[0183] The embodiments shown herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, so that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, the detailed description should not be understood in a limiting sense, and the scope of the various embodiments is limited only by the appended claims and the full scope of equivalents granted by these claims.

[0184] As used herein, the term "or" may be interpreted in an inclusive or exclusive sense. In addition, multiple instances may be provided for resources, operations, or structures described herein as single instances. In addition, the boundaries between various resources, operations, modules, engines, and data repositories are arbitrary to some extent, and specific operations are shown in the context of specific illustrative configurations. Other allocations of functions may be envisioned, and they may fall within the scope of various embodiments of the present disclosure. Typically, structures and functions represented as separate resources in the example configuration may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of embodiments of the present disclosure as represented by the appended claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for accessing a computer device, the method comprising: acquiring a first image using a computing device; comparing the first image to a first image template to determine whether the person corresponding to the first image is authorized to use the computing device; displaying a request for user credentials to authenticate the user based on a comparison of the first image to the first image template; receiving the user credentials; storing the first image as a second image template based on determining that the user credential authorizes the user to use the computing device; associating the second image template with a geographic location to enable authentication of the user using the second image template based on the geographic location; acquiring a second image using the computing device; determining that the computing device is within a threshold distance of the geographic location; comparing the second image to the second image template in response to determining that the computing device is within the threshold distance to determine whether the user is authorized to use the computing device; as well as Based on a comparison of the second image to the second image template, access to the computing device is granted.

2. The method according to claim 1, further comprising: assessing said geographical location; Responsive to the evaluation of the geographic location, deleting the second image template; as well as After deleting the second image template, the first image template is designated for authorizing the user.

3. The method according to claim 2, wherein: Deleting the second image template includes deleting the second image template in response to determining that the computing device is beyond the threshold distance of the geographic location.

4. The method according to claim 1, wherein: In response to the user being authenticated via the user credentials, the geographic location is associated with the second image template.

5. The method according to claim 1, further comprising: determining an ambient light level associated with the first image; And wherein storing the first image as the second image template further comprises: comparing the ambient light level to an ambient light level threshold; and In response to determining that the ambient light level exceeds the ambient light level threshold, the first image is stored as the second image template.

6. The method according to claim 1, wherein: A comparison of the first image with the first image template results in a failure of image-based authentication; as well as Upon failure of image-based authentication, the user credentials authorize the user; and the method further comprises: In response to a failure of the image-based authentication, incrementing a failure count value; determining whether the failure count value exceeds a failure count threshold; and The first image template is replaced with the second image template so that the first image template is no longer used for image-based authentication.

7. A system for accessing a computer device, the system comprising: a computer-readable storage device having computer-executable instructions stored thereon; as well as one or more hardware processors communicatively coupled to the computer-readable storage device that execute the computer-executable instructions to configure the system to: acquiring a first image using a computing device; comparing the first image to a first image template to determine whether the person corresponding to the first image is authorized to use the computing device; displaying a request for user credentials to authenticate the user based on a comparison of the first image to the first image template; receiving the user credentials; storing the first image as a second image template based on determining that the user credential authorizes the user to use the computing device; associating the second image template with a geographic location to enable authentication of the user using the second image template based on the geographic location; acquiring a second image using the computing device; determining that the computing device is within a threshold distance of the geographic location; comparing the second image to the second image template in response to determining that the computing device is within the threshold distance to determine whether the user is authorized to use the computing device; as well as Access to the computing device is granted based on a comparison of the second image to the second image template.

8. The system according to claim 7, wherein: The system is also configured to: assessing said geographical location; Responsive to the evaluation of the geographic location, deleting the second image template; as well as After deleting the second image template, specifying the first image template for authorizing the user.

9. The system of claim 8, wherein: Deleting the second image template includes deleting the second image template in response to determining that the computing device is beyond the threshold distance of the geographic location.

10. The system according to claim 9, wherein: In response to the user being authenticated via the user credentials, the geographic location is associated with the second image template.

11. The system according to claim 7, wherein: The system is also configured to: determining an ambient light level associated with the first image; and wherein storing the first image as the second image template comprises: comparing the ambient light level to an ambient light level threshold; and In response to determining that the ambient light level exceeds the ambient light level threshold, the first image is stored as the second image template.

12. The system of claim 7, wherein: Comparison of the first image to the first image template results in a failure of image-based authentication; and Upon failure of image-based authentication, the user credentials authorize the user; and the system is further configured to: In response to a failure of the image-based authentication, incrementing a failure count value; determining whether the failure count value exceeds a failure count threshold; and The first image template is replaced with the second image template so that the first image template is no longer used for image-based authentication.

13. A system for accessing a computer device, the system comprising: means for acquiring a first image using a computing device; means for comparing the first image to a first image template to determine whether the person corresponding to the first image is authorized to use the computing device; means for displaying a request for user credentials to authenticate the user based on a comparison of the first image to the first image template; means for receiving said user credentials; means for storing the first image as a second image template based on determining that the user credentials authorize the user to use the computing device; means for associating the second image template with a geographic location to enable use of the second image template to authenticate the user based on the geographic location; means for acquiring a second image using the computing device; means for determining that the computing device is within a threshold distance of the geographic location; means for comparing the second image to the second image template to determine whether the user is authorized to use the computing device in response to determining that the computing device is within the threshold distance; as well as Means for granting access to the computing device based on comparing the second image to the second image template.

Citation Information

Patent Citations

  • Method and device for securely sharing images across untrusted channels

    CN103765811A

  • Systems and methods to permit an attempt at authentication using one or more forms of authentication

    CN107800680A

  • Biometric recognition

    US20120257797A1

  • Electronic device and method for user identification

    US20130216108A1

  • System and method for providing secure access to an electronic device using multifactor authentication

    US20170063852A1