USE OF MULTIPERSPECTIV IMAGE SENSORS FOR THE AUTHENTICATION OF TOPOGRAPHIC FEATURES

Abstract

This document describes systems and techniques aimed at using multi-perspective image sensors for authenticating topographic features. A computing device (102), comprising image sensors (104) and a biometric authentication manager (108), is configured to capture images depicting the topographic features of an object from two or more perspectives. Based on the images (104), the biometric authentication manager (108) determines topographic features of the object and compares these determined topographic features with previously captured topographic features to determine whether the object and the previously imaged object are the same. In response to determining that the object and the previously imaged object are the same, the biometric authentication manager (108) modifies an authorization for a resource associated with the computing device (102).By implementing the techniques and systems described above, the biometric authentication manager (108) is suitable for repurposing the image sensors (104) as a biometric authentication scanner.

Description

GENERAL STATE OF THE ART

[0001] Dedicated biometric authentication systems (e.g., facial authentication systems, fingerprint authentication systems) are seamless, established methods for authenticating a user of a computing device (e.g., a smartphone, a laptop). These systems provide the user with secure, convenient user authentication, allowing the user to change a permission for a resource associated with the computing device (e.g., unlocking the device, unlocking an application on the device). However, implementing one of these dedicated biometric authentication systems in a computing device may require specialized hardware, space within a computing device enclosure, and specialized materials.

[0002] For example, a fingerprint authentication system that utilizes an optical under-display fingerprint scanner (UDFPS) may require a display material that becomes semi-transparent during fingerprint scanning or a transparent adhesive to bond the UDFPS to an underside of the display. In another example, allocating space for the UDFPS within a computing device enclosure may require enlarging the enclosure or shrinking the size of a computing device battery. This could compromise an intended product design (e.g., thinning a smartphone) for the computing device by increasing its physical size and weight, making it cumbersome for the user. It could also reduce the battery life of the computing device, making it unreliable for all-day use.As another example, a facial authentication system may utilize one or more cameras, a dot-matrix projector, and an infrared sensor for low-light functionality. These hardware modules may occupy space in a front panel of the computing device's chassis. If the manufacturer wishes to maintain a large display, the manufacturer may allocate space above the display, increasing the size and weight of the computing device and reducing the screen-to-body ratio. Alternatively, the manufacturer may allocate space in a notch of the display, but this would sacrifice a uniform, uninterrupted display. SUMMARY

[0003] This document describes systems and techniques directed toward using multi-perspective image sensors to authenticate topographical features. A computing device comprising image sensors and a biometric authentication manager is configured to capture images depicting topographical features of an object from two or more perspectives. Based on the images, the biometric authentication manager determines topographical features of the object and compares the determined topographical features to previously captured topographical features to determine whether the object and the previously imaged object are the same object. In response to determining that the object and the previously imaged object are the same object, the biometric authentication manager changes a credential for a resource associated with the computing device.

[0004] In aspects, a method for using multi-perspective image sensors to authenticate topographical features is disclosed. The method includes capturing one or more images, the one or more images depicting topographical features of an object from two or more perspectives, the one or more images captured by one or more image sensors, the one or more image sensors having two or more image capture perspectives. For example, the one or more image sensors may be integrated into a back of a housing of a computing device as part of one or more cameras.The two or more image capture perspectives may be possible by using one sensor with a suitable pixel count and pixel pitch, or two physically separated sensors configured to provide the two or more image capture perspectives. The method further includes determining, based on the one or more images, the topographical features of the object. In one example, the object may be a fingertip, a palm, or other suitable biometric identifier. The topographical features may be minutiae of a fingerprint, wrinkles in the palm, or other unique physical characteristics of another biometric identifier. The method further includes comparing the topographical features of the object to previously imaged topographical features of a previously imaged object to provide a comparison result.The method further includes determining, based on the comparison result and a threshold confidence, that the object and the previously mapped object are one and the same object. In response to determining that the object and the previously mapped object are the same object, the method includes changing a permission for a resource associated with a computing device. For example, changing the permission may include unlocking the computing device so that the user can access functions and features of an operating system of the computing device. Changing the permission may also include unlocking a resource or an application associated with the computing device, provided that the device is already in an unlocked state.

[0005] In additional aspects, a computing device is disclosed. The computing device includes one or more image sensors, one or more processors, and a memory. The memory stores instructions that, when executed by the one or more processors, cause the one or more processors to implement a biometric authentication manager responsible for repurposing the one or more image sensors as a biometric authentication scanner by performing the method described herein.

[0006] The details of one or more implementations are set forth in the accompanying drawings and the following detailed description. Other features and advantages will be apparent from the detailed description, the drawings, and the claims. This summary is provided to introduce subject matter further described in the detailed description. Accordingly, a reader should not consider the summary to describe essential features or to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The details of one or more aspects of converting one or more image sensors into fingerprint authentication scanners are described in this document with reference to the following drawings, in which the use of the same numbers in different cases may indicate similar features or components: Fig. 1 illustrates an example implementation of an example computing device having one or more image sensors, one or more processors, memory, and a biometric authentication manager; Fig. 2 illustrates an exemplary implementation of the exemplary computing device of Fig. 1, which is configured to use one or more image sensors as a biometric authentication scanner; Fig. 3 illustrates an exemplary implementation of the computing device of Fig. 2 in detail; Fig. Figure 4 illustrates an example implementation of a Bayer color filter array; Fig. 5 illustrates an example implementation of a dual pixel autofocus (DPAF) image sensor; Fig. 6 illustrates an example implementation of an example computing device configured to use one or more image sensors as a biometric authentication scanner; Fig. 7 illustrates an exemplary operating environment of a user of the computing device of Fig. 6; and Fig. Figure 8 illustrates a method for using image sensors as a biometric authentication scanner. DETAILED DESCRIPTIONOverview

[0008] Many computing devices (e.g., smartphones, tablets, personal computers) include a dedicated biometric authentication system. The dedicated biometric authentication system can be a facial authentication system, a fingerprint authentication system, and so on. Depending on the intended product design, a computing device manufacturer may prefer one method of dedicated biometric authentication over another. For example, if the intended product design involves a notchless display, the manufacturer may choose to integrate a fingerprint scanner into a front panel of a computing device housing below the notchless display.In another example, the manufacturer may choose to integrate an optical UDFPS beneath a portion of the notchless display in addition to the notchless display if the intended product design is to achieve the largest possible screen-to-body ratio of the computing device.

[0009] However, incorporating a dedicated biometric authentication system into the computing device may require specific hardware modules, space within the computing device's enclosure, and specialized materials. For example, a fingerprint authentication system utilizing an optical UDFPS may require a display material that becomes semi-transparent during fingerprint scanning or a transparent adhesive to bond the UDFPS to a bottom surface of the display. As an additional example, a facial authentication system may be designed to operate in low-light situations by integrating an infrared camera module into the front of the computing device's enclosure. The manufacturer may implement a dedicated biometric authentication system, such as the examples above, by integrating the necessary hardware modules or third-party technology.Alternatively, the manufacturer may develop first-party hardware modules or technologies to integrate them. The examples provided may increase the material costs for the computing device, which may ultimately increase the consumer price of the computing device for the user. Additionally, depending on the integrated hardware modules or technologies, an intended product design (e.g., thinness, notchless display, all-day battery life) for the computing device may be compromised.

[0010] Compromised product design, increased physical size and weight, shorter battery life, or increased consumer costs are just a few examples of how the inclusion of a dedicated biometric authentication system can lead to a poor user experience. This document describes systems and techniques aimed at using multi-perspective image sensors to authenticate topographical features, thus avoiding the compromised product design, increased physical size and weight, shorter battery life, or increased consumer costs often associated with the inclusion of a dedicated biometric authentication system.

[0011] The following describes operating environments, techniques that may be used in the operating environments, and example methods. While systems and techniques for converting one or more image sensors into fingerprint authentication scanners are described, it should be understood that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations, and reference is made to the operating environment merely by way of example. Example device

[0012] Fig. 1 illustrates an example implementation 100 of an example computing device 102 including one or more image sensors 104, a display 106, and a biometric authentication manager 108 configured to repurpose the image sensors 104 as a biometric authentication scanner. In this implementation, the computing device 102 is a budget smartphone that does not include a dedicated biometric authentication system. The image sensors 104 can be a variety of image sensors, including monochrome image sensors, Bayer filter image sensors, phase detection autofocus sensors, dual-pixel autofocus sensors, and so on. The image sensors 104 are integrated into a back of a housing of the computing device 102 as part of a camera. A camera lens assembly and a camera lens cover (e.g., glass, plastic) are located in front of the image sensors 104.The display 106 is integrated as a portion of a front side of the housing of the computing device 102.

[0013] In one example, a user 110 wishes to interact with the computing device 102. The user 110 picks up the computing device 102 in a locked state 112-1, as represented by the display 106-1. In response to the user 110 picking up the computing device 102 while in the locked state 112-1, the biometric authentication manager 108 displays a prompt 114 to place a finger on the camera lens cover. The user 110 places a finger on the camera lens cover, after which the biometric authentication manager 108 instructs the camera to capture one or more images using the image sensors 104. The one or more images, in this example, represent topographical features of a fingerprint of the finger.However, the object exhibiting topographical features may be any object suitable for biometric authentication, including a palm, a thumb, and so on. Based on the one or more images, the biometric authentication manager 108 determines the topographical features of the fingerprint. The topographical features may include, for example, minutiae of the fingerprint's peaks, valleys, and patterns.

[0014] In response to determining the topographical features of the fingerprint, the biometric authentication manager 108 compares the topographical features to previously captured topographical features of a previously imaged object to provide a comparison result. For example, the previously imaged object may be the same fingerprint of the user 110, images of which may have been captured during an initial setup process of the computing device 102. Based on the comparison result, the biometric authentication manager 108 determines that the fingerprint of the user 110 and the previously imaged object are the same object. In an example where a threshold confidence is not met, the biometric authentication manager 108 may display an error message detailing that the fingerprint and the previously imaged object are not the same object.

[0015] In response to determining that the fingerprint and the previously imaged object are the same fingerprint of user 110, biometric authentication manager 108 changes a permission (e.g., unlock) for a resource (e.g., an operating system) associated with computing device 102. By performing the method detailed above, biometric authentication manager 108 activates a biometric authentication scanner that user 110 can use to authenticate, changing the permission for the resource associated with computing device 102.

[0016] For example, without the biometric authentication scanner enabled by the biometric authentication manager 108, the user 110 may need to authenticate by other means, such as entering a password, PIN, pattern, or other non-biometric method. These alternative methods allow for errors, such as incorrectly entering a password, and security concerns, such as a malicious user obtaining and correctly entering the password. In many cases, incorrectly entering a password is simply an inconvenience for the user 110, who can simply re-enter the password, hopefully correctly. However, if the user 110 incorrectly enters the password multiple times (e.g., 10, 20 times), depending on the operating system of the computing device 102, the operating system may prevent the user 110 from unlocking the computing device 102 for a period of time (e.g., seconds, minutes, hours).This can be frustrating for user 110. On the other hand, security concerns are of paramount importance to many computing device users. For example, assume that user 110 previously opened an application containing sensitive personal information (e.g., financial information, health information) using computing device 102, and that this application is still open on computing device 102. If the malicious user gains access to computing device 102, the malicious user could obtain the sensitive personal information and use it to negatively impact user 110 (e.g., steal their identity, steal their money). This is not only a serious concern for user 110, but also another example that harms the user's experience.However, these and other concerns are mitigated for the user 110 of the computing device 102 having the biometric authentication manager 108 configured to repurpose the one or more image sensors 104 as a biometric authentication scanner.

[0017] Illustrated in detail Fig. 2 an exemplary implementation 200 of the computing device 102 of Fig. 1, configured to use the one or more image sensors 104 as a biometric authentication scanner (e.g., repurposing existing cameras). The computing device 102 is illustrated as a variety of example devices, including consumer electronics devices. As non-limiting examples, the computing device 102 may be a smartphone 102-1, a tablet 102-2, a laptop computer 102-3, a desktop all-in-one computer 102-4, a smartwatch 102-5, smart glasses 102-6, a gaming controller 102-7, a smart home speaker 102-8, and a microwave oven 102-9. Although not shown, the computing device 102 may also be an ATM, an audio / video recording device, a health monitoring device, a home automation system, a home security system, a gaming console, a drone, a household appliance, and so on.It should be noted that the computing device 102 may be portable, non-portable but mobile, or relatively immobile (e.g., a desktop computer, a household appliance). It should also be noted that the computing device 102 may be used with or embedded in many computing devices 102 or peripherals, such as in motor vehicles or a device attached to a personal computer. The computing device 102 may include additional information for clarity or brevity. Fig. 2 include omitted components and interfaces.

[0018] As illustrated, computing device 102 includes image sensors 104, one or more processors 202, and computer-readable media (CRM 204). Image sensors 104 may include one or more of any suitable image sensor (e.g., phase detection autofocus sensor, dual pixel autofocus (DPAF) sensor, charge-coupled device sensor). Processors 202 may include one or more of any suitable single-core or multi-core processor (e.g., central processing unit, graphics processing unit). CRM 204 includes memory media 206 and data storage media 208. An operating system (OS 210), applications 212, and a biometric authentication manager 108-1 are implemented as computer-readable instructions on CRM 204.These computer-readable instructions may be executed by processors 202 to provide some or all of the functionality described herein. For example, processors 202 may perform specific OS 210 computational tasks directed toward repurposing image sensors 104 as a biometric authentication scanner. CRMs 204 may include one or more non-transitory storage devices, such as a solid-state drive, a rotary magnetic drive, random access memory, or any type of storage media suitable for storing electronic instructions, each coupled to a data bus. The term "coupled" may refer to two or more elements that are in direct contact (physical, electrical, optical, etc.).), or refer to two or more elements that are not in direct contact with each other but nevertheless work together or interact with each other.

[0019] In aspects, various implementations of the biometric authentication manager 108-1 may include one or more integrated circuits, a system-on-chip, a secure key store, hardware embedded with firmware stored on read-only memory, a circuit board with various hardware components, or any combination thereof. As described herein, the biometric authentication manager 108-1 may include one or more components of the computing device 102 configured to repurpose the image sensors 104 as a biometric authentication scanner. In additional aspects, the biometric authentication manager 108-1 may be implemented as the computing device 102.

[0020] As illustrated, the computing device 102 also includes input / output ports 214 (I / O 214). The I / O ports 214 enable the computing device 102 to interact with other computing devices or users via peripherals, transmitting any combination of digital, analog, and radio frequency signals. The I / O ports 214 may include any combination of internal or external ports, such as Universal Serial Bus (USB) ports, audio ports, video ports, dual in-line memory module card slots, express peripheral component slots, and the like. Various peripherals may be operatively coupled to the I / O ports 214, such as human input devices, external CRM, speakers, displays, or other peripherals.

[0021] Computing device 102 also includes a communications system 216 that enables communication of device data, such as received data, transmitted data, or other data as described herein, and may provide connectivity to one or more networks and other devices connected thereto. Communication system 216 may include, for example, near-field communication transceivers, wireless local area network (WLAN) radios, wireless wide area network (WWAN) radios, and infrared transceivers. Data communicated via communication system 216 may be packetized or framed depending on the communication protocol or standard with which computing device 102 communicates.The communication system 216 may include wired interfaces, such as fiber optic or Ethernet interfaces, that facilitate communication over a local area network, a private network, an intranet, or the Internet. The communication system 216 may also include wireless interfaces, such as Wi-Fi and Bluetooth interfaces, that facilitate communication over a WLAN, a WWAN, or a cellular network.

[0022] Although not shown, computing device 102 may include a system bus, interconnect, or data transmission system coupled to the various components within computing device 102. The system bus, interconnect, or data transmission system may include any one or a combination of various bus structures, such as a memory bus, a peripheral bus, a USB, and / or a local bus utilizing any of a variety of bus architectures.

[0023] The computing device further includes a display 106 (e.g., a touchscreen display, a liquid crystal display (LCD)). The computing device may include or utilize any of a variety of displays, including an in-plane switching LCD, a vertical alignment LCD, an organic light-emitting diode display (OLED), and so forth. The display may be referred to as a screen so that content (e.g., images, videos) may be displayed on the screen.

[0024] Fig. 3 illustrates in more detail an example implementation 300 of the computing device 102 including the one or more image sensors 104. As illustrated in a detailed view 300-1, the computing device 102 includes a camera 302 in a rear side of a housing of the computing device 102. The one or more image sensors 104 are integrated into the camera 302.

[0025] A detailed view 300-2 shows a cross-sectional view of the camera 302. For clarity in the detailed view, some components of the camera 302 may be omitted. As illustrated, the camera 302 includes a transparent cover layer 304 (e.g., glass, plastic) disposed as the top layer. A lens assembly 306 consisting of a plurality of lenses 308 is disposed beneath the cover layer 304. A microlens array 310 is disposed beneath the lens assembly. A color filter array 312 (CFA 312) comprising a plurality of color filter elements 314 is disposed beneath the CFA 312. The one or more image sensors 104 are disposed beneath the CFA 312. In this example implementation 300, the one or more image sensors 104 are integrated into the camera 302 as a single image sensor 104-1.The image sensor 104-1 includes a plurality of photodiodes 316 that are capable of detecting intensities of the light rays, but not colors of the light rays.

[0026] Although not shown, the lens assembly 306 may include a motor operatively coupled (e.g., physically, magnetically) to one or more of the lenses 308. For example, the motor may translate one or more of the lenses 308 to appropriately focus light rays onto the image sensor 104-1 as part of an autofocus system. The lens assembly 306 and the microlens array 310 focus light rays onto light-sensitive areas of the image sensor 104-1. Depending on the application of the camera 302, the CFA 312 may be one of a variety of suitable CFAs, including a red-yellow-yellow-blue CFA, a cyan-yellow-yellow-magenta CFA, a Bayer CFA, and so on. The CFA 312 filters the intensities of the light rays according to wavelength range, so that the separate filtered intensities detected by the photodiodes 316 contain color information.

[0027] Fig. 4 illustrates an exemplary implementation 400 of the CFA 312 from Fig. 3, which is implemented as a Bayer CFA 312-1. As illustrated in a top view, the Bayer CFA 312-1 is a six-by-six square grid of square color filter elements 314. Although a square grid of square color filter elements is shown, the shape of the grid and the shape of the color filter elements 314 can be any suitable shape (e.g., rectangle). Additionally, the dimension of the square grid can be any suitable dimension (e.g., 1,000 by 1,000, 1,920 by 1,080). As also illustrated, the color filter elements 314 can be one of three different color filter elements 314 suitable for filtering the intensities of the light rays by wavelength range. In this example, the color filter elements 314 are divided into green color filter elements 314-1, blue color filter elements 314-2 and red color filter elements 314-3.Each two-by-two grid of color filter elements 314 includes two green color filter elements 314-1, one blue color filter element 314-2, and one red color filter element 314-3. This bold-bordered two-by-two grid may be referred to as a pixel 402.

[0028] Fig. 5 illustrates an exemplary implementation 500 of the image sensor 104-1 of Fig. 3, which is implemented as a dual-pixel autofocus sensor 104-2 (DPAF sensor 104-2). As illustrated in a top view, the DPAF sensor 104-2 consists of an array of dual pixels 502, each indicated by a square with a solid border. The array of dual pixels 502 is arranged in a six-by-six square grid. Although a six-by-six square grid is shown, the grid of dual pixels 502 can have any suitable dimension (e.g., 1,000 by 1,000) and shape (e.g., a rectangle). As illustrated, the dual pixel 502 is divided into a left photodiode 504 and a right photodiode 506.

[0029] In a conventional application of the DPAF sensor 104-1, light beams are focused onto the left photodiode 504 and the right photodiode 506 by the microlens array 310. Before an image of an object is captured, a difference in the photodiode amplitudes between the left photodiode 504 and the right photodiode 408 is calculated. The difference in the photodiode amplitudes can be referred to as the phase of the dual pixel 502. If the phase is within a threshold confidence unequal to or close to zero, for example, one or more of the lenses 308 operatively coupled to a motor can be positioned by the motor until the phase is within the threshold confidence equal to or close to zero. Once the phase is within the threshold confidence equal to or close to zero, the object of the image is in focus.When capturing the image of the object, the photodiode amplitudes of the left photodiode 504 and the right photodiode 506 are combined to form a full dual pixel.

[0030] However, the systems and techniques described in this document for using multi-perspective image sensors to authenticate topographical features take the opposite approach. By positioning (e.g., by a motor) the one or more lenses 308 such that the object of the image is not in focus, the phase of the dual pixel 502 is changed (e.g., maximized). This document now turns to an exemplary operating environment in which the maximized phase is used to enable the repurposing of the DPAF sensor 104-2 as a biometric authentication scanner. Example operating environment

[0031] Fig. 6 illustrates an example implementation 600 of a user 602 of a computing device 604. The computing device 604 includes a display 606, processors 608, a camera 610, and the biometric authentication manager 108 configured to use one or more image sensors as a biometric authentication scanner. The camera 610 includes a DPAF sensor 612 and an LED flash 614.

[0032] Fig. 7 illustrates an example operating environment 700 of user 602 of computing device 604. In this example, user 602 picks up computing device 604 in a locked state 702, as indicated by display 606-1. At 700-1, in response to user 602 picking up computing device 604 and tapping display 606-1, biometric authentication manager 108 presents user 602 with a prompt 704 to apply user input to a cover layer of camera 610. At 700-2, biometric authentication manager 108 may detect the user input, in this example, a fingertip 706 comprising a fingerprint, through a proximity sensor or other suitable sensor of computing device 604. In another example, the biometric authentication manager 108 may recognize the fingertip 706 by determining (e.g.,by light level) that the camera 610 is obscured by the fingertip 706. Also at 700-2, in response to determining that the fingertip 706 obscures the camera 610, the biometric authentication manager 108 may instruct the camera 610 to achieve a focus distance (e.g., by positioning one or more lenses via a motor) corresponding to an out-of-focus image. For example, the focus distance may be set to infinity while the subject of the image, the fingertip 706, rests at less than one centimeter on the cover layer of the camera 610. This may maximize the phase of each dual pixel 502 in the DPAF sensor 612.

[0033] Continuing at 700-2, the biometric authentication manager 108 may instruct the camera 610 to activate the LED flash 614 to illuminate the fingertip 706 to increase the signal-to-noise ratio. In some examples, the LED flash 614 in the camera 610 may be positioned such that the fingertip 706 is illuminated in a direction sufficient to alter or emphasize features of the fingerprint from a first perspective to a greater extent than from a second perspective, thereby facilitating their identification. In this example, the first perspective and the second perspective are viewed by the left photodiode 504 and the right photodiode 506, respectively, of each dual pixel 502 of the DPAF sensor 612. Altering or emphasizing the features of the fingerprint may enable greater resolution of those features.

[0034] Also at 700-2, instead of combining the photodiode amplitudes of the left and right photodiodes of each dual pixel 502, the biometric authentication manager 108 may instruct the camera 610 to capture two sets of fingerprint image data, one set from the left photodiode 504 and one set from the right photodiode 506. At 700-3, the biometric authentication manager 108 may instruct the processors 608 to calculate a disparity map that captures the phase of each dual pixel photodiode pair from the image data captured at 700-2. The biometric authentication manager 108 may input the disparity map into a user-defined normalizer function to generate an embedding 708 (e.g., an embedding 708-1) or other suitable representation (e.g., a hash code, a sequence of numbers) of the image data that does not include visual information.At 700-4, a final embedding 710 of the fingerprint of the user 602 may be generated in several steps, wherein at each step the image data captured at 700-2 and 700-3 is compressed.

[0035] At 700-5, the biometric authentication manager 108 may instruct the processors 202 to compare the final embedding 710 of the fingerprint to a library of embeddings 712. During an initial setup process of the computing device 604, the biometric authentication manager 108 may generate the library of embeddings 712 by requesting the user 602 to scan a fingerprint multiple times at slightly different locations, calculating a final embedding for each scan, and storing the final embeddings in a memory of the computing device 604. At 700-6, the biometric authentication manager 108 may provide a comparison result 714 in response to the comparison.The biometric authentication manager 108 may determine, based on the comparison result 714 and a threshold confidence, that the final fingerprint embedding 710 and the fingerprint used to generate the library of embeddings 712 are the same fingerprint. At 700-7, in response to determining that the final fingerprint embedding 710 and the fingerprint library of embeddings 712 are from the same fingerprint, the biometric authentication manager 108 transitions the computing device from the locked state 702 to an unlocked state 716 shown by the display 606-2.

[0036] In implementations, the computing device 604 may be in the unlocked state 716, but a high-privilege resource (e.g., a financial application) of the computing device 604 that requires access authorization may be in a locked state. In such implementations, the biometric authentication manager 108 may perform the operations described herein to unlock the high-privilege resource. In other implementations, the computing device 604 may be in the unlocked state, the high-privilege resource may be in the locked state, and the computing device 604 may include a dedicated biometric authentication system, such as a fingerprint scanner, on a front panel of the computing device 604. The high-privilege resource, in these implementations, may require two-factor authentication (2FA) to be unlocked.The biometric authentication manager 108 may perform the operations described herein as one of two factors of 2FA, the other factor being, for example, the fingerprint scanner on the front of the computing device 604. Optionally or additionally, the biometric authentication manager 108 may perform the described operations concurrently with a scan of a second finger by the fingerprint scanner on the front of the computing device 604. In another implementation, the computing device 604 may be in a locked state, require 2FA to unlock from the locked state, and include a dedicated biometric authentication system, such as a facial scanner on a front of the computing device 604.The biometric authentication manager 108 may perform the operations described herein as one of two factors of 2FA, the other being the facial scanner on the front of the computing device 604. The biometric authentication manager 108 may perform the operations simultaneously or sequentially, for example, upon a scan of a face by the facial scanner on the front of the computing device 604. Although a facial scanner is described on a front of the computing device 604, the dedicated biometric authentication system may be any suitable system (e.g., UDFPS, capacitive fingerprint scanner, optical facial scanner) on any suitable side (e.g., a front, a back) of the computing device 604.For example, biometric authentication manager 108 may perform the method described herein using a front-facing or rear-facing camera as one of two factors of 2FA. Meanwhile, a dedicated capacitive fingerprint scanner on an opposite or the same side of computing device 604 may scan a different user's fingertip as the other factor of 2FA.

[0037] Although a single DPAF sensor has been described, any suitable image sensor(s) configured to provide two or more image capture perspectives may be used. For example, two image sensors are physically separated and configured to provide the two or more image capture perspectives that may be utilized. In another example, a single non-DPAF image sensor may be used, configured to provide the two or more image capture perspectives. Exemplary procedures

[0038] Fig. Figure 8 outlines a method 800 that enables the use of image sensors as a biometric authentication scanner. The method is shown as sets of blocks that specify operations performed. The method is not necessarily limited to the order or combinations of the shown sets of blocks for performing the operations by the respective blocks. Furthermore, one or more of the operations may be repeated, combined, reorganized, or linked to provide a variety of additional or alternative methods. In sections of the following discussion, reference may be made to the exemplary implementation of Fig. 1 and on the Fig. 2 to 7, reference may be made to the detailed entities, which reference is merely exemplary. The techniques are not limited to the performance of one or more entities operating on a computing device.

[0039] At 802, a biometric authentication manager receives or captures one or more images, the one or more images representing topographical features of an object from two or more perspectives, the one or more images captured by one or more image sensors, the one or more image sensors having two or more image capture perspectives. For example, the object may be a user's fingertip, and the image sensors may be one or more cameras (e.g., zoom cameras, macrophotography cameras, telephotography cameras, slow-motion cameras) integrated into a back of a housing of a computing device. The cameras may be cameras with two or more image capture perspectives. The cameras with two or more image capture perspectives may be configured to perform the method described herein.Alternatively, cameras with two or more image capture perspectives (e.g., cameras for general photography, cameras for telephotography) can be repurposed to perform the method described in this document. In implementations, the biometric authentication manager can capture images with or receive images from the one or more cameras with the two or more image capture perspectives.

[0040] At 804, the biometric authentication manager determines the topographical features of the object based on the one or more images. For example, the topographical features may be minutiae (e.g., peaks, valleys, patterns) of a user's fingertip fingerprint.

[0041] At 806, the biometric authentication manager compares the topographical features of the object with previously captured topographical features of a previously imaged object to provide a comparison result. For example, the previously imaged object may be the user's fingertip. The previously captured topographical features may therefore be the minutiae of the fingerprint. The comparison result may be based on a correlation of common patterns (e.g., loops, swirls, arches), bump characteristics (e.g., ends, bifurcations, bridges), and other suitable minutiae points between the current fingerprint image and the previously captured fingerprint images. For example, the current and previously captured fingerprint images may be decomposed into weighted sums of polar shapelet-based functions, separating the fingerprint images into components with rotational symmetries.The comparison result can also be based on a correlation between the current and previously acquired fingerprint images of these components with rotational symmetries.

[0042] At 808, the biometric authentication manager determines, based on the comparison result from 806, that the object and the previously imaged object are one and the same object. The biometric authentication manager may do this, for example, to a threshold confidence, which may be a value (e.g., 90%, 95%) representing a similarity between the previously imaged fingertip and the current fingertip. Alternatively, the biometric authentication manager may determine that the object and the previously imaged object are the same object within a margin of error. The margin of error may be a value (e.g., 10%, 5%) representing a difference between the previously imaged fingertip and the current fingertip.

[0043] At 810, in response to determining that the object and the previously imaged object are the same object, the biometric authentication manager changes a credential for a resource associated with a computing device. The resource associated with the computing device could be an operating system of the computing device. In this example, the change in credential could be to unlock the computing device so that a user can access and enjoy features of the operating system. Additional examples

[0044] Additional examples are provided in the following section.

[0045] Example 1: A method comprising: capturing one or more images, the one or more images depicting topographical features of an object from two or more perspectives, the one or more images captured by one or more image sensors, the one or more image sensors having two or more image capture perspectives; determining, based on the one or more images, the topographical features of the object; comparing the topographical features of the object to previously captured topographical features of a previously imaged object to provide a comparison result; determining, based on the comparison result and a threshold confidence, that the object and the previously imaged object are one and the same object;and in response to determining that the object and the previously mapped object are the same object, changing a permission for a resource associated with a computing device;

[0046] Example 2: The method of example 1, wherein: the object having topographical features is a fingertip of a user; and the previously imaged object is the user's fingertip.

[0047] Example 3: The method of any preceding example, wherein the one or more image sensors are located on a backside of the computing device.

[0048] Example 4: The method of Example 3, wherein the one or more image sensors are one or more cameras of the computing device, and further comprising, prior to capturing the one or more images, displaying a prompt to place a finger on a camera lens cover, the camera lens cover covering a front surface of the one or more image sensors.

[0049] Example 5: The method of any preceding example, wherein the method is performed in response to the computing device being in an unlocked state but the resource of the computing device being in a locked state, and wherein changing the permission for the resource unlocks the resource associated with the computing device.

[0050] Example 6: The method of Example 5, wherein the resource is a financial account or other high-privilege resource that requires two-factor authentication, and the method provides one of two factors of two-factor authentication by determining that the object and the previously imaged object are the same object.

[0051] Example 7: The method of example 6, wherein another of the two factors of the two-factor authentication is a prior authentication for unlocking the computing device.

[0052] Example 8: The method of example 6, wherein another of the two factors of the two-factor authentication is a simultaneous fingerprint authentication performed on a different side of the computing device than a side on which the capturing of the one or more images is performed.

[0053] Example 9: The method of any of the preceding examples, wherein the one or more images are received from an image capturing device having a dual-pixel image sensor configured to provide the two or more image capturing perspectives.

[0054] Example 10: The method of any of the preceding examples, wherein the one or more images are received from an image capturing device having at least two image sensors configured to provide the two or more image capturing perspectives, wherein the at least two image sensors are physically separated from each other.

[0055] Example 11: The method of any preceding example, wherein determining the topographical features of the object generates an embedding, wherein the embedding is a numerical representation of the topographical features of the object.

[0056] Example 12: The method of Example 11, wherein comparing the topographical features compares the embedding to a previously generated embedding, wherein the previously generated embedding is a numerical representation of the previously captured topographical features of the previously imaged object.

[0057] Example 13: The method of any of the preceding examples, further comprising, prior to capturing the one or more images, determining that the object having the topographical features occludes the one or more image sensors.

[0058] Example 14: The method of Example 13, further comprising, in response to determining that the object having the topographical features obscures the one or more image sensors, and prior to or upon capturing the one or more images, illuminating the object.

[0059] Example 15: The method of example 14, wherein illuminating the object is performed in a direction sufficient to change features from one of the two or more perspectives to a greater extent than from another of the two or more perspectives, wherein the change in features enables greater resolution of the topographical features of the object.

[0060] Example 16: A method according to any one of the preceding examples, wherein the topographical features are minutiae of a fingerprint.

[0061] Example 17: A computing device comprising: one or more image sensors; one or more processors; and a memory storing: instructions that, when executed by the one or more processors, cause the one or more processors to implement a biometric authentication manager to provide biometric authentication utilizing the one or more image sensors by performing the method of any preceding example.

[0062] Example 18: A computer-readable storage medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of Examples 1 to 16. Conclusion

[0063] For the purposes of this discussion, an example is described in which a biometric authentication manager of a computing device (e.g., a smartphone) analyzes information associated with a user (e.g., fingerprint image data). In addition to the above description, a user may be provided with controls that allow the user to make choices about whether and when systems, programs, and / or features described herein may enable collection of information (e.g., fingerprint image data). The biometric authentication manager may be configured to use the information only after receiving explicit permission to use the data from the user of the computing device.For example, in situations where the biometric authentication manager may analyze image data for fingerprint characteristics to authenticate the user, individual users may be provided with the ability to provide input to control whether programs or features of the computing device can collect and use the data. Further, individual users may be able to continuously control what programs can or cannot do with the information. In addition, collected information may be pre-processed in one or more ways prior to transmission, storage, or other use so that personally identifiable information is removed. As described above, the raw image data of a user's fingerprint may be transformed (e.g.,compressed) that does not contain any personal information before being stored, for example, on the CRM of the computing device. This allows the user to control whether information about the user and the user's computing device is collected and how that information, if collected, may be used by the biometric authentication manager, the computing device, and / or a remote computing system.

[0064] Unless the context dictates otherwise, the use of the word "or" in this document may be considered the use of an "inclusive or" or an expression that allows the inclusion or application of one or more elements linked by the word "or" (e.g., a phrase "A or B" may be interpreted to allow only "A," to allow only "B," or to allow both "A" and "B"). In addition, as used in this document, a phrase that refers to "at least one of" a list of elements refers to any combination of those elements, including individual elements. For example, "at least one of a, b, or c" can cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other sequence of a, b, and c).Furthermore, elements shown in the accompanying drawings and terms discussed in this specification may identify one or more elements or terms, so that singular or plural forms of the elements and terms may be referred to interchangeably in this written description.

[0065] Although implementations of systems and techniques for converting one or more existing image sensors into a biometric authentication scanner, as well as devices enabling this, have been described in language specific to certain features and / or methods, the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary implementations of converting one or more existing image sensors into a fingerprint authentication scanner.

Claims

[1] Method comprising: Capturing one or more images, the one or more images representing topographical features of an object from two or more perspectives, the one or more images being captured by one or more image sensors, the one or more image sensors having two or more image capturing perspectives; Determining, based on the one or more images, the topographical features of the object; Comparing the topographical features of the object with previously recorded topographical features of a previously imaged object to provide a comparison result; Determining, based on the comparison result, that the object and the previously imaged object are one and the same object; and in response to determining that the object and the previously mapped object are the same object, changing a permission for a resource associated with a computing device. [2] The method of claim 1, wherein: the object having topographical features is a user's fingertip; and the previously imaged object is the user's fingertip. [3] The method of claim 1 or claim 2, wherein the one or more image sensors are located on a backside of a computing device and the one or more image sensors are one or more cameras of the computing device, and further comprising, prior to capturing the one or more images, displaying a prompt to place a finger on a camera lens cover, the camera lens cover covering a frontside of the one or more image sensors. [4] The method of any preceding claim, wherein the method is performed in response to the computing device being in an unlocked state but the resource of the computing device being in a locked state, and wherein changing the permission for the resource unlocks the resource associated with the computing device. [5] The method of claim 4, wherein the resource is a financial account or other high-privilege resource that requires two-factor authentication, and the method provides one of two factors of two-factor authentication by determining that the object and the previously imaged object are the same object. [6] The method of claim 5, wherein another of the two factors of the two-factor authentication is a simultaneous fingerprint authentication performed on a different side of the computing device than a side on which the capturing of the one or more images is performed. [7] The method of any preceding claim, wherein the one or more images are received from an image capturing device having a dual-pixel image sensor configured to provide the two or more image capturing perspectives. [8] The method of any preceding claim, wherein the one or more images are received from an image capturing device having at least two image sensors configured to provide the two or more image capturing perspectives, the at least two image sensors being physically separated from each other. [9] A method according to any one of the preceding claims, wherein determining the topographical features of the object generates an embedding, the embedding being a numerical representation of the topographical features of the object. [10] The method of claim 9, wherein comparing the topographical features compares the embedding to a previously generated embedding, wherein the previously generated embedding is a numerical representation of the previously recorded topographical features of the previously imaged object. [11] The method of any preceding claim, further comprising, prior to capturing the one or more images, determining that the object having the topographical features obscures the one or more image sensors. [12] The method of claim 11, further comprising, in response to determining that the object having the topographical features obscures the one or more image sensors, and prior to or upon capturing the one or more images, illuminating the object. [13] The method of claim 12, wherein illuminating the object is performed in a direction sufficient to change features from one of the two or more perspectives to a greater extent than from another of the two or more perspectives, the change in features enabling greater resolution of the topographical features of the object. [14] Computing device comprising: one or more image sensors; one or more processors; and a memory that stores the following: Instructions that, when executed by the one or more processors, cause the one or more processors to implement a biometric authentication manager to provide biometric authentication using the one or more image sensors by performing the method of any preceding claim. [15] A computer-readable storage medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 13.

Images