Systems and methods for correcting distortion in biometric images

Abstract

A system and method for correcting distortion within a biometric image. A biometric image is generated based on first sensor data acquired from a sensing device. The first sensor data may include distortion generated in response to a change in distance or tilt between components of a corresponding electronic device. Furthermore, second sensor data corresponding to a calibration image is acquired from the sensing device. A corrected biometric image is generated based at least in part on the biometric image and the sensor data.

Description

Technical Field

[0001] Embodiments disclosed herein relate generally to electronic devices and, more particularly, to correcting distortion in an image of an input object. Background Art

[0002] Biometric sensing devices, including image sensor devices, can be used in a variety of electronic systems. The image sensor device can include a sensing region defined by a surface, wherein the image sensor device determines the presence, position, force, and / or motion of one or more input objects. The image sensor device can be used to provide an interface for an electronic system. For example, the image sensor device can be used as an input device for a larger computing system, such as a touchpad integrated into or attached to a laptop or desktop computer. Image sensor devices are also commonly used in smaller computing systems, such as touchscreens integrated into cellular phones. Summary of the Invention

[0003] In one embodiment, a method for compensating for distortion in a biometric image includes acquiring first sensor data from a sensing device and generating a biometric image based at least in part on the first sensor data. The method also includes acquiring second sensor data from the sensing device and generating a corrected biometric image based at least in part on the biometric image and the second sensor data. The second sensor data corresponds to a calibration image.

[0004] In one embodiment, an electronic device includes a display device, a sensor, and a processing system. The processing system is coupled to the sensor and configured to receive first sensor data from the sensor. The processing system is further configured to generate a biometric image based at least in part on the first sensor data. Additionally, the processing system acquires second sensor data using a plurality of sensing electrodes and generates a corrected biometric image based at least in part on the biometric image and the second sensor data. The second sensor data corresponds to a calibration image.

[0005] In one embodiment, a processing system for a biometric device includes a sensor module and a determination module. The sensor module includes sensor circuitry. The sensor module is configured to receive first sensor data from a plurality of sensing elements and receive second sensor data from the plurality of sensing elements while displaying a calibration image. The second sensor data corresponds to the calibration image. The determination module is configured to generate a biometric image from the first sensor data and generate a corrected biometric image based at least in part on the biometric image and the second sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In order that the manner in which the above-described features of the present disclosure can be understood in detail, a more particular description of the present disclosure, briefly summarized above, may be given by reference to embodiments, some of which are illustrated in the accompanying drawings. It is to be noted, however, that the drawings illustrate only exemplary embodiments and are therefore not to be considered limiting of the scope of the invention, as the disclosure may admit to other equally effective embodiments.

[0007] Figure 1A is a schematic block diagram of a biometric device according to one or more embodiments.

[0008] Figure 1B and 1C is a schematic block diagram of an electronic device according to one or more embodiments.

[0009] Figure 1D Illustrated are example calibration images in accordance with one or more embodiments.

[0010] Figure 2A and 2B is a schematic block diagram of an electronic device according to one or more embodiments.

[0011] Figure 3 is a schematic block diagram of an electronic device according to one or more embodiments.

[0012] Figure 4A and 4B are images illustrating example distortion types in accordance with one or more embodiments.

[0013] Figure 5 A method for correcting a biometric image according to one or more embodiments is illustrated.

[0014] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements common to the figures. It is contemplated that elements disclosed in one embodiment may be advantageously utilized in other embodiments without specific recitation. The figures referred to herein should not be understood as being drawn to scale unless specifically noted. Likewise, for clarity of presentation and explanation, the figures are often simplified and details or components are omitted. The figures and discussion serve to explain the principles discussed below, where identical reference numerals represent identical elements. DETAILED DESCRIPTION

[0015] The following detailed description is merely exemplary in nature and is not intended to limit the present disclosure or the application and uses of the present disclosure. Furthermore, there is no intention to be bound by any express or implied theory presented in the foregoing background, summary or the following detailed description.

[0016] In various embodiments, a biometric sensing device includes an image sensor device configured to obtain one or more images of an input object. The biometric sensing device may be configured to remove distortion within the acquired biometric images, thereby increasing the likelihood that the user will be correctly identified and / or authenticated by the biometric sensing device. The distortion may be attributable to a change in the distance between the sensing surface of the biometric device and the image sensor device caused by the force applied by the input object. The input object may be a finger, a palm, a full hand, a partial hand, etc. The image sensor device may be an imager with collimation, and the image may be an optical image.

[0017] Figure 1A A biometric sensing device 100 is illustrated. The biometric sensing device 100 includes a sensor 120 and a processing system 110. The biometric sensing device 100 can be configured to utilize one or more of various biometric sensing methods, techniques, and / or devices to acquire an image of one or more features of an input object. An image of one or more features of an input object may be referred to as a biometric image. In one embodiment, the biometric sensing device 100 may be configured to acquire an image of features of a fingerprint (such as ridges, valleys, minutiae, etc.). Such an image may be referred to as a fingerprint image.

[0018] In various embodiments, the biometric sensing device 100 may be configured to capture a biometric image of an input object for use in authenticating a user within an electronic system. For example, the biometric sensing device 100 may be configured to utilize a biometric image to grant or deny a user access to an electronic device (e.g., a mobile phone, tablet, laptop, personal computer, etc.). The term "authentication" may refer to identification, verification, confirmation, identification, and / or authorization.

[0019] Sensor 120 can be any sensing device configured to capture reflected light from a sensing surface. In one embodiment, sensor 120 includes multiple photodetectors configured to capture reflected light. For example, sensor 120 may include multiple photodiodes. In one or more embodiments, each photodetector corresponds to a pixel of sensor 120. Alternatively, the photodetectors may be grouped to form pixels of sensor 120, such that each pixel is formed by two or more photodetectors. Furthermore, sensor 120 may include a collimator having a lens with depth of field. In one embodiment, sensor 120 is an imaging device such as a camera. For example, sensor 120 may include a complementary metal oxide semiconductor (CMOS) image sensor, a charge coupled device (CCD) image sensor, or the like. In such an embodiment, sensor 120 may include multiple pixels arranged in two dimensions and configured to receive reflected or transmitted light from an input object.

[0020] In one or more embodiments, sensor 120 can utilize the principle of direct illumination of an input object, which, depending on the configuration, may or may not be in contact with the input surface of the sensing area. One or more light sources and / or light-directing structures (e.g., collimators, lenses, reflectors, etc.) can be used to direct light to the sensing region of the sensing area. For example, a portion of a display can provide light to this region. Throughout this specification, "sensing region" refers to the area above the imaging system that the sensing region can detect, whether in or out of focus or illuminated, and "sensing area" refers to the area within the focus or illuminated area. In one embodiment, illumination of the input object can be provided by TIR (total internal reflection) from outside the display or from within (i.e., the light source is the display). When an input object approaches the sensing area, light is reflected from the surface of the input object. This reflection can be detected by the sensing elements of sensor 120 and used to determine information about the input object. Alternatively, sensor 120 can image light absorbed by the input object (i.e., not reflected from the cover layer of the biometric device). Note that light reflected by the input object can be imaged (bright field) or absorption of light without reflection by the cover layer at the input object contact point can be imaged (dark field).

[0021] In one embodiment, the display device 160 may include a glass or plastic (e.g., polyimide) substrate for TFT circuitry and / or other circuitry that may be used to provide a visual and / or provide other functionality. The display device 160 may also include a cover layer disposed over the display circuitry, which may also provide an input surface for the sensor 120. The cover layer may include a cover lens, sometimes referred to as a cover glass or lens. Example cover lens materials include optically transparent amorphous solids, such as chemically hardened glass, and optically transparent crystalline structures, such as sapphire. The flexible cover layer may be composed of polymers such as PET (polyethylene terephthalate), transparent polyimide, and polyurethane or other hard coatings. In one embodiment, the cover layer may serve as a sensing surface, for example, a surface where input objects are positioned to be sensed by the sensor 120.

[0022] In one embodiment, the display device 160 may be configured to display a calibration image. The calibration image (e.g., Figure 1CThe calibration image 180 may include a repeating pattern of elements 182 that can be utilized by the processing system 110 to remove distortions from the biometric image. The calibration image may include a regular or irregular (e.g., coded) pattern of elements 182. Furthermore, the calibration image may include a pattern of one or more dots, lines, bars, circles, squares, rectangles, and / or any other shape. In one embodiment, as will be discussed in further detail below, the calibration image 180 may be imaged by the biometric sensing device 100 (e.g., by internal reflection at or near a sensing surface) and utilized by the processing system 110 to remove one or more distortions from the fingerprint image.

[0023] In one embodiment, sensor 120 can be coupled to substrate 170 of electronic device 150. For example, sensor 120 can be coupled to substrate 170 via one or more adhesive layers. In one embodiment, substrate 170 can be part of a frame of electronic device 150. For example, substrate 170 can be part of a middle frame of electronic device 150.

[0024] Further references Figure 1A , the processing system 110 can be electronically (eg, electrically, communicatively) coupled to the sensor 120 and configured to control the sensor 120 to obtain sensor data that can be utilized to generate an image. For example, the processing system 110 can be coupled via a substrate (eg, Figure 1B substrate 170 or Figure 1C One or more traces (not shown) of the substrate 171 of the embodiment are electrically coupled to the sensor 120.

[0025] The processing system 110 includes a sensor module 112, a determination module 114, and a template storage 116. In addition, the processing system 110 may include an optional display driver module 118. In one or more embodiments, the processing system 110 includes part or all of one or more integrated circuit (IC) chips and / or other circuit components.

[0026] Sensor module 112 may include sensing circuitry configured to receive sensor data from sensor 120. Furthermore, sensor module 112 may instruct sensor 120 to acquire sensor data. In one embodiment, the sensor data is raw data received from sensor 120. For example, the sensor data may be unprocessed prior to being received by sensor module 112. In other embodiments, the sensor data is raw data that is substantially unprocessed prior to being received from sensor 120. For example, the sensor data may be filtered, converted from an analog domain to a digital domain, etc. by sensor 120 before being transmitted to sensor module 112.

[0027] The determination module 114 may be configured to process the sensor data. For example, the determination module 114 may correct one or more distortions within the sensor data and generate a corrected biometric image that may be used for user authentication, identification, etc. In one embodiment, correcting the distortion may include at least one of minimizing and removing the effects of the distortion within the sensor data before generating the biometric image. In other embodiments, correcting the distortion may include at least one of minimizing and removing the effects of the distortion from the biometric image.

[0028] In one embodiment, determination module 114 compares the corrected biometric image to templates stored in template storage 116. Determination module 114 can authenticate or identify the user based on a determination that one or more of the templates match the corrected biometric image. In other embodiments, determination module 114 transmits the corrected biometric image and one or more matching templates to a processor external to processing system 110 for user authentication and / or identification. In one embodiment, determination module 114 transmits the corrected biometric image to a processor external to processing system 110 for template matching and user authentication and / or identification. Determination module 114 can also be configured to generate a user template from sensor data received using sensor 120 and store the user template in template storage 116.

[0029] Template storage 116 may include one or more non-transitory computer-readable storage media. Template storage 116 may be configured to store an enrollment view of a biometric image of a user's fingerprint, e.g., a user template. Template storage 116 may also be configured for long-term storage of information. In some examples, template storage 116 includes a non-volatile storage element. Non-limiting examples of non-volatile storage elements include a magnetic hard drive, an optical memory device, a solid-state memory device, flash memory, a magnetic storage device, or an electrically programmable programmable memory (EPROM) or an electrically erasable programmable programmable memory (EEPROM).

[0030] In one embodiment, the determination module 114 compares a fingerprint image generated from sensor data received using the sensor 120 with a user template stored in the template storage 116 to determine whether to authenticate the user. If the user is authenticated, the user can be provided with access to the electronic device (e.g., the electronic device 150 along with the data and programs on the device). The determination module 114 can also be configured to generate a user template from the sensor data received using the sensor 120 and store the user template in the template storage 116.

[0031] The display driver module 118 includes display driver circuitry configured to control the display of a display device (e.g., Figure 1BFor example, the display driver module 118 may drive an update signal to an element of the display of the display device 160 to update the displayed image. The display driver module 118 may include one or more source drivers, a display timing controller, and / or gate selection circuits, which may be used to receive display data, generate a display update signal, and drive the display update signal to the display to update the display. In one embodiment, when capturing a biometric image and / or an image of a sensing surface, the display driver module 118 may drive one or more organic light emitting diode (OLED) pixels of the display device 160 to provide illumination for the sensor 120.

[0032] Figure 1B An electronic device 150a is illustrated in accordance with one or more embodiments. The electronic device 150a may be a computing device, such as a mobile phone, a tablet, a laptop computer, a personal computer, or the illustrated embodiment, and includes a display device 160 and a biometric sensing device 100. In some embodiments, the sensor 120 is configured to detect an input object on at least a portion of an active area of ​​a display of the display device 160. The active area of ​​the display may correspond to the area of ​​the display in which an image is displayed. The display device 160 may be any type of display capable of displaying a visual interface to a user and may include any type of light emitting diode (LED). The display device 160 may also be foldable, flexible, or rigid and may be flat, curved, or have other geometric shapes.

[0033] In one embodiment, the display device 160 includes a substrate 162, a display circuit 164, pixels 166, and a cover layer 168. The substrate 162 can be a glass or plastic substrate. The display circuit 164 can be disposed within one or more layers of the substrate 162. The display circuit 164 can include TFT circuits and / or other circuits. The display circuit 164 can include, among other things, one or more transistors and capacitors. The display circuit 164 is coupled to the pixels 166 and controls the brightness of each of the pixels 166. Each of the pixels 166 includes one or more sub-pixels. The sub-pixels of each pixel 166 are driven to a predetermined brightness level to set one or more of the brightness and color output by each pixel 166.

[0034] In one embodiment, the display circuitry 164 is coupled to data lines, gate lines, and emission lines disposed within one or more layers of the substrate 162. Display update signals are driven onto sub-pixels of each pixel 166 via the data lines, gate lines, and emission control lines to update the image displayed by the display of the display device 160. For example, gate select signals can be driven on the gate lines to select and deselect pixels 166 for display updates. Pixel data signals are driven onto the selected pixels via the data lines to update the image displayed by the display of the display device 160. Additionally, emission control signals can be driven on the emission control lines to control the overall brightness of the display of the display device 160. In one embodiment, the display driver module 118 is coupled to the display circuitry 164 and configured to drive the display update signals.

[0035] The electronic device 150a may also include a cover layer 168 disposed over the display device 160. The cover layer 168 may be coupled to the display device 160 via an optically clear adhesive (OCA) layer 169. The cover layer 168 may include a cover lens, which is sometimes referred to as a cover glass or lens. Example cover lens materials include optically transparent amorphous solids, such as chemically hardened glass or polymers, such as transparent polyimide (e.g., with a hard coating), and optically transparent crystalline structures, such as sapphire. In one embodiment, the cover layer 168 may serve as a sensing surface, for example, a surface where an input object is positioned to be sensed by the sensor 120. Alternatively, one or more optional layers may be disposed over the cover layer 168. The optional layers may include one or more sensor electrode layers for a capacitive sensing device, an input surface, and the like.

[0036] In one embodiment, sensor 120 may be coupled to substrate 170 of electronic device 150a. For example, sensor 120 may be coupled to substrate 170 via one or more adhesive layers (not shown). In one embodiment, substrate 170 is a flexible printed circuit (FPC). In one embodiment, display device 160 is coupled to substrate 170 via adhesive at selected points on display device 160 (e.g., edges of display device 160) to minimize strain. In one embodiment, substrate 170 is adhered to a frame (not shown) of display device 160. For example, substrate 170 may be adhered to a center frame (not shown) of display device 160.

[0037] In one embodiment, distortion in the sensor data may be caused in response to a change in the distance and / or tilt between one or more layers of display device 160 and sensor 120. For example, a change in the distance and / or tilt between sensor 120 and one or more of substrate 162, display circuitry 164, and pixels 166 may cause distortion in the sensor data. Additionally, distortion in the sensor data may be caused in response to a change in the distance and / or tilt between sensor 120 and cover layer 168.

[0038] Figure 1C An electronic device 150b is shown in accordance with one or more embodiments. The electronic device 150b is similar to Figure 1B The electronic device 150a is configured as an electronic device, however, Figure 1B In the embodiment of the present invention, when the sensor 120 is disposed between the substrate 170 and the display device 160, Figure 1C In an embodiment, a first side of substrate 171 is adhered to display device 160 and sensor 120 is adhered to a second side of substrate 171. In addition, substrate 171 includes an opening 172 above sensor 120 to allow reflected light to propagate to sensor 120.

[0039] In one embodiment, substrate 171 may be part of a frame (not shown) of display device 160. For example, substrate 171 may be part of a middle frame of display device 160.

[0040] In one embodiment, the display device 160 may be configured to display a calibration image, such as Figure 1D For example, calibration image 180 may be displayed on one or more layers of display device 160 or on a layer external to display device 160 (e.g., cover layer 168 or OCA layer 169). Calibration image 180 may include a display image that can be processed by processing system 110 ( Figure 1A ) can be a repeating or non-repeating pattern of elements (e.g., element 182) utilized to remove, compensate, or correct distortion from the acquired image. Alternatively, the elements of calibration image 180 can be randomly or pseudo-randomly positioned within calibration image 180. In one embodiment, calibration image 180 can be displayed on overlay 168 ( Figure 1B 、 1C ). For example, calibration image 180 can be displayed on either side of overlay 168. In other embodiments, calibration image 180 can be displayed on either side of pixel 166 ( Figure 1B 、 1C ) and the cover layer 168 are displayed on the layers of the display device 160. For example, the OCA layer 169 ( Figure 1B 、 1C). Furthermore, in various embodiments, calibration image 180 can be displayed on the TFT layer of display circuitry 164. In one or more embodiments, OCA layer 169 can be patterned to include element 182 of calibration image 180. For example, calibration image 180 can be embedded within OCA layer 169. In other embodiments, cover layer 168 can be patterned to include element 182 of calibration image 180. For example, calibration image 180 can be embedded within cover layer 190. In other embodiments, calibration image 180 can be embedded within display device 160 or attached to display device 160.

[0041] In various embodiments, each element (eg, element 182) and the area between the elements of the calibration image may correspond to one or more pixels 166 ( Figure 1B 、 1C ). For example, the areas between elements of the calibration image and each element may correspond to a set of two or more pixels 166.

[0042] In one or more embodiments, elements of the calibration image (e.g., element 182) correspond to an illuminated area (illuminated portion) and a non-illuminated area (non-illuminated portion) of the display of display device 160. For example, the area defined by the elements may correspond to one of the illuminated area and the non-illuminated area, and the area between the elements may correspond to the other of the illuminated area and the non-illuminated area.

[0043] Although the elements of calibration image 180 are illustrated as including element 182, in other embodiments, the elements of calibration image 180 may include one or more points, lines, bars, circles, squares, rectangles, other shapes, or combinations thereof. In one embodiment, the elements of calibration image 180 include one or more of a common size, shape, and orientation. In other embodiments, one or more of the elements of the calibration image may differ in size, shape, and / or orientation from another one or more of the elements. As will be discussed in further detail below, calibration image 180 may be imaged by biometric sensing device 100 and utilized by processing system 110 to remove, compensate for, or correct one or more distortions from an acquired image (e.g., a biometric image). In one or more embodiments, at least the center of each element of calibration image 180 corresponds to the correct known spacing of pixels within sensor 120 and may be used to correct an acquired image (e.g., a biometric image).

[0044] In various embodiments, the illuminated and non-illuminated areas of calibration image 180 may be generated based on an orthogonal code. For example, a shift register driven by a signal generated by a pseudorandom code may be used to generate the illuminated and non-illuminated areas. The length of the code may correspond to the area of ​​the display of display device 160 driven by calibration image 180.

[0045] Figure 2A An embodiment of an electronic device 250a is illustrated in accordance with one or more embodiments. Figure 2A In the embodiment illustrated in FIG, the sensor 120 includes a sensing element 122, a lens 124, and a collimator filter layer 126. Figure 2A A sensor including multiple lenses 124 is illustrated, but in one embodiment, sensor 120 includes a single lens disposed over a two-dimensional array of sensing elements 122 without a collimator. A collection of more than one lens, mirror, or collimator may include one or more focusing elements between the sensing surface and the sensing elements.

[0046] In one embodiment, sensor 120 is an imager, and sensing element 122 is a photodetector of the imager, such as a photodiode. Furthermore, each photodetector can correspond to a pixel of sensor 120. Alternatively, multiple photodetectors can be grouped to form each pixel of sensor 120. Sensor 120 can include one or more additional layers, such as one or more filter layers. In one embodiment, sensor 120 includes one or more infrared (IR) filter layers disposed between sensing element 122 and lens 124. Furthermore, a reflective mirror element can be used instead of a refractive lens element to focus light.

[0047] In one embodiment, the sensing element 122 is configured to convert light into a current signal, and the signal can be accumulated into an electric charge (e.g., on a capacitor). In one or more embodiments, the sensing element 122 is arranged in a two-dimensional array on the substrate of the sensor 120. In addition, the sensing element 122 can be arranged as a pixel matrix that can be controlled via select lines and readout lines to capture light and output a corresponding current signal. For example, each sensing element can be coupled to a select line to activate the sensing element for sensing and a readout line for outputting the current signal to an amplifier. In one or more embodiments, the amplifier converts the current signal or accumulated charge into a voltage signal. The amplifier and / or other readout circuits (e.g., an analog-to-digital converter, a multiplexer, etc.) can be arranged within the integrated circuit of the sensor 120 or within the sensor module 112.

[0048] Lenses 124 may be positioned above sensing elements 122 and configured to focus light (i.e., near a focal point rather than at an imaging point) onto a corresponding one of sensing elements 122. Furthermore, lenses 124 may be arranged in an array similar to or different from the array of sensing elements 122. In the illustrated embodiment, each lens 124 is positioned above one of sensing elements 122. However, in other embodiments, at least one of lenses 124 is positioned above three or more sensing elements 122 arranged in a two-dimensional array, with the sensing elements in an image plane corresponding to the focal point of the lens and sensing region. In various embodiments, lenses 124 may be biconvex or diffractive lenses. More than one focusing element (e.g., a lens or focusing mirror) may be positioned in the optical path from the sensing region to the sensing elements. In one embodiment, lenses 124 may be configured to reflect light onto sensing elements 122. In such an embodiment, lenses 124 are positioned below sensing elements 122, such that sensing elements 122 are between cover layer 168 and lenses 124. In one embodiment, cover layer 168 is the sensing surface.

[0049] The collimator filter layer 126 is disposed between the lens 124 and the sensing element 122 and is configured to prevent crosstalk between adjacent sensing elements 122. In one or more embodiments, the collimator filter layer 126 can be configured to condition light using an array of apertures. The apertures can also be referred to as collimator holes, collimator filter holes, collimator through-holes, or holes, among other things. In one embodiment, the collimator filter layer 126 only allows reflected light rays that are incident at normal or near normal incidence to the collimator filter layer 126 to pass through and reach the sensing element 122. In one embodiment, the collimator filter layer 126 includes an opaque layer having an array of apertures. In addition, the collimator filter layer 126 can be laminated, stacked, or built directly onto the sensing element 122. In one or more embodiments, the collimator filter layer 126 is monolithic.

[0050] Figure 2B An electronic device 250b is shown according to one or more embodiments. The electronic device 250b is similar to the electronic device 250a ( Figure 2A However, the sensor 220 of the electronic device 250b includes a camera 230. For example, the camera 230 may be a CMOS or CCD image sensor, as described above, among others.

[0051] Sensor 220 is coupled to a processing system 110 similar to the processing system of sensor 120 ( Figure 1AIn one embodiment, one or more elements of the processing system 110 may be part of the camera 230 . For example, one or more of the sensor module 112 , the determination module 114 , and the template storage 116 may be part of the camera 230 .

[0052] The camera 230 is configured to capture sensor data corresponding to an input object placed on the overlay 168. In addition, the camera 230 can be configured to capture sensor data corresponding to an image displayed on one or more layers of the display device 160 or the electronic device 250b. The sensor data captured by the camera 230 can be processed as described with respect to FIG.

[0053] Figure 3 An embodiment of an electronic device 350 that undergoes deflection due to a force applied by an input object 310 is illustrated. In such an embodiment, a fingerprint image of the input object 310 captured by the sensor 120 may be distorted due to the deflection causing the position of one or more portions of the input object 310 to be altered compared to when no deflection occurs. For example, the magnification (or reduction) of the fingerprint image may vary corresponding to the amount of deflection of the display device 160, thereby distorting the corresponding fingerprint image (e.g., by shifting the focus of the collimated light, by bending the cover lens relative to the focus to change the image plane). In other embodiments, the fingerprint image may undergo other types of distortion. For example, the fingerprint image may undergo distortion due to the shape of the lens 124, changes in luminosity, and one or more layers within the display device 160 twisting, tilting, and / or vertical movement in response to deformation of the display device 160 relative to the sensor 120. In one embodiment, movement of the display device 160 relative to the sensor 120 may occur because the display device may be flexible. In one embodiment, the electronic device 350 may include the sensor 120 ( Figure 2A ) instead of sensor 220 ( Figure 2B ).

[0054] Figure 4A Various types of distortion are shown in the example images. Figure 4A As shown in FIG, image 402 includes no distortion, image 404 includes barrel distortion, where one or more portions of the image curve inward or bulge outward, image 406 includes pincushion distortion, where one or more portions of the image curve outward, and image 408 includes mustache distortion, where one or more portions of the fingerprint image curve outward and one or more portions of the fingerprint image curve inward. In one or more embodiments, the fingerprint image may experience at least one of these types of distortions due to the shape of lens 124, changes in luminosity, and one or more layers within display device 160 being distorted, tilted, and / or moving vertically in response to deformation of display device 160 relative to sensor 120.

[0055] Figure 4B A fingerprint image 412 without distortion, a fingerprint image 414 subjected to barrel distortion, a fingerprint image 416 subjected to pincushion distortion, and a fingerprint image 418 subjected to mustache distortion are illustrated. Figure 4B As illustrated in , a fingerprint image that has undergone barrel distortion (e.g., fingerprint image 414) has one or more portions that curve outward, as compared to a fingerprint image that has not undergone distortion (e.g., fingerprint image 412). Note that the visible portion of an undistorted image does not depend on translation or rotation, whereas a distorted image (e.g., with scale distortion within the image) depends on translation and rotation. Furthermore, a fingerprint image that has undergone pincushion distortion (e.g., fingerprint image 416) has one or more portions that curve inward, as compared to a fingerprint image that has not undergone distortion (e.g., fingerprint image 412). Additionally, a fingerprint image that has undergone mustache distortion (e.g., fingerprint image 418) has one or more portions that curve inward and one or more portions that curve outward, as compared to a fingerprint image that has not undergone distortion (e.g., fingerprint image 412). Therefore, removing distortion from a fingerprint image or sensor data before performing authentication can increase the likelihood of a match between the fingerprint image and a template.

[0056] In other embodiments, the fingerprint image may experience other types of distortion that are not described in Figure 4B For example, a fingerprint image may experience photometric distortion, where the brightness of one or more regions of the fingerprint image is different from the brightness of another region of the fingerprint image. In one embodiment, the photometric distortion may correspond to vignetting, where one or more corners and / or edges of the fingerprint image are darker than other regions of the fingerprint image. Additionally, a fingerprint image may experience magnification distortion, where one or more regions of the fingerprint image have a higher magnification than another region. In one or more embodiments, a fingerprint image may experience any combination of the above distortions. Although the description of the fingerprint image is provided with respect to the fingerprint image, Figure 4A and 4B , but in other embodiments, similar descriptions may apply to other types of biometric images.

[0057] Figure 5A method 500 for reducing the effects of distortion within a fingerprint image, according to one or more embodiments, is illustrated. At step 510, first sensor data is acquired. For example, the first sensor data may be acquired by the biometric sensing device 100. In one embodiment, the display device 160 may illuminate the cover layer 168, and light reflected by features of an input object (e.g., the input object 310) may be received by the sensing element 122 of the sensor 120. The light received by the sensing element 122 may be converted into a result signal and provided to the sensor module 112, and the sensor module 112 may generate the first sensor data from the result signal. For example, the sensor circuit of the sensor module 112 may receive the result signal and convert the result signal from a current signal to a voltage signal to generate the first sensor data. In various embodiments, converting the result signal may include converting the result signal from a digital domain to an analog domain.

[0058] In one embodiment, a fingerprint image may be generated from the first sensor data. For example, the processing system 110 may determine a biometric image from the first sensor data. The biometric image may be a fingerprint image. Alternatively, the biometric image may be another type of image that can be used to authenticate or identify a user. In one embodiment, the determination module 114 may receive the first sensor data from the sensor module 112 and generate a biometric image from the first sensor data. The biometric image may include one or more distortions. For example, the biometric image may include distortions due to the sensing surface (e.g., Figure 1B The distortion of the deformation of the cover layer 168) wherein the sensing surface is deflected toward the sensor 120, as Figure 3 Furthermore, in one or more embodiments, the deflection may not be uniform, such that some portions of the sensing surface may deflect more than other portions of the sensing surface.

[0059] In various embodiments, the deflection of the sensing surface may distort the biometric image. For example, the biometric image may experience one or more of barrel distortion, pincushion distortion, mustache distortion, magnification distortion, and photometric distortion. Due to the distorted fingerprint image, the processing system 110 may be unable to authenticate the user, thereby undesirably causing valid users to be rejected. For example, a determination may be made that the distorted biometric image does not match one of the stored templates, resulting in the user not being authenticated.

[0060] At step 520, second sensor data is acquired. For example, the second sensor data may be acquired by the biometric sensing device 100. In one embodiment, the sensor module 112 instructs the sensor 120 to receive the result signal while the calibration image (e.g., calibration image 180) is displayed on the display device 160. The result signal may be provided to the sensor module 112 and converted into the second sensor data. In one embodiment, the display driver module 118 may instruct the sensor module 112 to provide a first instruction indicating a start time for displaying the calibration image and a second instruction indicating an end time for displaying the calibration image. The sensor module 112 may provide instructions to the sensor 120 to acquire the result signal during a period of time between the start time and the end time of displaying the calibration image.

[0061] At step 530, a corrected biometric image is generated. In one embodiment, the corrected biometric image may be generated based at least in part on the biometric image and the second sensor data. In one or more embodiments, the determination module 114 may determine a correction factor from the second sensor data.

[0062] In one embodiment, the correction coefficients may correspond to a geometric model generated from the second sensor data. For example, the determination module 114 may receive the second sensor data from the sensor module 112 and may determine one or more measurements of the calibration image from the sensor data. In various embodiments, reference Figure 1D For calibration image 180 in FIG. 1 , determination module 114 may determine width 183 and / or height 184 of any one or more of elements (e.g., element 182) of calibration image 180, and / or distances between two or more elements (e.g., distance 185 and distance 186). Furthermore, determination module 114 may be configured to determine the orientation of one or more features of calibration image 80 and / or the position of one or more features of the calibration image.

[0063] In one or more embodiments, determination module 114 may determine distances between one or more features of calibration image 180. For example, distances 185 and / or 186 may be calculated between each parallel side of each element 182 of calibration image 180. In other embodiments, the distances may correspond to distances and / or dimensions of any other features within the calibration image. For example, the distance between the centers of each element may be determined. The distances may be calculated for different portions of the calibration image. For example, a first distance may be calculated for a portion of the calibration image corresponding to a first amount of deflection experienced by display device 160, and a second distance may be calculated for a portion of the calibration image corresponding to a second amount of deflection different from the first amount of deflection experienced by display device 160. The first portion of the calibration image may correspond to an edge region of display device 160, and the second portion of the calibration image may correspond to an interior region of display device 160. The edge region of display device 160 may experience less deflection than the interior region of display device 160.

[0064] The determination module 114 may also determine a correction factor based on the calculated distance. In one or more embodiments, the determination module 114 may calculate one or more correction factors based on the calculated distance. For example, different correction factors may be calculated for different portions of the calibration image so that different amounts of distortion can be removed from corresponding portions of the fingerprint image. One method for determining local distortion of the calibration image is to measure the autocorrelation of the windowed portion of the image using a similar windowed undistorted calibration image, which may be pre-measured or pre-calculated. By maximizing the autocorrelation of the scaled, rotated, or shifted calibration image, the local distortion can be estimated. Another method for determining local distortion is to apply a spatial 2D Fourier transform technique (e.g., on a windowed portion of the calibration image) to determine the spatial phase and frequency changes caused by the distortion. Various optimization and convolution techniques may be used to determine the distortion of a larger area calibration image using local estimates of scale, rotation, and shift. Multiple local estimates may be combined or interpolated to determine the distortion (e.g., scale, rotation, shift, gamma, etc.) at each pixel of the measured distorted calibration image. Compensation coefficients to correct for distortion (e.g., unscaled, unrotated, unshifted, degammad) can then be determined based on the measured distortion of the calibration image. In one embodiment, coefficients of a Brown distortion model (e.g., a Brown-Conrady model) can be extracted to correct for radial and tangential distortions 180 so that different amounts of distortion can be removed from corresponding portions of the fingerprint image.

[0065] Determination module 114 may apply correction factors to the biometric image to generate a corrected fingerprint image. Applying the correction factors to the biometric image corrects distortion within the biometric image. In one embodiment, applying the correction factors to the biometric image includes comparing distances calculated from different regions of the calibration image with similar regions of the biometric image and adjusting the biometric image based on the correction factors. For example, the distance between regions of the biometric image corresponding to features of the calibration image may be reduced or increased based on the correction factor(s). In various embodiments, features of the calibration image may correspond to pixels of the biometric image. For example, the distance between features or between the centers of features of the calibration image may correspond to the distance between pixels of the biometric image. In one embodiment, a feature of the calibration image corresponds to the distance between every two or more pixels. For example, a feature of the calibration image corresponds to the distance between two pixels separated by one or more pixels. Furthermore, the placement, size, and number of features of the calibration image may correspond to the distances between the pixels of the biometric image to be measured. It may be useful to upsample the image to a higher resolution (e.g., to 1.5x, 2x, 3x, 4x, etc.) for compensating a distorted fingerprint image to an undistorted fingerprint image, although downsampling (e.g., back to the original resolution) after compensation for matching may also be useful.

[0066] In one embodiment, determination module 114 may increase the distance between biometric image pixels determined to be less than the distance of the correction factor and / or decrease the distance between biometric image pixels determined to be greater than the distance of the correction factor. Furthermore, determination module 114 may increase or decrease the distance between biometric image pixels by a common amount based on the correction factor. In another embodiment, a local rotation and scaling matrix (e.g., interpolated calibration data) constructed from the correction factor may be used to compensate (e.g., undistort) an image or upsample the image by changing the local basis.

[0067] In one embodiment, the determination module 114 may generate a corrected image from the second sensor data. For example, the corrected image may be used to correct distortion within the biometric image. Furthermore, the determination module 114 may compare the corrected image with the first sensor data to generate a corrected biometric image. In one embodiment, the determination module 114 is configured to generate a biometric image from the first sensor data and compare the biometric image to the corrected image. For example, the determination module 114 may measure one or more features of the corrected image and use those measurements to adjust the fingerprint image and generate a corrected fingerprint image. Relative brightness corrections (e.g., due to illumination, vignetting, etc.) for the calibration image, along with local base distortion corrections (e.g., from radial and tangential lens distortion), are extracted from the calibration image and used to compensate the biometric image and generate the corrected biometric image. For example, the measured distance between a first point and a second point in the correction image may be compared to the distance to a corresponding point in the biometric image. The relative positions of points within the biometric image may be adjusted so that the distance between points in the biometric image is similar to the measured distance between points in the correction image. In one embodiment, one or more regions of the correction image may be compared to corresponding regions of the biometric image to generate a corrected fingerprint image.

[0068] In one embodiment, determination module 114 may determine the corrected image by adjusting elements of the corrected image so that the elements of the corrected image match corresponding elements of the displayed calibration image. For example, determination module 114 may adjust one or more of the size, shape, and position of elements of the corrected image so that each element of the corrected image matches a corresponding element of the calibration image. The amount by which each element of the corrected image is adjusted may be used to generate a corrected biometric image. For example, first sensor data may be adjusted based on the amount by which elements of the corrected image are adjusted. In one embodiment, the biometric image generated from the first sensor is adjusted based on the amount by which elements of the corrected image are adjusted. For example, the position of one or more pixels of the biometric image may be adjusted based on the amount by which one or more elements of the corrected image are adjusted. In one embodiment, the position of one or more pixels of the biometric image may be adjusted based on the amount by which elements in a corresponding portion of the corrected image are adjusted.

[0069] In one embodiment, the determination module 114 may compare the corrected fingerprint image to one or more user templates stored in the template storage 116. Based on a determination that the corrected biometric image matches one or more of the user templates, the user may be one or more of authenticated, verified, confirmed, authenticated, authorized, and identified, among other things.

[0070] Therefore, the embodiments and examples set forth herein are presented in order to best explain embodiments according to the present technology and its specific applications, and to thereby enable those skilled in the art to make and use the present disclosure. However, those skilled in the art will recognize that the foregoing description and examples have been presented for purposes of illustration and example only. The set forth description is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed.

[0071] Reference Numbers

[0072] 100 biometric sensing devices

[0073] 110 processing system

[0074] 112 sensor module

[0075] 114 Confirm module

[0076] 116 Template Storage Unit

[0077] 118 optional display driver modules

[0078] 120 sensors

[0079] 122 sensing elements

[0080] 124 lens

[0081] 126 collimator filter layer

[0082] 150 electronic devices

[0083] 150A electronic equipment

[0084] 150B electronic equipment

[0085] 160 display devices

[0086] 162 substrate

[0087] 164 display circuit

[0088] 166 pixels

[0089] 168 covering layer

[0090] 169th floor

[0091] 170 substrate

[0092] 171 substrate

[0093] 172 opening

[0094] 180 calibration image

[0095] 182 components

[0096] 183 width

[0097] 184 height

[0098] 185 distance

[0099] 186 distance

[0100] 190 Covering Layer

[0101] 220 sensor

[0102] 230 camera

[0103] 250A electronic equipment

[0104] 250B electronic equipment

[0105] 310 Input Object

[0106] 350 electronic equipment

[0107] 402 Images

[0108] 404 images

[0109] 406 images

[0110] 408 images

[0111] 412 fingerprint image

[0112] 414 fingerprint image

[0113] 416 Fingerprint image experiencing pincushion distortion

[0114] 418 fingerprint image with beard distortion

[0115] 500 Method

[0116] 510 steps

[0117] 520 steps

[0118] 530 steps

Claims

1. A method for compensating for distortion in a biometric image, the method comprising: acquiring first sensor data from a sensing device; generating the biometric image based at least in part on the first sensor data; acquiring second sensor data from the sensing device, the second sensor data corresponding to a calibration image displayed on a display device, wherein the calibration image includes elements associated with pixels of the display device; as well as A corrected biometric image is generated based at least in part on the biometric image and the second sensor data.

2. The method of claim 1 , further comprising authenticating a user based at least in part on the corrected biometric image.

3. The method according to claim 1, further comprising: generating one or more correction coefficients from the second sensor data; as well as The biometric image is adjusted based at least in part on the one or more correction factors. The method of claim 1 , further comprising generating a corrected image from the second sensor data.

5. The method of claim 4, wherein generating the corrected biometric image comprises comparing the biometric image to the corrected image.

6. The method of claim 5, wherein comparing the biometric image to the corrected image comprises comparing a first portion of the biometric image to a first portion of the corrected image.

7. The method of claim 1 , wherein acquiring the first sensor data from the sensing device comprises: irradiating a sensing surface; as well as Light reflected by one or more features of an input object at the sensing surface is detected.

8. The method of claim 1 , wherein acquiring the second sensor data from the sensing device comprises: The second sensor data is acquired while the calibration image is displayed on the display device.

9. The method of claim 8, wherein the elements further correspond to illuminated and non-illuminated portions of the calibration image.

10. An electronic device comprising: Display devices; sensor; as well as a processing system coupled to the sensor, the processing system configured to: receiving first sensor data from the sensor; generating a biometric image based at least in part on the first sensor data; receiving second sensor data from the sensor, the second sensor data corresponding to a calibration image displayed on the display device, wherein the calibration image includes elements associated with pixels of the display device; as well as A corrected biometric image is generated based at least in part on the biometric image and the second sensor data.

11. The electronic device of claim 10, wherein the processing system is configured to authenticate a user based at least in part on the corrected biometric image. 12 . The electronic device of claim 10 , wherein the sensor comprises a plurality of sensing elements disposed on a first substrate, and wherein the first substrate is coupled to the display device.

13. The electronic device of claim 10 , wherein the processing system is further configured to generate one or more correction coefficients from the second sensor data, and wherein generating the corrected biometric image comprises adjusting the biometric image based at least in part on the one or more correction coefficients.

14. The electronic device of claim 10, wherein the processing system is further configured to generate a corrected image from the second sensor data, and wherein generating the corrected biometric image comprises comparing the biometric image to the corrected image. 15 . The electronic device of claim 10 , wherein the processing system is further configured to acquire the second sensor data while displaying the calibration image on the display device.

16. A processing system for a biometric sensing device, the processing system comprising: A sensor module comprising a sensor circuit, the sensor module being configured to: receiving first sensor data from a plurality of sensing elements; as well as receiving second sensor data from the plurality of sensing elements while displaying a calibration image on a display device, wherein the calibration image includes elements associated with pixels of the display device, and wherein the second sensor data corresponds to the calibration image; as well as Identify modules that are configured to: generating a biometric image from the first sensor data; and A corrected biometric image is generated based at least in part on the biometric image and the second sensor data.

17. The processing system of claim 16, wherein the determination module is further configured to authenticate a user based at least in part on the corrected biometric image.

18. The processing system of claim 16, wherein the determination module is further configured to generate one or more correction coefficients from the second sensor data, and wherein generating the corrected biometric image comprises adjusting the biometric image based at least in part on the one or more correction coefficients.

19. The processing system of claim 16, wherein the determination module is further configured to generate a corrected image from the second sensor data, and wherein generating the corrected biometric image comprises comparing the biometric image to the corrected image.

20. The processing system of claim 16, wherein the sensor module is further configured to receive the first sensor data by detecting light reflected by one or more features of an input object.

Images