Image compensation for fingerprint sensors deployed in flexible devices

Abstract

Some methods can involve receiving fingerprint image data and a first set of background image data from a fingerprint sensor; and determining first-processed fingerprint image data via subtraction of the first set of background image data from the fingerprint image data. Some methods can involve obtaining force data corresponding to a force applied to the fingerprint sensor at a time of obtaining the fingerprint image data. Some methods can involve obtaining a second set of background image data corresponding to the force data. Some methods can involve determining second-processed fingerprint image data based at least in part on the first-processed fingerprint image data and the second set of background image data; and outputting the second-processed fingerprint image data. In some examples, determining the second-processed fingerprint image data can involve a machine learning model. Some examples can involve estimating residual noise based on the force data.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Application No. 18 / 585,744 (Case No. 2306515 / QUALP621US), filed December 19, 2023, entitled “IMAGE COMPENSATION FORFINGERPRINT SENSOR DEPLOYED IN A FLEXIBLE DEVICE,” which is incorporated herein by reference. This application also relates to International Application (PCT) No. PCT / CN2023 / 139721 (Case No. 2305700WO01 / QUALP620WO), filed December 19, 2023, entitled “FORCE-COMPENSATED FINGERPRINT IMAGING FOR FLEXIBLE DEVICE IMPLEMENTATIONS,” which is incorporated herein by reference. Technical Field

[0002] This disclosure relates in its entirety to flexible devices including fingerprint sensors (such as flexible display devices) and methods of using such devices. Related technical descriptions

[0003] Fingerprint sensors, including but not limited to ultrasonic fingerprint sensors, have been incorporated into devices such as smartphones, ATMs, and automobiles for user authentication. Some fingerprint sensors are deployed in flexible display devices such as flexible mobile phones. Obtaining satisfactory fingerprint image data from fingerprint sensors deployed in flexible display devices can be challenging. Improved devices and methods for operating such devices are desirable. Summary of the Invention

[0004] The systems, methods, and apparatus disclosed herein each have several innovative aspects, and no single aspect is solely responsible for the desired properties disclosed herein.

[0005] Some of the innovative aspects of the subject matter described in this disclosure can be implemented in a method. The method may involve receiving fingerprint image data from a fingerprint sensor by a control system. The method may involve obtaining a first set of background image data by the control system. The method may involve determining first-processed fingerprint image data by the control system by subtracting the first set of background image data from the fingerprint image data. The method may involve obtaining force data corresponding to a force applied to the fingerprint sensor when the fingerprint image data is obtained by the control system. The method may involve obtaining a second set of background image data corresponding to the force by the control system. The method may involve determining second-processed fingerprint image data by the control system based at least in part on the first-processed fingerprint image data and the second set of background image data. The method may involve outputting the second-processed fingerprint image data.

[0006] In some examples, determining the fingerprint image data for this second processing may involve a machine learning model.

[0007] According to some examples, determining that the fingerprint image data of the second processing may involve providing the fingerprint image data of the first processing and the second set of background image data to a trained neural network implemented by the control system.

[0008] In some examples, the fingerprint image data can be obtained from an ultrasonic fingerprint sensor. In some such examples, the method may involve obtaining multiple sets of fingerprint image data from the fingerprint sensor, each set corresponding to a different distance gate delay. Determining the fingerprint image data for the second processing may be based in part on at least one set of fingerprint image data from the multiple sets of fingerprint image data.

[0009] According to some examples, the method may involve estimating residual noise in the first set of background image data, the second set of background image data, or both. In some such examples, the method may involve determining fingerprint image data for an additional set of processing based at least in part on the fingerprint image data from the first processing and the residual noise estimate, or based on the fingerprint image data from the second processing and the residual noise estimate.

[0010] In some examples, the estimation of the residual noise may be based at least in part on the force data. According to some examples, estimating the residual noise may involve feeding the fingerprint image data and noise data to a trained neural network implemented by the control system. In some examples, the noise data may correspond to a structure within or near the fingerprint sensor. According to some examples, the noise data may correspond to a structure within or near a specific region of the fingerprint sensor. In some examples, the structure may correspond to a patterned backing layer of the fingerprint sensor or a patterned backing layer near the fingerprint sensor.

[0011] Other inventive aspects of the subject matter described in this disclosure can be implemented in an apparatus. The apparatus may include a fingerprint sensor and a control system. In some examples, at least a portion of the control system may be electrically connected to the fingerprint sensor. In some embodiments, a mobile device may be or may include the apparatus. For example, a mobile device may include the apparatus disclosed herein.

[0012] The control system may include one or more general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or combinations thereof. According to some examples, the control system may be configured to: receive fingerprint image data from the fingerprint sensor; obtain a first set of background image data; determine first-processed fingerprint image data by subtracting the first set of background image data from the fingerprint image data; obtain force data corresponding to the force applied to the fingerprint sensor when the fingerprint image data is obtained; obtain a second set of background image data corresponding to the force; determine second-processed fingerprint image data based at least in part on the first-processed fingerprint image data and the second set of background image data; and output the second-processed fingerprint image data.

[0013] Based on some examples, it is determined that the fingerprint image data of the second processing may involve a machine learning model. In some examples, determining that the fingerprint image data of the second processing may involve providing the fingerprint image data of the first processing and the second set of background image data to a trained neural network implemented by the control system.

[0014] According to some examples, the fingerprint sensor may be an ultrasonic fingerprint sensor or may include the ultrasonic fingerprint sensor. In some such examples, the control system may be further configured to: acquire multiple sets of fingerprint image data from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; and determine the fingerprint image data for the second processing in part based on at least one set of fingerprint image data from the multiple sets of fingerprint image data.

[0015] In some examples, the control system may be further configured to estimate residual noise in the first set of background image data, the second set of background image data, or both.

[0016] According to some examples, the control system may be further configured to determine fingerprint image data for an additional set of processing based at least in part on the fingerprint image data and residual noise estimation of the first processing, or based on the fingerprint image data and residual noise estimation of the second processing. In some examples, the estimation of the residual noise may be based at least in part on the force data. According to some examples, estimating the residual noise may involve feeding the fingerprint image data and noise data to a trained neural network implemented by the control system.

[0017] In some examples, the device may include a backing layer adjacent to the fingerprint sensor, and the noise data may correspond to the backing layer. According to some examples, the backing layer may be a patterned backing layer. In some examples, the device may include a reinforcing layer. According to some examples, the device may be a foldable mobile device.

[0018] Some or all of the operations, functions, and / or methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. Therefore, some innovative aspects of the subject matter described herein may be implemented in one or more non-transitory media on which software is stored. For example, the software may include instructions for controlling one or more devices to perform one or more methods disclosed herein. Attached Figure Description

[0019] Details of one or more specific embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. It should be noted that the relative dimensions in the following drawings may not be drawn to scale. The same reference numerals and names in different drawings denote the same elements.

[0020] Figure 1 This is a block diagram illustrating example components of a device according to some disclosed specific implementations.

[0021] Figure 2A and Figure 2B It shows the way Figure 1 An example of a cross-section of a specific implementation of the device.

[0022] Figure 3 An example of a specific implementation of a foldable display is shown.

[0023] Figure 4 It is a flowchart that indicates a registration process based on an example.

[0024] Figure 5 It is a flowchart that indicates a fingerprint authentication process based on an example.

[0025] Figure 6 The diagram illustrates a box involved in training a neural network to remove backing patterns from fingerprint image data, based on an example.

[0026] Figure 7A and Figure 7B An example of image data corresponding to different forces applied to a test pattern is shown from the fingerprint sensor system 102.

[0027] Figure 8 Another example of training a neural network to remove backing patterns from fingerprint image data is shown.

[0028] Figure 9 An example of training a neural network to extract noise from fingerprint image data is shown.

[0029] Figure 10 Another example of training a neural network to extract noise from fingerprint image data is shown.

[0030] Figure 11 The standard procedure for subtracting background image data from fingerprint image data is described.

[0031] Figure 12 An example of a currently published process for removing background image data from fingerprint image data is described.

[0032] Figure 13 Another currently disclosed process for removing background image data from fingerprint image data is described.

[0033] Figure 14 Another currently disclosed process for removing background image data from fingerprint image data is described.

[0034] Figure 15A and Figure 15B An additional currently disclosed process for removing background image data from fingerprint image data is described.

[0035] Figure 16 This is a flowchart providing example frames of some of the methods disclosed in this article.

[0036] Figure 17A , Figure 17B , Figure 17C and Figure 17D An example of a force sensor integrated into the circuitry of an ultrasonic fingerprint sensor is shown. Detailed Implementation

[0037] For the purpose of describing the innovative aspects of this disclosure, the following description relates to certain specific embodiments. However, those skilled in the art will readily recognize that the teachings herein can be applied in many different ways. The specific embodiments described can be implemented in any device, apparatus, or system, including biometric systems as disclosed herein. Furthermore, it is contemplated that the described implementations can be included in or associated with a variety of electronic devices, such as, but not limited to: mobile phones, Internet-enabled multimedia cellular phones, mobile TV receivers, wireless devices, smartphones, smart cards, wearable devices (such as bracelets, armbands, wristbands, rings, headbands, patches, etc.), Bluetooth devices, etc. ® Devices, Personal Data Assistants (PDAs), Wireless Email Receivers, Handheld or Portable Computers, Netbooks, Notebooks, Smartbooks, Tablets, Printers, Copiers, Scanners, Fax Equipment, Global Positioning System (GPS) Receivers / Navigators, Cameras, Digital Multimedia Players (such as MP3 Players), Camcorders, Game Consoles, Wristwatches, Clocks, Calculators, Television Monitors, Flat Panel Displays, Electronic Reading Devices (e.g., E-readers), Mobile Health Devices, Computer Monitors, Automatic Displays (including odometer and speedometer displays, etc.), Cockpit Controls and / or Displays, Cameras This includes machine-view displays (such as displays for rearview cameras in vehicles), electronic photographs, electronic billboards or signs, projectors, building structures, microwave ovens, refrigerators, stereo systems, cassette recorders or players, DVD players, CD players, VCRs, radios, portable storage chips, washing machines, dryers, washer / dryer units, ATMs, parking timers, packaging (such as in electromechanical systems (EMS) applications, including microelectromechanical systems (MEMS) applications, along with non-EMS applications), aesthetic structures (such as image displays on a piece of jewelry or clothing), and various EMS devices. The teachings herein can also be applied to applications such as, but not limited to, electronic switching devices, radio frequency filters, sensors, accelerometers, gyroscopes, motion sensing devices, magnetometers, inertial components for consumer electronics, components of consumer electronics products, car doors, steering wheels or other automotive parts, varactor diodes, liquid crystal devices, electrophoresis equipment, drive systems, manufacturing processes, and electronic test equipment. Therefore, these teachings are not intended to be limited to the specific implementations depicted in the figures, but rather have broad applicability, as will be obvious to those skilled in the art.

[0038] As noted above, obtaining satisfactory fingerprint image data from a fingerprint sensor deployed in a flexible display device can be challenging. (As used herein, the term "finger" can refer to any finger, including the thumb. Accordingly, the term "fingerprint" as used herein can refer to the fingerprint of any finger, including the thumb. Data received from a fingerprint sensor may sometimes be referred to herein as "fingerprint sensor data," "fingerprint image data," etc., although the data is typically received from the fingerprint sensor system in the form of electrical signals. Therefore, such image data may not necessarily be perceived as an image by a human without additional processing.)

[0039] Flexible display devices may include, for example, flexible organic light-emitting diode (OLED) displays without a stabilizing backing or a stabilizing display cover glass. To improve device stability, flexible display devices may include reinforcing layers, such as metal layers, which can pose additional challenges to obtaining satisfactory fingerprint image data. Some flexible display devices may include a flexible backing layer close to the fingerprint sensor. In some cases, an air gap may exist between the flexible backing layer and the fingerprint sensor. The flexible backing layer can cause unexpected background patterns in the fingerprint image data, especially when forces are applied due to finger pressure, finger lifting, device folding, device rolling, etc. Evaluating frequent changes in the background image (obtained when the finger is not pressing the fingerprint sensor) consumes significant power and computational resources and can negatively impact the reliability of the fingerprint sensor.

[0040] In some implementations, the device may include a patterned flexible backing layer adjacent to the fingerprint sensor. In some cases, an air gap may exist between the patterned flexible backing layer and the fingerprint sensor. By monitoring the force applied to the fingerprint sensor and the corresponding expected change in the background pattern in the fingerprint image data caused by the patterned flexible backing layer, the control system can filter out such background patterns through image processing, such as image reconstruction and denoising. Some examples may involve estimating residual noise based at least in part on the received force data and the known pattern of the backing layer. In some examples, estimating residual noise may involve feeding the fingerprint image data and noise data to a trained neural network implemented by the control system.

[0041] Specific embodiments of the subject matter described herein can be implemented to achieve one or more of the following potential advantages. According to some examples, when different forces are applied to a fingerprint sensor, fingerprint image quality can be improved by filtering out background patterns in the fingerprint image data caused by the patterned flexible backing layer. By monitoring the current finger force or current finger pressure, in some examples, a relatively higher level of fingerprint image quality can be obtained than without such force or pressure data. (Although force and pressure are different, since pressure is force per unit area, the terms "force" and "pressure" are sometimes used interchangeably herein.) Higher levels of fingerprint image quality can lead to more accurate fingerprint scans, lower false rejection rates, and higher user satisfaction. Some of the disclosed examples may involve examples where relatively less background image data is obtained compared to previously implemented methods for processing fingerprint image data obtained from flexible devices. Such embodiments can result in relatively lower power and computational resource consumption and can improve the lifespan and reliability of the fingerprint sensor.

[0042] Figure 1 This is a block diagram illustrating example components of a device according to some of the disclosed embodiments. As with the other disclosed embodiments, Figure 1 The types, quantities, and arrangements of the components shown are for illustrative purposes only. Other embodiments may have different types, quantities, and / or arrangements of components. In this example, device 101 includes a fingerprint sensor system 102 and a control system 106. Some embodiments of device 101 may include an interface system 104, a display system, and / or a force sensor 110.

[0043] The fingerprint sensor system 102 can be any suitable type of fingerprint sensor system, such as an optical fingerprint sensor system, a capacitive fingerprint sensor system, a resistive fingerprint sensor system, a radio frequency-based fingerprint sensor system, etc. In some examples, the fingerprint sensor system can be or may include an ultrasonic fingerprint sensor system. Detailed examples are provided herein.

[0044] Some specific implementations of device 101 may include interface system 104. In some examples, interface system 104 may include a wireless interface system. In some specific implementations, interface system 104 may include a user interface system, one or more network interfaces, one or more interfaces between control system 106 and memory system, and / or one or more interfaces between control system 106 and one or more external device interfaces (e.g., ports or application processors).

[0045] Interface system 104 may be configured to provide communication between the components of device 101 (which may include wired or wireless communication, such as electrical communication, radio communication, etc.). In some such examples, interface system 104 may be configured to provide communication between control system 106 and fingerprint sensor system 102, between control system 106 and display system 108 (if present), and between control system 106 and force sensor 110 (if present). According to some such examples, interface system 104 may, for example, couple at least a portion of control system 106 to fingerprint sensor system 102 (and display system 108 and / or force sensor 110, if present) via a conductive material such as conductive metal wires or traces.

[0046] According to some examples, interface system 104 may be configured to provide communication between device 101 and other devices and / or humans. In some such examples, interface system 104 may include a user interface system having one or more user interfaces. The user interface system may, for example, include one or more speakers, touch and / or gesture sensor systems, haptic feedback systems, etc. Although in Figure 1 Although not shown in this document, the optional display system 108 can be considered as part of the interface system 104.

[0047] In some examples, interface system 104 may include one or more network interfaces and / or one or more external device interfaces (such as one or more Universal Serial Bus (USB) interfaces and / or Serial Peripheral Interface (SPI)). In some implementations, device 101 may also include a memory system in addition to the memory that control system 106 may include. In some examples, interface system 104 may include at least one interface between control system 106 and memory system.

[0048] Control system 106 may include one or more general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or combinations thereof. According to some examples, control system 106 may include dedicated components for controlling fingerprint sensor system 102 (and display system 108 and / or force sensor 110, if present). Control system 106 may also include (and / or be configured to communicate with) one or more memory devices such as one or more random access memory (RAM) devices, read-only memory (ROM) devices, etc. Therefore, device 101 may have a memory system including one or more memory devices, but... Figure 1The memory system is not shown. In some implementations, the functionality of the control system 106 may be partitioned among one or more controllers or processors, such as between a dedicated sensor controller and an application processor of a mobile device. Some examples are described below.

[0049] Force sensor 110 (if present in device 101) may be a piezoresistive sensor, a capacitive sensor, a thin-film sensor (e.g., a polymer-based thin-film sensor), or another suitable type of force sensor. If force sensor 110 includes a piezoresistive sensor, the piezoresistive sensor may comprise silicon, metal, polycrystalline silicon, and / or glass. In some instances, fingerprint sensor system 102 and force sensor 110 may be mechanically coupled. In some such examples, force sensor 110 may be integrated into the circuitry of fingerprint sensor system 102. Several examples are disclosed herein. However, in other specific implementations, force sensor 110 may be separate from fingerprint sensor system 102. In some examples, fingerprint sensor system 102 and force sensor 110 may be indirectly coupled. For example, fingerprint sensor system 102 and force sensor 110 may each be coupled to a portion of device 101. In some such examples, fingerprint sensor system 102 and force sensor 110 may each be coupled to a portion of control system.

[0050] However, some implementations may not include a force sensor 110 separate from the fingerprint sensor system 102. In some such examples, the control system 106 may be configured to perform force detection and / or pressure detection based at least in part on fingerprint sensor data from the fingerprint sensor system 102.

[0051] In some embodiments, the device may include a display stack comprising a display system 108. According to some examples, the display stack may be a foldable display stack including a reinforcement 112 and display stack layers. The reinforcement 112 (when present) may have a relatively high acoustic impedance, for example, 10 MRayl or greater. In some embodiments, the reinforcement 112 may be or may include a metal layer (e.g., a stainless steel layer with an acoustic impedance of approximately 47 MRayl). In some examples, the display stack layers may include layers of light-emitting diode (LED) displays (such as organic light-emitting diode (OLED) displays). Some examples of display stack layers are provided in this disclosure.

[0052] In some examples, device 101 may include a backing 114, which may also be referred to herein as backing layer 114. According to some examples, backing layer 114 may have a pattern on one or more surfaces. For example, backing layer 114 may have a pattern on the surface of a fingerprint sensor adjacent to the fingerprint sensor system 102. Control system 106 may be "aware" of the pattern of backing layer 114. In other words, backing layer pattern data indicating or corresponding to the pattern of backing layer 114 may be stored in a data structure of memory accessible by control system 106. In some cases, backing layer 114 may be described as part of a fingerprint sensor stack and / or part of fingerprint sensor system 102.

[0053] Device 101 can be used in a variety of different contexts, some examples of which are disclosed herein. For example, in some embodiments, a mobile device may include at least a portion of device 101. In some embodiments, a wearable device may include at least a portion of device 101. Wearable devices may be, for example, wristbands, armbands, rings, headbands, or patches. In some embodiments, control system 106 may reside in more than one device. For example, a portion of control system 106 may reside in a wearable device, and another portion of control system 106 may reside in another device such as a mobile device (e.g., a smartphone). In some such examples, interface system 104 may also reside in more than one device.

[0054] Figure 2A and Figure 2B It shows the way Figure 1 An example of a cross-section of a portion of a specific embodiment of the device. As with other disclosed embodiments, Figure 2A and Figure 2B The types, quantities, and arrangements of components shown are for illustrative purposes only. Other specific implementations may include components of different types, quantities, and / or arrangements.

[0055] In these examples, finger 205 is shown as the outer surface 215 of touch device 101, which in this example is the outer surface of a polyethylene terephthalate (PET) layer. According to these examples, an optically clear adhesive (OCA) layer bonds an ultra-thin glass (UTG) layer to the PET layer and the polarizer layer of display system 108. Here, the layers of display system 108 also include a display panel and a protective film layer, all of which in this example are bonded together by a pressure-sensitive adhesive. According to these examples, display system 108 is bonded to reinforcing layer 112 via another PSA layer. Reinforcing layer 112 and display system 108 can be considered as part of a display stack, partly because the reinforcing layer helps provide structural integrity to display system 108.

[0056] In these examples, the fingerprint sensor system 102 includes a fingerprint sensor 202 and force sensing portions 210a and 210b. Therefore, in these examples, the fingerprint sensor system 102 is configured for both force sensing and fingerprint acquisition. References below... Figures 17A to 17D A more detailed example of a fingerprint sensor system configured for both force sensing and fingerprint acquisition is described. In some alternative examples, the device may include a force sensor 110 separate from the fingerprint sensor system 102.

[0057] According to these examples, device 101 includes a backing layer 114 separated from fingerprint sensor 202 and force sensing portions 210a and 210b by air gap 220. In these examples, backing layer 114 is flexible. In some examples, backing layer 114 may include a foam, such as polystyrene foam, expanded polystyrene (EPS) foam, or a similar material. Figure 2B In the example shown, the backing layer 114 has a patterned surface 225 adjacent to the fingerprint sensor 202 and the force sensing portions 210a and 210b. Here, the patterned surface 225 includes angular protrusions 222 and rounded protrusions 224 separated by flat portions 226. In other examples, the patterned surface 225 may include other shapes, other patterns, or both. In some examples, backing layer pattern data indicating or corresponding to the pattern of the patterned surface 225 may be stored in a data structure of memory accessible by the control system 106 (not shown). For example, the backing layer pattern data may include information about the location of the angular protrusions 222 and the rounded protrusions 224, information about the size of the angular protrusions 222 and the rounded protrusions 224, etc.

[0058] Figure 3 An example of a specific implementation of a foldable display is shown. According to this example, the device 101 can be folded along the hinge region 305. In this example, Figure 2B An example of the fingerprint sensor system 102 and the backing layer 114 is shown in dashed outline, indicating that they are within the device 101. In this example, Figure 2B The reinforcing member 112, components of the display system 108, etc., reside between the outer surface 215, the fingerprint sensor system 102, and the backing layer 114. According to this example, the backing layer 114 extends across an area larger than that occupied by the fingerprint sensor system 102.

[0059] Figure 4 It is a flowchart that indicates a registration process based on an example. Figure 4 The box can be, for example, made of Figure 1 The control system 106 (at least partially) performs this. As with other methods disclosed herein, Figure 4The method 400 outlined herein may include more or fewer boxes than those indicated. Furthermore, the boxes of the methods disclosed herein are not necessarily executed in the indicated order. In some examples, some boxes of the methods disclosed herein may be executed concurrently.

[0060] In this example, the registration process begins at box 405. For example, box 405 may involve presenting one or more graphical user interfaces (GUIs) corresponding to the registration process on a display device, receiving user input, etc. For example, box 405 may involve: presenting one or more GUIs prompting the user to initiate the registration process; and receiving user input confirming that the user is ready and willing to participate in the registration process. For example, box 405 may involve presenting a GUI that prompts the user to place one or more fingers on a fingerprint sensor area indicated on the display device. This corresponds to the “Finger Press” text between boxes 405 and 410.

[0061] For simplicity, the following discussion will focus on the process of registering a single finger. Alternative examples may involve registering multiple fingers simultaneously.

[0062] In this example, box 410 relates to: detecting a finger press event on the fingerprint sensor area of ​​the fingerprint sensor system 102; and determining the area of ​​the finger press. According to this example, box 415 relates to receiving force sensor data from a force sensor corresponding to the finger press. In some examples, the force sensor may be part of the fingerprint sensor system 102, while in other examples, the force sensor may be separate from the fingerprint sensor system 102.

[0063] According to this example, box 420 relates to: scanning a finger over the fingerprint sensor area; and obtaining fingerprint image data from the finger. In some examples, boxes 415 and 420 may be performed simultaneously or substantially simultaneously. In this example, box 425 relates to denoising the fingerprint image data obtained in box 420. For example, box 425 may relate to subtracting background or "air" image data from the fingerprint image data. Background image data is typically obtained when there is no finger or other object on or near the fingerprint sensor area.

[0064] In this example, box 430 relates to generating a finger force map. For example, a finger force map can be generated based on force data obtained in box 415 and the area of ​​finger pressure, which can be determined in boxes 410 and / or 415. In some examples, the finger force map may simply indicate the average finger force over the area of ​​finger pressure. In other examples, the finger force map indicates varying forces over the area of ​​finger pressure, such as the area of ​​highest finger force and one or more surrounding areas with lower finger force.

[0065] According to this example, box 435 relates to classifying the backing pattern. In this example, box 435 relates to classifying the backing pattern at least in part based on the area of ​​finger pressure and / or the finger force map generated in box 430. For example, box 435 may relate to determining the area of ​​the backing pattern corresponding to the area of ​​finger pressure. For example, box 435 may relate to searching or querying a data structure in memory that stores backing layer pattern data. The backing layer pattern data may indicate or correspond to a pattern of the backing layer on a surface adjacent to the fingerprint sensor system 102.

[0066] In this example, box 440 relates to denoising one or more backing patterns in the denoised fingerprint image data output by box 425, based at least in part on the finger force map generated in box 430 and the backing pattern classification in box 435. The backing pattern can be considered a type of noise and is sometimes referred to herein as noisy data, or simply a type of noisy data. In some examples, box 435 may relate to searching or querying a memory data structure storing corresponding backing patterns from the finger force and fingerprint image data. According to some examples, the data structure including corresponding backing patterns from the finger force and fingerprint image data may be obtained at least in part during a factory calibration process. In some examples, the data structure including corresponding backing patterns from the finger force and fingerprint image data may be obtained at least in part in a manner such as... Figure 4 The registration process is obtained during the registration process. According to some examples, the data structure may include finger force, the corresponding backing pattern in the fingerprint image data, and temperature. This can be advantageous because the material properties of the backing layer can vary with temperature. In some examples, box 435 may be executed by a trained neural network implemented by a control system.

[0067] According to this example, box 445 relates to image post-processing for additional image enhancement. In this example, the image post-processing of box 445 is performed on the output of box 440 after denoising the background pattern using a force-based method. In some examples, the image post-processing of box 445 may involve wavelet-based denoising, filter-based denoising, multi-image fusion, fingerprint feature detection and enhancement, or a combination thereof.

[0068] In this example, boxes 450 and 455 relate to determining whether the registration process is complete. For example, box 450 may relate to storing the currently acquired fingerprint template and determining whether the registration process for a specific finger is complete, and box 455 may relate to determining whether the entire registration process is complete. In some examples, box 450 or box 455 may relate to determining whether additional finger force data will be acquired for a specific finger. If the process is complete (box 460), registration data, which may include, but is not limited to, the fingerprint data template, can be stored in box 465. In some implementations, box 460 may relate to extracting fingerprint features, such as fingerprint details, from fingerprint image data acquired during the registration process. However, if it is determined in box 455 that the registration process is not complete, the process continues to box 470, which is called "finger lift". Between boxes 455 and 470, in some examples, a user prompt may be provided to the user to lift the finger currently being registered.

[0069] In this example, box 470 relates to "finger lift" determination. In other words, box 470 relates to determining when a finger involved in the registration process 400 has been lifted from, or is being lifted from, the fingerprint sensor area. In some cases, the finger may be lifted in response to a user prompt, which may include a GUI provided by device 101. According to this example, box 475 relates to obtaining finger force data from a force sensor corresponding to the "finger lift" event. In some examples, box 475 may relate to obtaining a single finger force, while in other examples, box 475 may relate to obtaining multiple (in other words, two or more) finger force data measurements when the finger is lifted from the fingerprint sensor area.

[0070] According to this example, box 480 involves obtaining one or more air images, also referred to herein as background images, via the fingerprint sensor system 102. The one or more background images are typically obtained when no finger is touching the fingerprint sensor area, and in this example, after the finger has been completely lifted from the fingerprint sensor area.

[0071] In this example, box 485 relates to generating a background force map. In this example, the finger has been lifted from the fingerprint sensor area, so there is currently no finger force. Therefore, box 485 may relate to generating a force map based on one or more finger forces measured during the "finger lift" event. For the time interval after the finger has been lifted from the fingerprint sensor area, especially in the area where finger force was just applied, residual deformation of the fingerprint sensor area, backing, etc., may still exist. In some examples, box 485 may relate to generating multiple force maps based on each of the multiple finger forces obtained in box 475.

[0072] According to this example, box 490 relates to updating the current background image data based on one or more aerial images obtained in box 480. In some examples, box 490 may involve: updating the background image data structure to include background image data corresponding to one or more aerial images obtained in box 480; and storing the updated background image data structure in memory. In this example, box 495 relates to image post-processing. For example, box 495 may involve filtering, fusion, etc. In this example, after box 495, the process proceeds to "finger press" box 410. Between boxes 495 and 410, in some examples, a user prompt may be provided to place the finger on the fingerprint sensor area.

[0073] Figure 5 It is a flowchart that indicates a fingerprint authentication process based on an example. Figure 5 The box can be, for example, made of Figure 1 The control system 106 executes (at least partially). As with other methods disclosed herein, method 500 may include more or fewer boxes than indicated. Furthermore, the boxes in the methods disclosed herein are not necessarily executed in the indicated order. In some examples, some boxes in the methods disclosed herein may be executed concurrently.

[0074] In this example, the fingerprint authentication process begins at box 505. For example, box 505 may involve presenting one or more graphical user interfaces (GUIs) corresponding to the fingerprint authentication process on a display device, receiving user input, etc. For example, box 505 may involve presenting one or more GUIs prompting the user to initiate fingerprint authentication. For example, box 505 may involve presenting a GUI that prompts the user to place one or more fingers on the fingerprint sensor area indicated on the display device. This corresponds to the "finger press down" branch following box 505. For simplicity, the following discussion will refer to boxes of fingerprint authentication processes involving a single finger. Alternative fingerprint authentication examples may involve multiple fingers.

[0075] In these examples, boxes 510 to 545 and 575 to 595 may correspond to Figure 4 Boxes 410 to 445 and 475 to 495 are executed as described above. However, in this example, box 550 involves attempting to match fingerprint features extracted from currently acquired fingerprint image data with fingerprint features extracted from fingerprint image data acquired during the fingerprint registration process. According to this example, box 555 involves determining whether the matching attempt in box 550 was successful. If so, access to the device can be provided. In this example, if it is determined in box 555 that the matching attempt in box 550 was successful, one or more data structures of the previously acquired fingerprint features are updated in box 560 and stored in box 565.

[0076] In this example, if it is determined in box 555 that the matching attempt in box 550 is unsuccessful, method 500 may involve providing the user with one or more additional fingerprint authentication attempts. In some examples, method 500 may involve providing the user with prompts via one or more GUIs for the user to lift a finger from the fingerprint sensor area, place a finger in the fingerprint sensor area, place another finger in the fingerprint sensor area, or take other actions.

[0077] Figure 6 The diagram illustrates a bounding box involved in training a neural network to remove backing patterns from fingerprint image data, based on an example. In this example, Figure 6 Clean fingerprint image data 605 and reference fingerprint image data 625 represent fingerprint image data corresponding to a fingerprint image that does not include noise corresponding to the backing pattern. According to this example, point 610 is added to the clean fingerprint image data 605 to produce input fingerprint image data 615 provided to the neural network 620. Point 610 may, for example, correspond to a backing pattern caused by a specific portion of the backing layer 114. In some examples, point 610 may correspond to a backing pattern caused by a specific portion of the backing layer 114 when a specific force or range of force is applied to a region of the fingerprint sensor. In this example, the neural network 620 is an autoencoder deep learning network. Other implementations may involve other types of neural networks (such as, for example, U-Net or ResNet), more layers, fewer layers, etc.

[0078] In this example, the recovered data 630 generated by the neural network 620 is compared with reference fingerprint image data 625. The neural network 620 is updated based on the differences between the reference fingerprint image data 625 and the recovered data 630. For example, the neural network 620 can be trained until convergence. For example, convergence is achieved when one or more differences between the reference fingerprint image data 625 and the recovered data 630 are less than a threshold. In some examples, the neural network 620 can be trained using points 610 corresponding to each of a plurality of forces or corresponding to each of a plurality of force ranges until convergence is achieved. According to some examples, the neural network 620 can be trained using points 610 corresponding to each of a plurality of regions of the fingerprint sensor system 102 and / or the backing layer 114 until convergence is achieved.

[0079] Figure 7A and Figure 7B Examples of image data corresponding to different regions of a test pattern obtained from the fingerprint sensor system 102 are shown. In both examples, the device 101 includes a backing layer 114 having a patterned surface adjacent to the fingerprint sensor system 102. Figure 7A The example shown is an image obtained when no force is applied to the fingerprint sensor system 102. Figure 7BThe example shown is an image obtained when a force of 300 grams is applied to the fingerprint sensor system 102. Figure 7B In the study, numerous artifacts 705 caused by the patterned surface of the backing layer 114 can be observed when a force of 300 grams is applied.

[0080] Figure 8 Another example is shown of training a neural network to remove backing patterns from fingerprint image data. In this example, as... Figure 6 As shown, clean fingerprint image data 605 and reference fingerprint image data 625 represent fingerprint image data corresponding to a fingerprint image that does not include noise corresponding to the backing pattern, and point 610 is added to clean fingerprint image data 605 to produce input fingerprint image data 615 provided to neural network 620. (The above refers to...) Figure 6 The comments on the components also apply. Figure 8 The corresponding components.

[0081] Figure 8 The example is different Figure 6 The example, because Figure 8 This includes adding different noise 805. According to this example, the different noise 805 is similar to but different from point 610. In this example, the different noise 805 includes a parallelogram shape at a location corresponding to the point position of point 610. In other examples, the different noise 805 may include shapes other than parallelograms, such as squares, triangles, ellipses, stars, etc. In some alternative examples, the location of the different noise 805 may not exactly correspond to the point position of point 610. According to some examples, point 610 may correspond to an artifact produced by the backing layer 114 when a first force is applied to the fingerprint sensor system 102, and the different noise 805 may correspond to an artifact produced by the backing layer 114 when a second force is applied to the fingerprint sensor system 102. Here, the different noise 805 is provided separately to the neural network 620. In some alternative examples, the different noise 805 may be added to the input fingerprint image data 615.

[0082] Figure 9An example of training a neural network to extract noise from fingerprint image data is shown. In this example, clean fingerprint image data 605 does not include noise corresponding to a backing pattern. According to this example, dots and lines 910 are added to the clean fingerprint image data 605 to produce input fingerprint image data 915 provided to the neural network 620. Dots and lines 910 may, for example, correspond to a backing pattern caused by a specific portion of the backing layer 114. In some examples, dots and lines 910 may correspond to a backing pattern caused by a specific portion of the backing layer 114 when a specific force or range of force is applied to a region of the fingerprint sensor. In this example, the neural network 620 is an autoencoder deep learning network. Other specific implementations may involve other types of neural networks, more layers, fewer layers, etc.

[0083] Reference above Figure 6 and Figure 8 The examples described are different, in Figure 9 In the example shown, reference data 925 corresponds to point 910 and line 910. Therefore, in Figure 9 Instead of training the neural network 620 to produce a cleaner fingerprint image, the neural network 620 is trained to produce noise 940 for the extraction of points and lines similar to the reference data 925.

[0084] Figure 10 Another example of training a neural network to extract noise from fingerprint image data is shown. In this example, as in... Figure 9 In the process, dots and lines 910 are added to clean fingerprint image data 605 to produce input fingerprint image data 915 provided to neural network 620. Reference data 925 corresponds to dots and lines 910. (The above refers to...) Figure 9 The comments on the components also apply. Figure 10 The corresponding components.

[0085] Figure 10 The example is different Figure 9 The example, because Figure 10This includes adding different noise 1005. According to this example, different noise 1005 is similar to but different from the point and line 910. In this example, different noise 1005 includes a parallelogram shape at the location corresponding to the point position of the point and line 910. In other examples, different noise 1005 may include shapes other than parallelograms, such as squares, triangles, ellipses, stars, etc. According to this example, different noise 1005 also includes lines that are slightly different from the lines of the point and line 910. For example, in the upper right corner of different noise 1005, there are two lines, with the right line shorter than the left. In the upper right corner of point and line 910, there are also two lines. However, the right line is longer than the left. Here, different noise 1005 is provided separately to the neural network 620. In some alternative examples, different noise 1005 may be added to the input fingerprint image data 915.

[0086] Figure 11 A conventional process for subtracting background image data from fingerprint image data is described. The linear background subtraction module 1115 can be implemented, for example, by a control system according to instructions stored on one or more computer-readable media. In this example, the linear background subtraction module 1115 receives fingerprint image data 1110 and background image data 1105, and is configured to subtract the background image data 1105 from the fingerprint image data 1110 according to a linear subtraction process, and generate output fingerprint data 1120. Figures 11 to 13 In this context, "BG" represents background image data and "BG grid" is a background image corresponding to a temperature. For example, BG grids can be obtained at multiple temperatures. Some embodiments may involve interpolating between BG images obtained at intermediate temperatures to generate an estimate of the BG image over the entire temperature range. This temperature range may include all temperatures that can be used with a device including the disclosed embodiments, such as a mobile phone. Figures 11 to 13 In this context, "QDB" stands for "Fast Dynamic Background". In some examples, a QDB image can be an additional background or aerial image obtained when the noise in the air reaches a threshold level. Therefore, a QDB is typically a background image from a very recent time period.

[0087] Figure 12 An example of a currently disclosed process for removing background image data from fingerprint image data is described. Linear background subtraction module 1115 and nonlinear background subtraction module 1210 may be, for example, derived from... Figure 1 The control system 106 is implemented according to instructions stored on one or more computer-readable media. Figures 12 to 15B The linear background subtraction module 1115 shown is optional. Therefore, Figures 12 to 1 Some alternative implementations of the process shown in 5 may exclude the linear background subtraction module 1115.

[0088] According to this example, the nonlinear background subtraction module 1210 is or includes a neural network implemented by a control system. In this example, the neural network is a convolutional neural network (CNN), but in other examples, the neural network can be another type of neural network. In some examples, the neural network may be as described herein, for example, referenced... Figures 6 to 10 Training is performed using one or more graphs described in the diagram.

[0089] In this example, the linear background subtraction module 1115 receives fingerprint image data 1110 and background image data 1105, and is configured to subtract the background image data 1105 from the fingerprint image data 1110 according to a linear subtraction process, and provide output fingerprint data 1120 to the nonlinear background subtraction module 1210. According to this example, the CNN corresponding to the nonlinear background subtraction module 1210 is configured to receive the output fingerprint data 1120 and force data 1205 as input, and generate output fingerprint data 1220.

[0090] Figure 13 Another currently disclosed process for removing background image data from fingerprint image data is described. The linear background subtraction module 1115 and the linear / CNN background subtraction module 1310 can be, for example, derived from... Figure 1 The control system 106 is implemented according to instructions stored on one or more computer-readable media. According to this example, the linear / CNN background subtraction module 1310 is or includes a neural network implemented by the control system. In this example, the neural network is a convolutional neural network (CNN), but in other examples, the neural network can be another type of neural network. In some examples, the neural network may be as described herein, for example, referenced... Figures 6 to 10 Training is performed using one or more graphs described in the diagram.

[0091] In this example, the linear background subtraction module 1115 receives fingerprint image data 1110 and background image data 1105, and is configured to subtract the background image data 1105 from the fingerprint image data 1110 according to a linear subtraction process, and provide output fingerprint data 1120 to the linear / CNN background subtraction module 1310. According to this example, the CNN corresponding to the nonlinear background subtraction module 1210 is configured to receive the output fingerprint data 1120, force data 1205, and fingerprint image data 1305 as inputs, and generate output fingerprint data 1320. In this example, the fingerprint image data 1305 is obtained via an ultrasonic fingerprint sensor using multiple distance gate delays (RGDs). In this context, RGD is the time interval between transmitting ultrasonic waves and activating a receiver to detect ultrasonic waves reflected from a target object (such as a finger).

[0092] therefore, Figure 13 The process shown is very similar to Figure 12The process, except for: (1) in Figure 13 In the example shown, fingerprint image data 1305 is also input to a module that implements a neural network; and (2) a linear / CNN background subtraction module 1310 implements a linear background subtraction process, and a nonlinear background subtraction module 1210 implements a nonlinear background removal process.

[0093] Figure 14 Another currently disclosed process for removing background image data from fingerprint image data is described. As in the previous example, Figure 14 The module shown can be, for example, made by Figure 1 The control system 106 is implemented. In some such examples, Figure 14 The module shown can be made by Figure 1 The control system 106 is implemented according to instructions stored on one or more computer-readable media. In this example, the linear / CNN background subtraction module 1410 is... Figure 13 This is an instance of the linear / CNN background subtraction module 1310 and is or includes a neural network implemented by a control system. In this example, the neural network is a convolutional neural network (CNN), but in other examples, the neural network can be another type of neural network. In some examples, the neural network can be, for example, referenced herein. Figures 6 to 10 Training is performed using one or more graphs described in the diagram.

[0094] In this example, the linear background subtraction module 1115 receives fingerprint image data 1110 and background image data 1105, and is configured to subtract the background image data 1105 from the fingerprint image data 1110 according to a linear subtraction process, and provide output fingerprint data 1120 to the linear / CNN background subtraction module 1310. According to this example, the CNN corresponding to the nonlinear background subtraction module 1210 is configured to receive the output fingerprint data 1120, force data 1205, and fingerprint image data 1305 as inputs, and output fingerprint data 1320. In this example, the fingerprint image data 1305 is obtained via an ultrasonic fingerprint sensor using multiple range gate delays (RGDs).

[0095] therefore, Figure 14 The process shown is very similar to Figure 13 The process, except for the following: (1) Figure 14 The example shown includes a residual background estimation update module 1405 configured to provide an updated estimate 1407 of the residual background data based on force data 1205 and fingerprint data 1320; and (2) Figure 14The example shown includes a linear / CNN background subtraction module 1410, which is configured to produce output fingerprint data 1420 based on an updated estimate 1407 of fingerprint data 1320, force data 1205, and residual background data. The updated estimate 1407 of the residual background data can, for example, be based on a reference... Figure 4 and Figure 5 The process described is used to generate it.

[0096] Figure 15A and Figure 15B The currently disclosed process for removing background image data from fingerprint image data is described. As in the previous example, Figure 15A and Figure 15B The module shown can be, for example, made by Figure 1 The control system 106 is implemented. In some such examples, Figure 15A and Figure 15B The module shown can be made by Figure 1 The control system 106 is implemented according to instructions stored on one or more computer-readable media.

[0097] Besides reference Figure 14 In addition to the components described, Figure 15A and Figure 15B The examples shown also include a denoising module 1505. In these examples, the denoising module 1505 is configured to output denoised image data 1510 and extracted noisy image data 1512 based on force data 1205 and fingerprint data 1320. According to these examples, the denoising module 1505 is or includes a CNN. In other examples, the denoising module 1505 may be or may include another type of neural network. In still some other examples, the denoising module 1505 may be implemented without any type of neural network.

[0098] exist Figure 15A In the example shown, the denoising module 1505 is configured to output denoised image data 1510 to the linear / CNN background subtraction module 1410, and to output extracted noisy image data 1512 to the estimated residual background update module 1405. Figure 15B In the example shown, the denoising module 1505 is configured to output the extracted noisy image data 1512 to the estimated residual background update module 1405, but not to output the denoised image data 1510 to the linear / CNN background subtraction module 1410.

[0099] Figure 16 This is a flowchart providing example frames of some of the methods disclosed in this article. Figure 16 The box can be, for example, made of Figures 1 to 3The device 101 or similar device may be used. As with other methods disclosed herein, Figure 16 The method 1600 outlined herein may include more or fewer boxes than those indicated. Furthermore, the boxes of the methods disclosed herein are not necessarily executed in the indicated order. In some examples, some boxes of the methods disclosed herein may be executed concurrently.

[0100] According to this example, method 1600 is a method for processing fingerprint image data. In this example, box 1603 relates to receiving fingerprint image data from a fingerprint sensor (such as fingerprint sensor system 102) by a control system (such as control system 106). In some examples, box 1603 may serve as... Figure 5 This is performed as part of the "finger press down" series of operations shown and described in this article.

[0101] In this example, box 1605 relates to the acquisition of a first set of background image data by the control system. In some examples, the first set of background image data may be a previously acquired set of background image data already stored in a memory accessible by the control system.

[0102] According to this example, block 1607 relates to determining the fingerprint image data for the first processing by a control system via subtracting a first set of background image data from the fingerprint image data. Block 1607 may relate to any linear or nonlinear background removal process disclosed herein, or a similar process.

[0103] In this example, box 1609 relates to the control system obtaining force data corresponding to the force applied to the fingerprint sensor when acquiring fingerprint image data. According to some examples, this can be obtained from... Figure 1 Force sensor 110 acquires force data; depending on the specific implementation, this force sensor may or may not be part of the fingerprint sensor system 102. In some examples, blocks 1603 and 1609 may perform simultaneously or substantially simultaneously.

[0104] According to this example, box 1611 relates to obtaining a second set of background image data by a control system. In this example, the second set of background image data corresponds to the force obtained in box 1609. According to some examples, the second set of background image data may serve as... Figure 5 This is obtained as part of the "finger lift" series of operations shown and described herein. For example, the second set of background image data can be obtained within the time interval after the finger is lifted and after the finger applies force to the fingerprint sensor.

[0105] In this example, block 1613 relates to determining fingerprint image data for a second process by a control system based at least in part on fingerprint image data from a first process and a second set of background image data. According to this example, block 1615 relates to outputting the fingerprint image data from the second process.

[0106] According to some examples, determining the fingerprint image data for the second processing may involve a machine learning model. In some examples, determining the fingerprint image data for the second processing may involve providing the fingerprint image data from the first processing and the second set of background image data to a trained neural network implemented by the control system. In some such examples, the fingerprint image data may be obtained from an ultrasonic fingerprint sensor. According to some such examples, the method may involve obtaining multiple sets of fingerprint image data from the fingerprint sensor, each set corresponding to a different distance gate delay. In some such examples, the method may involve determining the fingerprint image data for the second processing based at least in part on at least one set of fingerprint image data from the multiple sets of fingerprint image data.

[0107] In some examples, the method may involve estimating residual noise in the first set of background image data, the second set of background image data, or both. In some such examples, the method may involve determining fingerprint image data for an additional set of processes based at least in part on the fingerprint image data from the first process and the residual noise estimate. In some examples, the method may involve determining fingerprint image data for an additional set of processes based at least in part on the fingerprint image data from the second process and the residual noise estimate. According to some examples, estimating the residual noise may be based at least in part on the force data. In some examples, estimating the residual noise may involve feeding the fingerprint image data and noise data to a trained neural network implemented by the control system.

[0108] According to some examples, the noise data may correspond to a structure within the fingerprint sensor, or to a structure near the fingerprint sensor. In some examples, the noise data may correspond to a structure within or near a specific area of ​​the fingerprint sensor. This structure may, for example, correspond to at least a portion of the patterned backing layer of the fingerprint sensor or at least a portion of the patterned backing layer near the fingerprint sensor.

[0109] Figure 17A , Figure 17B , Figure 17C and Figure 17D An example of a force sensor integrated into the circuitry of an ultrasonic fingerprint sensor is shown. Figure 17A , Figure 17B , Figure 17C and Figure 17D The specific implementation shown is Figure 2B An example of the combination of force sensor 116 and ultrasonic fingerprint sensor 102 is shown. Figure 17AA cross-section is shown through an example of a metal-oxide-semiconductor field-effect transistor (MOSFET), which in this example is a complementary metal-oxide-semiconductor (CMOS). Figure 17A The image shows only a single n-type thin-film transistor (NTFT) and a single p-type TFT (PTFT). However, actual ultrasonic fingerprint sensors with this type of structure typically have many NTFT / PTFT pairs (e.g., tens of thousands of NTFT / PTFT pairs).

[0110] Depending on the specific implementation, Figure 17A The various conductive layers stacked as shown can be used in a pressure sensor. In some examples, a portion of the pixel electrode layer can be used in a pressure sensor. In other examples, a portion of the source / drain (S / D) electrode layer can be used in a pressure sensor. According to some specific embodiments, a portion of the gate electrode layer can be used in a pressure sensor. In some examples, a portion of the polysilicon (poly-Si) layer can be used in a pressure sensor. In some specific embodiments, the polysilicon layer may include low-temperature polysilicon (LTPS).

[0111] Figure 17B An example top view of an ultrasonic fingerprint sensor is shown. In this example, the sensor pixel array and sensor peripheral driver each include, for example, Figure 17A Several examples of CMOS are shown. In this example, a portion of the pixel electrode layer is configured as a conductive component for a pressure sensor. According to this specific embodiment, pins 1, pin 2, and connection portion 1703 of the pixel electrode layer are configured as pressure sensor electrodes. In this example, in Figure 17A Other pins labeled "Sensor Operation Pin to ASIC" can be used to connect the ultrasonic fingerprint sensor to corresponding components of the control system. Depending on the specific implementation, the control system may or may not include an ASIC. According to some examples, a pressure sensor may also be included as part of one or more layers of piezoelectric material incorporated into the ultrasonic fingerprint sensor.

[0112] Figure 17C It shows Figure 17B The image shows a perspective view of an ultrasonic fingerprint sensor. Figure 17C The cross-sectional line A / A' is also shown, which corresponds to Figure 17D The cross-section shown.

[0113] Figure 17D Through Figure 17C The image shows a simplified cross-section of an ultrasonic fingerprint sensor. Similar to... Figure 17A , Figure 17DThe example shows only a single NTFT / PTFT pair, while actual ultrasonic fingerprint sensors with this type of structure typically have many NTFT / PTFT pairs. The cross-sectional line A / A' is shown traversing the pixel electrode layer and, in this example, includes both the pixel electrode and the pressure sensor electrode. In an alternative example where a portion of a deeper layer (such as a portion of a source / drain (S / D) electrode layer, a portion of a gate electrode layer, or a portion of a polysilicon layer) is used to form the pressure sensor electrode, the device may include vias to connect the deeper layer to chip pins or other corresponding components of the control system.

[0114] Specific implementation examples are described in the following numbered clauses:

[0115] 1. A method for processing fingerprint image data, the method comprising: receiving fingerprint image data from a fingerprint sensor by a control system; obtaining a first set of background image data by the control system; determining first processed fingerprint image data by the control system by subtracting the first set of background image data from the fingerprint image data; obtaining force data by the control system corresponding to a force applied to the fingerprint sensor when the fingerprint image data is obtained; obtaining a second set of background image data corresponding to the force by the control system; determining second processed fingerprint image data by the control system based at least in part on the first processed fingerprint image data and the second set of background image data; and outputting the second processed fingerprint image data.

[0116] 2. The method according to Clause 1, wherein determining the fingerprint image data to be processed involves a machine learning model.

[0117] 3. The method according to Clause 1 or Clause 1, wherein determining the fingerprint image data of the second processing involves providing the fingerprint image data of the first processing and the second set of background image data to a trained neural network implemented by the control system.

[0118] 4. The method according to any one of claims 1 to 3, wherein the fingerprint image data is obtained from an ultrasonic fingerprint sensor, the method further comprising: obtaining multiple sets of fingerprint image data from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; and determining the fingerprint image data for the second processing in part based on at least one set of fingerprint image data from the multiple sets of fingerprint image data.

[0119] 5. The method according to any one of clauses 1 to 4, the method further comprising estimating residual noise in the first set of background image data, the second set of background image data, or both.

[0120] 6. The method according to Clause 5, the method further comprising determining an additional set of processed fingerprint image data based at least in part on the fingerprint image data of the first processed fingerprint and the residual noise estimate, or based on the fingerprint image data of the second processed fingerprint and the residual noise estimate.

[0121] 7. The method according to Clause 5 or Clause 6, wherein the estimation of the residual noise is based at least in part on the force data.

[0122] 8. The method according to any one of clauses 5 to 7, wherein estimating the residual noise involves providing the fingerprint image data and noise data to a trained neural network implemented by the control system.

[0123] 9. The method according to Clause 8, wherein the noise data corresponds to a structure within or near the fingerprint sensor.

[0124] 10. The method according to Clause 9, wherein the noise data corresponds to a structure within or near a specific area of ​​the fingerprint sensor.

[0125] 11. The method according to Clause 9 or Clause 10, wherein the structure corresponds to a patterned backing layer of the fingerprint sensor or a patterned backing layer near the fingerprint sensor.

[0126] 12. An apparatus comprising: a fingerprint sensor; and a control system configured to: receive fingerprint image data from the fingerprint sensor; obtain a first set of background image data; determine first processed fingerprint image data by subtracting the first set of background image data from the fingerprint image data; obtain force data corresponding to a force applied to the fingerprint sensor when the fingerprint image data is obtained; obtain a second set of background image data corresponding to the force; determine second processed fingerprint image data based at least in part on the first processed fingerprint image data and the second set of background image data; and output the second processed fingerprint image data.

[0127] 13. The apparatus according to Clause 12, wherein determining the fingerprint image data processed in the second manner involves a machine learning model.

[0128] 14. The apparatus according to Clause 12 or Clause 13, wherein determining the fingerprint image data processed by the second process involves providing the fingerprint image data processed by the first process and the second set of background image data to a trained neural network implemented by the control system.

[0129] 15. The apparatus according to any one of claims 12 to 14, wherein the fingerprint sensor is an ultrasonic fingerprint sensor or includes the ultrasonic fingerprint sensor, and wherein the control system is further configured to: obtain multiple sets of fingerprint image data from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; and determine the fingerprint image data for the second processing in part based on at least one set of fingerprint image data from the multiple sets of fingerprint image data.

[0130] 16. The apparatus according to any one of clauses 12 to 15, wherein the control system is further configured to estimate residual noise in the first set of background image data, the second set of background image data, or both.

[0131] 17. The apparatus according to Clause 16, wherein the control system is further configured to determine an additional set of processed fingerprint image data based at least in part on the fingerprint image data of the first processed fingerprint and the residual noise estimate, or based on the fingerprint image data of the second processed fingerprint and the residual noise estimate.

[0132] 18. The apparatus according to Clause 16 or Clause 17, wherein the estimation of the residual noise is based at least in part on the force data.

[0133] 19. The apparatus according to any one of clauses 16 to 18, wherein estimating the residual noise involves providing the fingerprint image data and noise data to a trained neural network implemented by the control system.

[0134] 20. The apparatus according to Clause 19, wherein the apparatus includes a backing layer adjacent to the fingerprint sensor and wherein the noise data corresponds to the backing layer.

[0135] 21. The apparatus according to Clause 20, wherein the backing layer is a patterned backing layer.

[0136] 22. The device according to any one of clauses 12 to 21, wherein the device includes a reinforcing layer.

[0137] 23. The device according to any one of clauses 12 to 22, wherein the device is a foldable mobile device.

[0138] 24. An apparatus comprising: a fingerprint sensor; and a control member configured to: receive fingerprint image data from the fingerprint sensor; obtain a first set of background image data; determine first processed fingerprint image data by subtracting the first set of background image data from the fingerprint image data; obtain force data corresponding to a force applied to the fingerprint sensor when the fingerprint image data is obtained; obtain a second set of background image data corresponding to the force; determine second processed fingerprint image data based at least in part on the first processed fingerprint image data and the second set of background image data; and output the second processed fingerprint image data.

[0139] 25. The apparatus according to Clause 24, wherein determining the fingerprint image data processed in the second manner involves a machine learning model.

[0140] 26. The apparatus according to Clause 24 or Clause 25, wherein determining the fingerprint image data of the second processing involves providing the fingerprint image data of the first processing and the second set of background image data to a trained neural network implemented by the control member.

[0141] 27. The apparatus according to any one of claims 24 to 26, wherein the fingerprint sensor is an ultrasonic fingerprint sensor or includes the ultrasonic fingerprint sensor, and wherein the control member is further configured to: obtain multiple sets of fingerprint image data from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; and determine the fingerprint image data for the second processing in part based on at least one set of fingerprint image data from the multiple sets of fingerprint image data.

[0142] 28. One or more non-transitory computer-readable media storing instructions for performing a method, the method comprising: receiving fingerprint image data from a fingerprint sensor by a control system; obtaining a first set of background image data by the control system; determining first processed fingerprint image data by the control system by subtracting the first set of background image data from the fingerprint image data; obtaining force data corresponding to a force applied to the fingerprint sensor when the fingerprint image data is obtained by the control system; obtaining a second set of background image data corresponding to the force by the control system; determining second processed fingerprint image data by the control system based at least in part on the first processed fingerprint image data and the second set of background image data; and outputting the second processed fingerprint image data.

[0143] 29. One or more non-transitory computer-readable media as described in Clause 28, wherein determining the fingerprint image data of the second processing involves a machine learning model.

[0144] 30. One or more non-transitory computer-readable media as described in Clause 28 or Clause 29, wherein determining the fingerprint image data of the second processing involves providing the fingerprint image data of the first processing and the second set of background image data to a trained neural network implemented by the control system.

[0145] As used in this article, the phrase “at least one of the items” refers to any combination of these items, including a single member. As an example, “at least one of a, b, or c” is intended to cover: a, b, c, ab, ac, bc, and abc.

[0146] The various exemplary logics, logic blocks, modules, circuits, and algorithmic processes described in conjunction with the specific implementations disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. The interchangeability of hardware and software has been broadly described in terms of functionality and illustrated in the aforementioned exemplary components, blocks, modules, circuits, and processes. Whether such functionality is implemented in hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0147] Hardware and data processing means for implementing the various exemplary logic, logic blocks, modules, and circuits described herein can be implemented or executed using general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic components, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, or any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In some specific implementations, specific processes and methods can be performed by circuitry specific to a given function.

[0148] In one or more aspects, the described functionality may be implemented in hardware, digital electronic circuits, computer software, firmware, including the structures disclosed in this specification and their structural equivalents or any combination thereof. Specific implementations of the subject matter described in this specification may also be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus.

[0149] If implemented in software, the functions can be stored as one or more instructions or codes on or transmitted via a computer-readable medium such as a non-transitory medium. The processes of the methods or algorithms disclosed herein can be implemented in a processor-executable software module that can reside on a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium capable of transferring a computer program from one location to another. Storage media can be any available medium accessible to a computer. By way of example and not limitation, non-transitory media can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible to a computer. Additionally, any connection can be appropriately referred to as a computer-readable medium. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically reproduce data, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operation of a method or algorithm may reside as a set of code and instructions or any combination of code and instructions on a machine-readable medium and a computer-readable medium that may be incorporated into a computer program product.

[0150] Various modifications to the specific embodiments described herein will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the specific embodiments shown herein, but is to be accorded the widest scope consistent with the claims, principles, and novel features disclosed herein. The word “exemplary” (if any) is used herein specifically to mean “serving as an example, instance, or illustration.” Any specific embodiment described herein as “exemplary” is not necessarily to be construed as superior to or better than other specific embodiments.

[0151] Certain features described in this specification in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as operating in certain combinations and even originally claimed in this way, one or more features from the claimed combination may be removed from that combination in some cases, and the claimed combination may be for sub-combinations or variations thereof.

[0152] Similarly, although operations are depicted in a specific order in the figures, this should not be construed as requiring such operations to be performed in the shown specific order or sequential order, or to perform all illustrated operations to achieve the desired result. In some environments, multitasking and parallel processing are advantageous. Furthermore, the separation of the various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. Additionally, other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired result.

[0153] It should be understood that unless features in any particular embodiment of the description are explicitly identified as incompatible with each other, or the surrounding context suggests that they are mutually exclusive and not easily combined in a complementary and / or supporting sense, the general conception and ideas of this disclosure may be selectively combined with specific features of those complementary embodiments to provide one or more comprehensive but slightly different technical solutions. Therefore, it should also be understood that the above description is given by way of example only and may be modified in detail within the scope of this disclosure.

Claims

1. A method for processing fingerprint image data, the method comprising: The control system receives fingerprint image data from the fingerprint sensor; The control system obtains the first set of background image data; The control system determines the first processed fingerprint image data by subtracting the first set of background image data from the fingerprint image data; The control system obtains force data corresponding to the force applied to the fingerprint sensor when the fingerprint image data is acquired; The control system obtains a second set of background image data corresponding to the force; The control system determines the fingerprint image data for the second processing based at least in part on the fingerprint image data processed by the first processing and the second set of background image data; as well as Output the fingerprint image data processed in the second step.

2. The method of claim 1, wherein determining the fingerprint image data processed in the second step involves a machine learning model.

3. The method of claim 1, wherein determining the fingerprint image data processed by the second process involves providing the fingerprint image data processed by the first process and the second set of background image data to a trained neural network implemented by the control system.

4. The method according to claim 1, wherein the fingerprint image data is obtained from an ultrasonic fingerprint sensor, and the method further comprises: Multiple sets of fingerprint image data are obtained from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; as well as The fingerprint image data for the second processing is determined in part based on at least one set of fingerprint image data from the plurality of sets of fingerprint image data.

5. The method of claim 1, further comprising estimating residual noise in the first set of background image data, the second set of background image data, or both.

6. The method of claim 5, further comprising determining an additional set of processed fingerprint image data based at least in part on the fingerprint image data of the first processed fingerprint and the residual noise estimate, or based on the fingerprint image data of the second processed fingerprint and the residual noise estimate.

7. The method of claim 5, wherein the estimation of the residual noise is based at least in part on the force data.

8. The method of claim 5, wherein estimating the residual noise involves providing the fingerprint image data and noise data to a trained neural network implemented by the control system.

9. The method of claim 8, wherein the noise data corresponds to a structure within or near the fingerprint sensor.

10. The method of claim 9, wherein the noise data corresponds to a structure within or near a specific area of ​​the fingerprint sensor.

11. The method of claim 9, wherein the structure corresponds to a patterned backing layer of the fingerprint sensor or a patterned backing layer near the fingerprint sensor.

12. An apparatus comprising: Fingerprint sensor; and The control system is configured to: Receive fingerprint image data from the fingerprint sensor; Obtain the first set of background image data; The first processed fingerprint image data is determined by subtracting the first set of background image data from the fingerprint image data; Obtain force data corresponding to the force applied to the fingerprint sensor when the fingerprint image data is acquired; Obtain a second set of background image data corresponding to the force; The fingerprint image data for the second processing is determined at least in part based on the fingerprint image data processed by the first processing and the second set of background image data; as well as Output the fingerprint image data processed in the second step.

13. The apparatus of claim 12, wherein determining the fingerprint image data processed in the second step involves a machine learning model.

14. The apparatus of claim 12, wherein determining the fingerprint image data processed by the second process involves providing the fingerprint image data processed by the first process and the second set of background image data to a trained neural network implemented by the control system.

15. The apparatus of claim 12, wherein the fingerprint sensor is an ultrasonic fingerprint sensor or includes the ultrasonic fingerprint sensor, and wherein the control system is further configured to: Multiple sets of fingerprint image data are obtained from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; and The fingerprint image data for the second processing is determined in part based on at least one set of fingerprint image data from the plurality of sets of fingerprint image data.

16. The apparatus of claim 12, wherein the control system is further configured to estimate residual noise in the first set of background image data, the second set of background image data, or both.

17. The apparatus of claim 16, wherein the control system is further configured to determine an additional set of processed fingerprint image data based at least in part on the fingerprint image data of the first processed fingerprint and the residual noise estimate, or based on the fingerprint image data of the second processed fingerprint and the residual noise estimate.

18. The apparatus of claim 16, wherein the estimation of the residual noise is based at least in part on the force data.

19. The apparatus of claim 16, wherein estimating the residual noise involves providing the fingerprint image data and noise data to a trained neural network implemented by the control system.

20. The apparatus of claim 19, wherein the apparatus includes a backing layer adjacent to the fingerprint sensor and wherein the noise data corresponds to the backing layer.

21. The apparatus of claim 20, wherein the backing layer is a patterned backing layer.

22. The apparatus of claim 12, wherein the apparatus includes a reinforcing layer.

23. The device of claim 12, wherein the device is a foldable mobile device.

24. An apparatus comprising: Fingerprint sensor; and Control component, the control component being used for: Receive fingerprint image data from the fingerprint sensor; Obtain the first set of background image data; The first processed fingerprint image data is determined by subtracting the first set of background image data from the fingerprint image data; Obtain force data corresponding to the force applied to the fingerprint sensor when the fingerprint image data is acquired; Obtain a second set of background image data corresponding to the force; The fingerprint image data for the second processing is determined at least in part based on the fingerprint image data processed by the first processing and the second set of background image data; as well as Output the fingerprint image data processed in the second step.

25. The apparatus of claim 24, wherein determining the fingerprint image data processed in the second manner involves a machine learning model.

26. The apparatus of claim 24, wherein determining the fingerprint image data processed by the second process involves providing the fingerprint image data processed by the first process and the second set of background image data to a trained neural network implemented by the control member.

27. The apparatus of claim 24, wherein the fingerprint sensor is an ultrasonic fingerprint sensor or includes the ultrasonic fingerprint sensor, and wherein the control member is further configured to: Multiple sets of fingerprint image data are obtained from the fingerprint sensor, each set of fingerprint image data corresponding to a different distance gate delay; and The fingerprint image data for the second processing is determined in part based on at least one set of fingerprint image data from the plurality of sets of fingerprint image data.

28. One or more non-transitory computer-readable media storing instructions thereon for performing a method, the method comprising: The control system receives fingerprint image data from the fingerprint sensor; The control system obtains the first set of background image data; The control system determines the first processed fingerprint image data by subtracting the first set of background image data from the fingerprint image data; The control system obtains force data corresponding to the force applied to the fingerprint sensor when the fingerprint image data is acquired; The control system obtains a second set of background image data corresponding to the force; The control system determines the fingerprint image data for the second processing based at least in part on the fingerprint image data processed by the first processing and the second set of background image data; as well as Output the fingerprint image data processed in the second step.

29. One or more non-transitory computer-readable media of claim 28, wherein determining the fingerprint image data of the second processing involves a machine learning model.

30. One or more non-transitory computer-readable media according to claim 28, wherein determining the fingerprint image data of the second processing involves providing the fingerprint image data of the first processing and the second set of background image data to a trained neural network implemented by the control system.

Images