Methods and apparatuses for filtering pulse width modulation (PWM) noise from a fingerprint image captured with an optical under-display fingerprint sensor (UDFPS)

Abstract

This disclosure describes a method, apparatus, and technique for capturing a fingerprint image (130) using an electronic device (102) having an under-display fingerprint sensor (UDFPS) (122) embedded beneath a display screen (112) of a display system (110). The display system (110) utilizes a pulse width modulation circuit (114) to generate pulse width modulation (PWM) signals (204-10, 204-20) to control light emitted by the display screen (112). When the display screen (112) illuminates a user touch, the UDFPS (122) captures the light reflected from the user touch, thus capturing a fingerprint image (130). However, the captured fingerprint image (130) includes PWM noise (132). The electronic device (102) uses a noise filtering algorithm to filter out and / or reduce the PWM noise (132) in the captured fingerprint image (130). In one aspect, the noise filtering algorithm estimates and / or determines the PWM noise (132) in the captured fingerprint image (130). Then, a noise filtering algorithm reduces, extracts, and / or filters out PWM noise (132) from the captured fingerprint image (130).

Description

Background Technology

[0001] To preserve space on the display side of electronic devices, manufacturers can embed sensors beneath the display. These sensors can include optical under-display fingerprint sensors (UDFPS), ambient light sensors, cameras, and more. With optical UDFPS, when a user places their finger on the display and above the optical UDFPS, the electronic device uses the display's brightness to illuminate the finger. Example illumination systems use pulse width modulation (PWM) circuitry to enable the electronic device to generate PWM signals to control the brightness, refresh rate, and other parameters of the illuminated finger. Unfortunately, optical UDFPS may capture fingerprint images that include PWM noise. PWM noise increases temporal noise and can reduce the signal-to-noise ratio (SNR) of the captured fingerprint image. The reduced SNR of the captured fingerprint image decreases its quality. Fingerprints with lower image quality can increase the false acceptance rate, thus compromising the biometric security of the electronic device. Fingerprints with lower image quality can also increase the false rejection rate, resulting in a poor user experience. Summary of the Invention

[0002] This disclosure describes methods, apparatus, and techniques for capturing fingerprint images using an electronic device having an under-display fingerprint sensor (UDFPS) embedded beneath a display screen in a display system. The display system utilizes a pulse width modulation (PWM) circuit to generate a pulse width modulation (PWM) signal to control the light emitted by the display screen. When the display screen illuminates a user touch, the UDFPS captures the light reflected from the user's touch, thus capturing a fingerprint image. However, the captured fingerprint image includes PWM noise. The electronic device uses a noise filtering algorithm to filter out and / or reduce the PWM noise in the captured fingerprint image. On one hand, the noise filtering algorithm estimates and / or determines the PWM noise in the captured fingerprint image. Then, the noise filtering algorithm reduces, extracts, and / or filters out the PWM noise from the captured fingerprint image. The noise filtering algorithm is capable of using limited hardware and / or memory resources and can utilize relatively few computational resources.

[0003] On one hand, a computer-implemented method is described that performs principal component analysis on a background image associated with a sensor embedded beneath the display screen of a display system, the principal component analysis providing artifacts on the background image. The method then extracts principal component vectors from the artifacts and, in response to the extraction, determines at least one pulse width modulation (PWM) noise based on the principal component vectors. In response to the determination of the PWM noise, the method vectorizes the PWM noise to create a PWM noise vector and illuminates a user touch using light from the display screen. Illuminating the user touch causes light to reflect off the user's skin. The method then captures the reflected light at the sensor to provide a user-associated image and projects this user-associated image onto a vector space associated with the PWM noise vector to provide a user-associated image in the vector space. Based on the user-associated image in the vector space, the method filters out the PWM noise from the user-associated image to provide a noise-reduced image.

[0004] On the other hand, an electronic device includes a display screen of a display system, a pulse width modulation circuit of the display system, a sensor, one or more processors, and one or more computer-readable media having instructions thereon that, in response to execution by the one or more processors, perform the operations of the methods described above. In yet another aspect, a system, software, or apparatus includes operations that perform the methods described above.

[0005] This invention presents a simplified concept for capturing fingerprints using a fingerprint sensor, which will be further described in the detailed description and accompanying drawings below. For ease of description, this disclosure focuses on optical UDFPS embedded under the display screen of an electronic device (e.g., an organic light-emitting diode (OLED) display). However, the described concept can be used in many software, systems, and / or devices to filter, extract, and / or reduce noise associated with pulse width modulation signals. Optional features of one aspect, such as the methods described above, can be combined with other aspects. Attached Figure Description

[0006] Details of one or more aspects of a user device that has a fingerprint recognition system that improves fingerprint image quality by extracting pulse width modulation noise from the fingerprint image are shown in the figure below:

[0007] Figure 1 The illustration shows an example electronic device with a fingerprint recognition system, which has a fingerprint sensor embedded under the display screen of a display system;

[0008] Figure 2 The illustration shows an example of the cause of pulse width modulation (PWM) noise in the captured fingerprint image;

[0009] Figure 3The illustration shows an example artifact of phase and frequency variations including PWM noise;

[0010] Figure 4 The illustration shows various example artifacts caused by PWM noise;

[0011] Figure 5 The illustration depicts an example computer-implemented method for filtering PWM noise from a captured fingerprint image to provide a filtered fingerprint image with reduced PWM noise; and

[0012] Figure 6 The illustration shows an example of the patterns and details used in fingerprint authentication. Detailed Implementation

[0013] Example Environment

[0014] Figure 1 The illustration depicts an example environment 100 for electronic device 102. Electronic device 102 enables users to biometrically protect their devices using a fingerprint recognition system 120 (FIS 120) with at least one optical under-display fingerprint sensor 122 (UDFPS 122). Electronic device 102 provides biometric security by comparing a verification image with a registered image of the user's thumb, fingertip, or multiple fingertips. For example, electronic device 102 can utilize UDFPS 122 to capture a "verification image" and match the pattern and / or details of that verification image with a "registration image," where examples of the pattern and details are in... Figure 6 The diagram is shown in the image.

[0015] As described herein, a "verification image" is a fingerprint image used for authentication. A "registration image" is an image captured by the user device during registration, such as when a user first sets up electronic device 102 or application 104. Furthermore, a "registration image template" (registration template) can be a mathematical representation of the registration image. The registration template can be a vectorized representation of the registration image and, in addition to other advantages mentioned below, occupies less memory space in the user device. While advantageous in some respects, it is not necessary to use a vectorized representation of the registration image template to match the verification image with the registration image template. The described apparatus, method, and technique are capable of performing image-to-image (rather than vector-to-vector) comparisons and other representations to compare each verification image with the registration image.

[0016] Electronic device 102 may be a smartphone, tablet, laptop, desktop computer, computing watch, computing glasses, gaming system or controller, smart speaker system, appliance, television, entertainment system, audio system, automobile, autonomous vehicle (air, ground or underwater "drone"), touchpad, drawing tablet, netbook, e-reader, home security system, doorbell, refrigerator and other devices 120 with fingerprint recognition system.

[0017] Electronic device 102 may include a ratio Figure 1 The illustrated components may include more or fewer components. Electronic device 102 includes one or more application processors 106 and one or more computer-readable storage media (CRM 108). Application processor 106 may include any combination of one or more controllers, microcontrollers, processors, hardware processors, hardware processing units, digital signal processors, graphics processors, graphics processing units, etc. Application processor 106 processes computer-executable instructions (e.g., code, MATLAB® code) stored in CRM 108. CRM 108 may include any suitable memory medium and storage medium, such as volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., flash memory), optical media, magnetic media (e.g., disk or magnetic tape), etc. Furthermore, CRM 108 may store instructions, data (e.g., biometric data), and / or other information, and CRM 108 excludes propagated signals.

[0018] Electronic device 102 may also include application 104. Application 104 may be software, an app, a peripheral device, or other entity that requires or prefers to authenticate a user. For example, application 104 may be a security component of electronic device 102 or an access entity used to protect information accessible from electronic device 102. Application 104 may be online banking application software or a webpage that requires fingerprint recognition before logging into an account. Alternatively, application 104 may be part of an operating system (OS) that (typically) blocks access to electronic device 102 until the user's fingerprint is authenticated as that of the verified user. The verified user may execute application 104 partially or entirely on electronic device 102 or in the "cloud" (e.g., on a remote device accessed via the Internet). For example, application 104 may use an Internet browser and / or application programming interface (API) to provide an interface to an online account.

[0019] In addition, electronic device 102 may also include one or more input / output (I / O) ports (not shown) and display system 110. Display system 110 includes at least one display screen 112, such as an organic light-emitting diode (OLED) display screen. The display system also includes pulse width modulation circuitry 114, which may be a driver associated with display screen 112. Figure 1 (Not shown in the figure) Part of the pulse width modulation circuit 114. The pulse width modulation circuit 114 is capable of generating a pulse width modulation (PWM) signal that enables the electronic device 102 to control the brightness, refresh rate, etc. of the display screen 112.

[0020] To preserve space on the display side of the electronic device 102, the manufacturer can embed a sensor 116 (e.g., an ambient light sensor, a camera) and a UDFPS 122 below the display screen 112. However, the sensor 116 and UDFPS 122 may prefer different brightness levels of the display screen 112 and / or different duty cycles of the PWM signal. For example, an ambient light sensor may operate better when the PWM signal has a higher pixel off-time, or what may be referred to herein as a “blank time.” On the other hand, the UDFPS 122 may operate better when the PWM signal has a lower blank time. To meet these different requirements, the duty cycle of the PWM signal is less than 100%. Additionally, users may prefer a display screen 112 with a high refresh rate (e.g., 360 Hz). Generally, a higher refresh rate enables higher image quality on the display screen 112 (e.g., text, still images, videos). In all aspects, the UDFPS 122 can be calibrated to accommodate multiple operating modes covering varying duty cycles or varying frequencies of the PWM noise 132 (e.g., see [link to relevant documentation]). Figure 2 and Figure 3 ).

[0021] On one hand, the blanking time of the PWM signal and the refresh rate of the display 112 enable the UDFPS 122 to capture a fingerprint image 130 including PWM noise 132, as further described below. PWM noise 132 increases the temporal noise of the captured fingerprint image 130 and reduces the signal-to-noise ratio (SNR). The reduced SNR and the presence of PWM noise 132 on the captured fingerprint image 130 degrade the quality of the fingerprint image, thereby adversely affecting biometric security. In biometric security, success rate is typically characterized using a receiver operating curve (ROC), which can be represented as a graph illustrating the diagnostic capability of a binary classifier system as its discrimination threshold changes. More specifically, biometric security measurements may include the false acceptance rate (FAR) as a percentage of unauthorized access is granted by the fingerprint recognition system and the false rejection rate (FRR) as a percentage of authorized access is denied by the fingerprint recognition system. Qualitatively, a high success rate fingerprint recognition has a low false acceptance rate and a low false rejection rate.

[0022] To increase the success rate of the fingerprint recognition system 120, the electronic device 102 can remove and / or filter out PWM noise 132 from the captured fingerprint image 130 to generate, store, and use a filtered fingerprint image 134. The filtered fingerprint image 134 has an increased SNR, lower PWM noise 132, lower FAR, lower FRR, and provides better biometric security than the captured fingerprint image 130. Therefore, the filtered fingerprint image 134 enables the user to better protect the electronic device 102, application 104, functions, or their peripherals.

[0023] It should be understood that users can control their biometric data (e.g., fingerprints) because electronic device 102 can capture, collect, store, analyze, filter, and / or process user-associated information after receiving explicit permission from the user. Indeed, examples of electronic device 102 analyzing user-associated information (e.g., captured fingerprint images) are described throughout the disclosure. In the case of electronic device 102 authenticating a user based on a fingerprint, as discussed below, the user is provided with the opportunity to control whether a program or feature of the user device or remote system can collect a fingerprint and use it for current or subsequent authentication processes. Therefore, individual users can control what electronic device 102 can or cannot do with fingerprint images and other information associated with the user. User-associated information (e.g., registration images, verification images, captured fingerprint images), if previously stored, is preprocessed in one or more ways to remove personally identifiable information before transmission, storage, or other use. For example, a user device can encrypt a registration image before storing it. Preprocessing data in this way ensures that the information cannot be traced back to the user, thereby removing any personally identifiable information that can be inferred from the user's fingerprint. Alternatively, users can choose to forgo using biometric data to protect the security of electronic device 102. Instead, users can use a username, password, personal identification number (PIN), and / or a combination thereof to protect their electronic device 102 and / or application 104.

[0024] PWM noise

[0025] Figure 2 The illustration shows the environment 200 on the display side of an electronic device 102 (e.g., a smartphone). Figure 2 Here are some example reasons to help describe the PWM noise 132 in the captured fingerprint image 130. As illustrated, the electronic device 102 may include a speaker 202 and a display 112. To reserve space on the display side of the electronic device 102, the manufacturer may embed the sensor 116 and UDFPS 122 below the display 112. Figure 2 In the diagram, the electronic components embedded below the display screen 112 are shown in dashed lines.

[0026] On one hand, the driver and / or pulse width modulation circuitry 114 uses a display “rolling shutter” scheme to drive the pixels of the display screen 112, wherein successive rows of pixels 204 (display rows 204) are refreshed at a specific frequency (e.g., 60 Hz, 90 Hz, 360 Hz). As described herein, a “rolling shutter” scheme is a method of capturing and / or displaying images, wherein the images are captured by rapidly, vertically (e.g., by...) Figure 2The display rolling shutter scheme can capture and / or display each frame of a still image or video by scanning the image horizontally (not shown) or horizontally (not shown). The "rolling shutter" scheme differs from the "global shutter" scheme (not shown); in the "global shutter" scheme, the entire still image or video frame is captured and / or displayed at the same time. As illustrated by the direction of the arrow in display line 204, the display rolling shutter scheme can be a vertical scheme. Although not shown, the display rolling shutter scheme can have a vertical orientation compared to... Figure 2 The diagram shows a vertical scheme with the opposite direction. Alternatively, the rolling shutter scheme can be shown as a horizontal scheme, which can have a left-to-right or right-to-left direction.

[0027] exist Figure 2 In this diagram, a PWM signal drives the pixels of display line 204 with a duty cycle of less than 100%. As described herein, a "duty cycle" is the portion of a "cycle" of a PWM signal that turns on the pixels of display line 204. This cycle is the time required for the PWM signal to complete one on and off cycle. The duty cycle of a PWM signal can be expressed as a percentage or ratio. For example, an 80% duty cycle describes a PWM signal that is on for 80% of the time and off for 20% of the time during each cycle. Figures 204-10 and 204-20 help illustrate the first (204-10) and second (204-20) PWM signals with corresponding duty cycles of less than 100%. For consistency, simplicity, and clarity of the illustration, PWM signals 204-10 and 204-20 have the same frequency (e.g., 360 Hz)—therefore, the period 204-12 of the first PWM signal 204-10 is equal to the period 204-22 of the second PWM signal 204-20. Using a "rolling shutter" display scheme, during each cycle (e.g., 204-12), the first PWM signal 204-10 can turn on the pixels of display line 204 for approximately 80% of the time (e.g., 204-14, the on-time) and turn off the pixels of display line 204 for approximately 20% of the time (e.g., 204-16, the blanking time, the off-time). Similarly, during each cycle (e.g., 204-22), the second PWM signal 204-20 can turn on the pixels of display line 204 for approximately 95% of the time (e.g., 204-24) and turn off the pixels of display line 204 for approximately 5% of the time (e.g., 204-26). In this illustration, the first PWM signal 204-10 has an 80% duty cycle, and the second PWM signal 204-20 has a 95% duty cycle.

[0028] The duty cycle of the modulated PWM signal (e.g., 204-12, 204-22) modulates and / or controls the brightness output of the pixels of the display 112. A shorter blanking time (e.g., 204-26) and a higher refresh rate (e.g., 360Hz) allow the display 112 to display higher quality images with less "flicker." However, a shorter blanking time (e.g., 204-26) hinders the proper functioning of the sensor 116 (e.g., an ambient light sensor), as the ambient light sensor benefits from a longer blanking time. Therefore, the display 112 and UDFPS 122 prefer PWM signals with a higher duty cycle (e.g., 204-20), while the sensor 116 prefers PWM signals with a lower duty cycle (e.g., 204-10). Engineers, designers, and scientists strive to design and build electronic devices 102 that can successfully utilize all their components, applications, and peripherals. In order for the ambient light sensor to function properly, the electronic device 102 operates with a long blanking time (e.g., 204-16) of the duty cycle of the PWM signal.

[0029] To further describe and illustrate the possible causes of PWM noise 132, consider some operating principles of a fingerprint recognition system 120 with UDFPS 122. When a user places their thumb somewhere within the UDFPS active area 122-2 (e.g., touch area 122-1), UDFPS 122 captures a fingerprint image 130. More specifically, display 112 illuminates the user's skin touching the display 112 (e.g., using visible light). The illumination from the user's touch causes light to reflect off the user's skin (e.g., thumb). UDFPS 122 captures the reflected light. UDFPS 122 can utilize various image sensor technologies, such as complementary metal-oxide-semiconductor (CMOS) image sensors, charge-coupled device (CCD) image sensors, thin-film transistor (TFT) image sensors, or any image sensor that utilizes light. Assume that UDFPS 122 includes a CMOS image sensor with a certain number of pixels (e.g., 2 to 10 megapixels) arranged in rows and columns. UDFPS also uses a UDFPS "rolling shutter" scheme to capture fingerprint image 130, in which fingerprint image 130 is captured by consecutive rows of pixels of the CMOS image sensor (UDFPS row 206). The time for capturing fingerprint image 130 using the UDFPS "rolling shutter" scheme can be referred to as "integration time".

[0030] However, the integration time of the fingerprint recognition system 120 and UDFPS 122 is significantly slower than the refresh time of the display screen 112. The refresh time of the display screen 112 is inversely proportional to its refresh rate. The illumination of the display screen 112 with a 360Hz refresh rate can be driven by a PWM signal with a frequency of 360Hz and a duty cycle of 80%. In this case, the PWM signal controlling the illumination of the display screen 112 can include 35 or 36 blanking times during the integration time used to capture the fingerprint image 130.

[0031] Engineers, designers, and scientists strive to synchronize the clock (not shown) of the fingerprint recognition system 120 with the clock (not shown) of the display 112. More specifically, engineers attempt to design the fingerprint recognition system 120 with an integration time that is an integer multiple (e.g., an integer) of the period (e.g., 204–12) and duty cycle (e.g., 204–14) of the PWM signal. For example, the integration time could be 35 times the period and duty cycle of the PWM signal. Unfortunately, the corresponding clocks of the display 112 and the fingerprint recognition system 120 include corresponding jitter (e.g., 1%, 2%, 3%) due to variations in process, voltage, and temperature (PVT). On the one hand, the PWM signal may have difficulty maintaining a constant duty cycle, and / or the fingerprint recognition system 120 may have difficulty maintaining a constant integration time. Therefore, the combination of a shorter duty cycle (longer blanking time) of the PWM signal, a higher refresh rate of the display 112, and the jitter of the clocks of the display 112 and the fingerprint recognition system 120 can lead to PWM noise (e.g., Figure 1 132) appears in the captured fingerprint image (e.g., Figure 1 (130). Next. Figure 3 An example aspect of PWM noise 132 is illustrated.

[0032] Figure 3 The illustration depicts an example environment 300 of PWM noise 132. In one aspect, to determine PWM noise 132, the electronic device 102 can capture data that does not include… Figure 6 The example pattern and / or details are multiple (e.g., four) consecutive images of the same opaque (e.g., white) flat surface. If the integration time of the fingerprint recognition system 120 is equal to an integer (e.g., 35) multiple of the period (e.g., 204-12) and duty cycle (e.g., 204-14) of the PWM signal 204-10, then the electronic device 102 captures the same flat consecutive images. However, Figure 3 The illustration shows the base of four consecutive captured images 301 to 304 of a flat surface including dark stripes. These dark stripes are artifacts of PWM noise 132, since a flat surface does not contain stripes.

[0033] Furthermore, the number of dark stripes in the captured images 301 to 304 differs. On one hand, the different counts of dark stripes in the captured images 301 to 304 may be due to frequency variations in the PWM noise 132. To further examine the PWM noise 132, Figure 3 Enlarged images 301-2 to 304-2 of regions 301-1 to 304-1 of the base images 301 to 304 are shown respectively. Images 301-2, 302-2, 303-2, and 304-2 illustrate the phase variation of the PWM noise 132. On the one hand, the phase variation of the PWM noise can be manifested as a positional shift of dark stripes in the captured image (e.g., with bases 301 to 304, 301-1 to 304-1, 301-2 to 304-1). Therefore, Figure 3 The illustration shows that even after clock synchronization between the fingerprint recognition system 120 (with UDFPS 122) and the display system 110 (with display screen 112), the captured image includes PWM noise 132 with phase and frequency variations. These phase and frequency variations increase the unpredictability of the PWM noise 132. However, the electronic device 102 uses a noise filtering algorithm to remove and / or filter out the PWM noise 132 from the captured fingerprint image 130 to generate, store, and use a filtered fingerprint image 134. Next, Figure 4 The illustration shows how noise filtering algorithms can use principal component analysis (PCA) techniques to identify, model, estimate, extract, and / or determine PWM noise132

[0034] PWM noise estimation

[0035] Figure 4 Example environment 400 illustrates the basis 401 to 410 of a corresponding captured image on a non-transparent (e.g., white) flat (e.g., smooth) surface. Figure 4 Is Figures 1 to 3 The scenarios described are electronic device 102, display system 110 with display screen 112, and fingerprint recognition system 120 with UDFPS 122. The captured image of the non-transparent, flat surface can be considered and / or referred to as the "background image" of fingerprint recognition system 120. Therefore, artifacts captured in the background image (e.g., 401 to 410, pattern, brightness / darkness variations) are due to PWM noise 132.

[0036] PCA technology can model PWM noise 132, which can include variations in phase, frequency, and / or amplitude (e.g., brightness or darkness). In mathematics, image processing, signal processing, mechanics, neuroscience, and / or other engineering and scientific disciplines, PCA is a technique for computing principal components and / or principal component vectors. As described herein, PCA technology can be used to identify, model, estimate, extract, and / or determine artifacts in background images, such as... Figure 4 The figures 401 to 410 are illustrated in the diagram. The basics 401-410 identify, model, estimate, extract, and / or determine PWM noise 132. To reduce PWM noise 132, the electronics 102 can synchronize the startup cycle between the PWM signal used to control the light emitted by the display 112 and the integration time used to capture the fingerprint image 130 using UDFPS 122.

[0037] On one hand, in order to identify, model, estimate, extract, and / or determine PWM noise 132, electronic device 102 is capable of capturing a background image (e.g., with bases 401 to 410) by changing and / or simulating clock jitter, frequency shift, amplitude variation, and / or other parameters that can cause PWM noise 132. In doing so, a noise filtering algorithm can identify, model, estimate, extract, and / or determine PWM noise 132 in the captured fingerprint image 130 caused by corresponding clock jitter (e.g., 1%, 2%, 3%) of the clock of FIS 120 and / or the clock of display system 110. The noise filtering algorithm is also capable of identifying, modeling, estimating, extracting, and / or determining frequency shifts in PWM noise 132 (e.g., 301-2, 302-2, 303-2, 304-2). The frequency shift of PWM noise 132 may occur due to variations in the refresh rate of display 112 (e.g., 359.6 Hz to 360.4 Hz) and / or variations in the integration time of UDFPS 122 (e.g., 97.6 ms to 98.4 ms). Furthermore, the noise filtering algorithm is capable of identifying, modeling, estimating, extracting, and / or determining amplitude variations caused by changes in the amount of light captured by UDFPS 122 (e.g., UDFPS row 206). These amplitude variations may occur due to variations in the duty cycle (e.g., 204-14) of the PWM signal (e.g., 204-10) of display system 110, asynchrony between the clocks of FIS 120 and display system 110, and / or other factors. The noise filtering algorithm is capable of using limited hardware and / or memory resources and can utilize relatively few computational resources to ensure minimal disruption to the user experience.

[0038] Alternatively or additionally, instead of capturing artifacts on the background image, manufacturers can use datasheets, specifications, and / or models of the FIS 120 and display system 110 to calculate potential and / or expected artifacts. By doing so, the noise filtering algorithm can develop a database including PWM noise 132. Therefore, the noise filtering algorithm is able to identify, model, estimate, extract, and / or determine PWM noise 132 on individual electronic devices 102, electronic devices of the same brand and model, and / or models sharing the same display system 110 and the same fingerprint recognition system 120. The vectorized form of the background image including artifacts (e.g., 401 to 410) can be encrypted and stored in the database. Note that the database including background images with artifacts does not contain any user biometric data (e.g., fingerprint data). Therefore, the noise filtering algorithm further protects user privacy. Next, Figure 5 The illustration shows how a noise filtering algorithm uses a vectorized form of the background image (e.g., with bases 401 to 410) to filter out PWM noise 132 from a captured fingerprint image 130 to obtain a filtered fingerprint image 134.

[0039] Example Method

[0040] Figure 5 An example method 500 is illustrated for filtering PWM noise 132 from a captured fingerprint image 130 to obtain a filtered fingerprint image 134. Figure 5 Is Figures 1 to 4 The scenarios described are: electronic device 102, display system 110 with display screen 112, and fingerprint recognition system 120 with UDFPS 122. The operations performed in example method 500 can be compared with... Figure 5 The different sequences shown can be performed using additional or fewer steps. Example method 500's PCA technique is capable of modeling PWM noise 132, where PWM noise 132 includes variations in phase, frequency, and / or amplitude (e.g., brightness or darkness). PCA is a technique for extracting principal components and / or principal component vectors. In various aspects, PCA techniques analyze data (e.g., biometric data) to identify, model, estimate, extract, and / or determine artifacts in a background image.

[0041] In phase 502, method 500 uses... Figure 4 The PCA technique described herein performs principal component analysis (PCA) on a background image (e.g., with underlying 401 to 410). Figure 3 , Figure 4 and Figure 5 As shown, the background image may include artifacts (e.g., dark stripes). PCA techniques enable method 500 to identify artifacts (e.g., Figure 5(132). Then, method 500 extracts the principal component vectors of the background image in stage 504. Figure 4 As illustrated in the background images with bases 401 to 410, the first plurality of principal components (e.g., 402, 403) may contain more artifacts than the second plurality of principal components (e.g., 409). Therefore, in stage 506, method 500 determines which artifacts are caused by PWM noise 132. Method 500 may ignore some artifacts because they may contain significant Gaussian noise. For example, due to the reduced number of artifacts (e.g., almost no measurable and / or detectable dark stripes), method 500 may ignore background image 401. After method 500 determines which artifacts are caused by PWM noise 132, in stage 508, method 500 creates a PWM noise vector (e.g., 401-410) for each artifact. Figures 3 to 5 As shown, since the artifacts in the background image are striped, the horizontal portion of the PWM noise 132 is constant. Thus, method 500 can utilize a one-dimensional (instead of a two-dimensional) PWM noise vector. The use of a one-dimensional vector allows the noise filtering algorithm to utilize limited hardware and / or memory resources (e.g., CRM 108) and relatively few computational resources (e.g., application processor 106). Mathematically, method 500 can represent the PWM noise vector as a matrix denoted herein as [PCA_PWM]. These principal component vectors can be stored and accessed in the future.

[0042] In stage 510, method 500 illuminates the user touch (e.g., touch area 122-1) using light from display 112. The illumination causes light to reflect off the user's skin. Then, in stage 512, method 500 captures the reflected light at UDFPS 122 to provide an image associated with the user (e.g., a captured fingerprint image 130). However, in stage 512, the captured fingerprint image 130 is an unfiltered fingerprint image and includes PWM noise 132. In addition to artifacts (e.g., dark stripes) from the PWM noise 132, the captured fingerprint image 130 also has a reduced signal-to-noise ratio (SNR), further degrading the quality of the captured fingerprint image 130. Figure 5 In comparison with the filtered fingerprint image 134, the reduced SNR in the captured fingerprint image 130 is illustrated as a lighter gray shading of the pattern and / or details.

[0043] In stage 514, method 500 projects a user-associated image (e.g., the captured fingerprint image 130) onto a vector space associated with a PWM noise vector (e.g., 132, [PCA_PWM]). Projecting the user-associated image may include multiplying by the PWM noise vector, the transpose of the PWM noise vector matrix, and the vectorized form of the captured fingerprint image 130. Mathematically, stage 514 of method 500 can be expressed using Equation 1:

[0044] Equation 1

[0045] in The matrix represents the PWM noise vector. express Matrix transpose The matrix represents the vectorized form of the captured fingerprint image 130, and This represents a user-related image (e.g., a captured fingerprint image 130) projected onto the vector space of the PWM noise.

[0046] In stage 516, based on the user-associated image projected onto the vector space of the PWM noise, method 500 filters out the PWM noise 132 from the user-associated image (e.g., the captured fingerprint image 130) to provide a noise-reduced image (e.g., the filtered fingerprint image 134). Filtering out the PWM noise 132 can be achieved by subtracting the user-associated image in the vector space from the PWM noise vector. Mathematically, stage 516 of method 500 can be expressed using Equation 2:

[0047] Equation 2

[0048] in The matrix represents the vectorized form of the filtered fingerprint image 134.

[0049] Patterns and details

[0050] Figure 6 Examples of patterns (602, 604, and 606) and details (610 to 626) used in fingerprint matching are illustrated. Fingerprint analysis for matching purposes typically requires comparing patterns and / or details. Method 500 reduces the false acceptance rate, thereby improving the biometric security of electronic device 102. Method 500 also reduces the false rejection rate, thus providing a better user experience.

[0051] The three main patterns of fingerprint ridges are arch 602, ring 604, and spiral 606. Arch 602 is a fingerprint ridge that enters from one side of the finger, rises in the center to form an arc, and then exits from the other side of the finger. Ring 604 is a fingerprint ridge that enters from one side of the finger, forms a curve, and then exits on the same side of the finger. Spiral 606 is a fingerprint ridge that is circular around a central point. Details 610 to 626 are features of fingerprint ridges, such as ridge ends 610, bifurcations 612, short ridges 614, dots 616, bridges 618, breaks 620, branches 622, islands 624, double bifurcations 626, deltas 628, trifurcations 630, lakes or ridge peripheries (not shown), nuclei (not shown), etc.

[0052] For example, the presence of artifacts in the captured fingerprint image 130 (e.g., the verification image) due to PWM noise 132 may manifest as "valleys" in the verification image. Therefore, the electronic device may incorrectly reject the user when comparing the verification image with the registration image. However, the electronic device 102 uses... Figures 1 to 5 The method 500 described herein is used to filter out artifacts that may be incorrectly represented as "valleys" in fingerprint images.

[0053] In general, any component, module, method, and operation described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Some operations of the example methods can be described in the general context of executable instructions stored on computer-readable storage memory local and / or remote on a computer processing system, and implementations can include software applications, programs, functions, etc. Alternatively or additionally, any function described herein can be performed at least in part by one or more hardware logic components, including but not limited to field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.

[0054] The following are some examples:

[0055] Example 1. A computer-implemented method comprising: performing principal component analysis on a background image associated with a sensor embedded beneath a display screen of a display system, the principal component analysis providing artifacts on the background image; extracting principal component vectors from the artifacts; determining at least one pulse width modulation (PWM) noise based on the extraction, in response to the extraction; vectorizing the PWM noise to create a PWM noise vector in response to determining the PWM noise; illuminating a user touch with light from the display screen, the illumination being used to reflect the light from the user's skin; capturing the reflected light at the sensor to provide a user-associated image; projecting the user-associated image onto a vector space associated with the PWM noise vector to provide the user-associated image in the vector space; and filtering the PWM noise from the user-associated image based on the user-associated image in the vector space to provide a noise-reduced image.

[0056] Example 2: The computer-implemented method according to Example 1 further includes: creating a first matrix of the PWM noise based on the PWM noise vector, and creating a second matrix based on the user-associated image, wherein projecting the user-associated image multiplies the first matrix, the transpose of the first matrix, and the second matrix.

[0057] Example 3: The computer-implemented method according to Example 2, wherein the PWM noise is filtered out by subtracting the user-associated image in the vector space from the second matrix.

[0058] Example 4: The computer-implemented method according to at least one of Examples 1 to 3 further includes: authenticating the user based on the noise-reduced image.

[0059] Example 5: The computer-implemented method according to Example 4 further includes: enabling the use of a function, account, or peripheral device in response to authenticating the user.

[0060] Example 6: A computer-implemented method according to at least one of Examples 1 to 5, wherein the PWM noise (132) is vectorized to create a PWM noise vector based on a variable frequency in the PWM noise; and the number of PWM noise vectors corresponds to the number of frequencies of the PWM noise.

[0061] Example 7: A computer-implemented method according to at least one of Examples 1 to 6, wherein the PWM noise (132) is vectorized to create a PWM noise vector based on a change in one or more duty cycles of the display system (110); and the number of PWM noise vectors corresponds to the number of duty cycles of the display system (110).

[0062] Example 8: A computer-implemented method according to at least one of Examples 1 to 7, wherein the sensor is capable of capturing multiple images of an opaque, flat surface, without the user-associated image.

[0063] Example 9: The computer-implemented method according to Example 8, wherein the non-transparent flat surface is white.

[0064] Example 10: A computer-implemented method according to at least one of Examples 1 to 9, wherein determining the PWM noise further includes capturing a plurality of background images by simulating clock jitter to induce the PWM noise.

[0065] Example 11: A computer-implemented method according to at least one of Examples 1 to 10, wherein the PWM noise is identified, modeled and / or estimated by the electronic device to capture the at least one background image by changing and / or simulating clock jitter, frequency shift, amplitude variation and / or other parameters that cause the PWM noise.

[0066] Example 12: The computer-implemented method according to Example 10 or 11 further includes: determining the amplitude variation of the PWM noise.

[0067] Example 13: A computer-implemented method according to at least one of Examples 1 to 12, wherein the artifact comprises a fingerprint image generated from a thumb, fingertip, or multiple fingertips.

[0068] Example 14: A computer-implemented method according to at least one of Examples 1 to 13, wherein the background image is an image captured from an opaque, flat surface of a fingerprint recognition system.

[0069] Example 15: A computer-implemented method according to at least one of Examples 1 to 14, wherein capturing reflected light at the sensor to provide a user-associated image includes a vertically displayed rolling shutter scheme.

[0070] Example 16: A computer-implemented method according to at least one of Examples 1 to 14, wherein capturing reflected light at the sensor to provide a user-associated image includes a horizontally displayed rolling shutter scheme.

[0071] Example 17: A computer-implemented method according to Example 15 or 16, wherein a first PWM signal turns on the pixels of the display row for a period of more than 70%, particularly more than 80%, and turns off the pixels of the display row for a period of less than 30%, particularly less than 20%, and a second PWM signal turns on the pixels of the display row for a period of more than 90%, particularly more than 95%, and turns off the pixels of the display row for a period of less than 10%, particularly less than 5%.

[0072] Example 18: A user equipment includes: a display system comprising: a display screen; a pulse width modulation circuit; a sensor; one or more processors; and one or more computer-readable media having instructions thereon, the instructions performing operations of a computer-implemented method according to any one of Examples 1 to 17 in response to execution by the one or more processors.

[0073] in conclusion

[0074] Although various embodiments of this disclosure have been described in the foregoing description and illustrated in the accompanying drawings, it should be understood that this disclosure is not limited thereto, but can be embodied and practiced in various ways within the scope of the appended claims. It will be apparent from the foregoing description that various changes may be made without departing from the spirit and scope of this disclosure as defined by the appended claims.

Claims

1. A method for filtering pulse width modulation (PWM) noise, comprising: Principal component analysis is performed on a background image associated with a sensor (116) embedded below a display screen (112) of a display system (110), the principal component analysis providing artifacts on the background image; Extract principal component vectors from the artifacts; Based on the extracted principal component vector, at least one pulse width modulation (PWM) noise is determined (132); The determined PWM noise (132) is vectorized to create a PWM noise vector; The user touch is illuminated using light from the display screen (112), and the illumination is used to reflect the light off the user's skin; The reflected light at the sensor (116) is captured to provide the user with an associated image; The user-associated image is projected onto a vector space, which is associated with the PWM noise vector to provide the user-associated image in the vector space; as well as Based on the user-associated image in the vector space, the PWM noise is filtered out from the user-associated image (132) to provide a noise-reduced image.

2. The method according to claim 1, further comprising: Based on the PWM noise vector, a first matrix of the PWM noise (132) is created, and a second matrix is ​​created based on the user-associated image, wherein projecting the user-associated image multiplies the first matrix, the transpose of the first matrix, and the second matrix.

3. The method according to claim 2, wherein, The PWM noise is filtered out (132) by subtracting the user-associated image in the vector space from the second matrix.

4. The method according to claim 1, further comprising: User authentication is performed based on the noise-reduced image.

5. The method according to claim 4, further comprising: In response to authenticating the user, the use of features, accounts, or peripheral devices is enabled.

6. The method according to claim 1, wherein, The operation of vectorizing the PWM noise (132) to create a PWM noise vector involves vectorizing two or more frequencies in the PWM noise; and The number of PWM noise vectors corresponds to the number of the two or more frequencies of the PWM noise.

7. The method according to claim 1, wherein, Vectorizing the PWM noise (132) to create a PWM noise vector is based on the variation of one or more duty cycles of the display system (110); and The number of PWM noise vectors corresponds to the number of changes in one or more duty cycles of the display system (110).

8. The method according to claim 1, wherein, The sensor (116) can capture multiple images of an opaque, flat surface, without the image associated with the user.

9. The method according to claim 8, wherein, The opaque, flat surface is white.

10. The method according to claim 1, wherein, Determining the PWM noise (132) further includes capturing multiple background images by simulating clock jitter to induce the PWM noise (132).

11. The method according to claim 1, wherein, The PWM noise (132) is identified, modeled and / or estimated by the electronic device (102) to capture the background image by changing and / or simulating clock jitter, frequency shift, amplitude variation and / or other parameters that cause the PWM noise (132).

12. The method of claim 10, further comprising: Determine the amplitude variation of the PWM noise (132).

13. The method according to claim 1, wherein, The artifacts include fingerprint images generated from the thumb, or one or more fingertips.

14. The method according to claim 1, wherein, The background image is an image captured from an opaque, flat surface of the fingerprint recognition system.

15. The method according to claim 1, wherein, Capturing reflected light at the sensor (116) to provide a user-associated image includes a vertically displayed rolling shutter scheme.

16. The method according to claim 1, wherein, Capturing reflected light at the sensor (116) to provide a user-associated image includes a horizontally displayed rolling shutter scheme.

17. The method according to claim 15 or 16, wherein, The first PWM signal turns on the pixels of the display row for more than 70% of the time and turns off the pixels of the display row for less than 30% of the time, and the second PWM signal turns on the pixels of the display row for more than 90% of the time and turns off the pixels of the display row for less than 10% of the time.

18. A user equipment, comprising: Display system (110), the display system comprising: Display screen (112); and Pulse width modulation circuit (114); Sensor (116); One or more processors; and One or more computer-readable media having instructions thereon, the instructions performing operations of the method according to any one of claims 1 to 17 in response to execution by the one or more processors.

Images