Fingerprint recognition method, electronic device and storage medium

Abstract

A fingerprint recognition method, an electronic device and a storage medium. In the method, when a fingerprint image collected by an electronic device is a wet finger image of a user's finger that is wet, and the wet finger image fails to match a fingerprint template in a fingerprint template library, the electronic device can optimize the wet finger image to obtain an optimized fingerprint image. The electronic device then matches the optimized fingerprint image with the fingerprint template. By implementing the provided technical solution, when fingerprint recognition is performed on a user's finger that is wet, a fingerprint of the user can also be successfully recognized. Thus, when a user's finger is wet, an electronic device can also be successfully unlocked.

Description

A fingerprint recognition method, an electronic device, and a storage medium

[0001] This application claims priority to two Chinese patent applications filed on August 29, 2023, with application number 202311103165.7 entitled "A fingerprint recognition method and related device", and on September 8, 2023, with application number 202311162041.6 entitled "A fingerprint recognition method, electronic device and storage medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the fields of terminals and fingerprint recognition, and more particularly to a fingerprint recognition method, electronic device, and storage medium. Background Technology

[0003] Currently, with the rapid development of fingerprint recognition technology, its applications are becoming increasingly widespread. Fingerprint recognition technology can be used not only in access control and attendance systems, but also in mobile phones, tablets, laptops, cars, and bank payment systems. Mobile phones, tablets, and other electronic devices can collect users' fingerprints for device unlocking and payment verification.

[0004] However, when a user's fingers are wet or sweaty, electronic devices cannot accurately recognize fingerprints, resulting in fingerprint unlocking failure or fingerprint payment failure.

[0005] Therefore, improving the accuracy of fingerprint recognition on electronic devices when users' fingers are wet or sweaty is an urgent problem to be solved.

[0006] Summary of the Invention

[0007] This application provides a fingerprint recognition method, an electronic device, and a storage medium. The fingerprint recognition method can improve the accuracy of fingerprint recognition of wet finger fingerprint images by electronic devices.

[0008] In a first aspect, this application provides a fingerprint recognition method, which may include: an electronic device detecting a first operation by a user and acquiring a first fingerprint image of the user; the electronic device optimizing the first fingerprint image to obtain a second fingerprint image; determining that the second fingerprint image successfully matches a fingerprint template stored in the electronic device, and unlocking the electronic device.

[0009] In this way, when a user unlocks the device with a wet finger, the electronic device can optimize the captured fingerprint image before performing fingerprint recognition. This improves the accuracy of fingerprint recognition for wet fingers. Furthermore, allowing users to unlock the device with a wet finger enhances the user experience.

[0010] In one possible implementation, before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method may further include: determining that the first fingerprint image fails to match the fingerprint template and that the first fingerprint image satisfies a first condition.

[0011] The first condition may include one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold; the quality score of the first fingerprint image is less than a second threshold; the foreground area of ​​the first fingerprint image is less than a third threshold; the similarity between the first fingerprint image and the fingerprint template is greater than a fourth threshold; and the overlap between the first fingerprint image and the fingerprint template is greater than a fifth threshold.

[0012] In this way, the first condition can be used to determine whether the captured user fingerprint image is from a wet finger. If so, the wet finger fingerprint image can be optimized. This allows users to unlock their devices with wet fingers without additional hardware support, thus reducing device cost and complexity. Furthermore, there's no need to lower the fingerprint matching threshold, ensuring device security while guaranteeing unlocking with wet fingers.

[0013] In one possible implementation, the image metric of the second fingerprint image is higher than that of the first fingerprint image.

[0014] In one possible implementation, the image metrics of the second fingerprint image include one or more of the texture consistency of the second fingerprint image, the quality score of the second fingerprint image, and the foreground area of ​​the second fingerprint image; the image metrics of the first fingerprint image include one or more of the texture consistency of the first fingerprint image, the quality score of the first fingerprint image, and the foreground area of ​​the first fingerprint image.

[0015] In one possible implementation, the image metrics of the second fingerprint image being higher than those of the first fingerprint image include one or more of the following: the texture consistency of the second fingerprint image being higher than the fingerprint consistency of the first fingerprint image, the quality score of the second fingerprint image being higher than the quality score of the first fingerprint image, and the foreground area of ​​the second fingerprint image being higher than the foreground area of ​​the first fingerprint image.

[0016] Thus, optimizing the first fingerprint image can improve its image metrics. In one possible implementation, before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method may further include: determining that the first fingerprint image satisfies a second condition.

[0017] The second condition may include one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold, the quality score of the first fingerprint image is less than a second threshold, and the foreground area of ​​the first fingerprint image is less than a third threshold.

[0018] In this way, the electronic device can directly determine whether to optimize the first fingerprint image or directly match the first fingerprint image based on the second condition.

[0019] In one possible implementation, the method may further include the electronic device being in a screen-off state or a screen-locked state before the electronic device detects the user's first operation.

[0020] Secondly, a fingerprint recognition method is provided, which may include: an electronic device detecting a user's first operation and acquiring a first fingerprint image of the user; the electronic device optimizing the first fingerprint image to obtain a second fingerprint image; determining that the second fingerprint image successfully matches a fingerprint template stored in the electronic device, and the electronic device executing a payment.

[0021] In this way, when a user makes a fingerprint payment with a wet finger, the electronic device can optimize the captured fingerprint image before performing fingerprint recognition. This improves the accuracy of fingerprint recognition for wet fingers. Furthermore, allowing users to make fingerprint payments with wet fingers enhances the user experience.

[0022] In one possible implementation, before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method may further include: determining that the first fingerprint image fails to match the fingerprint template and that the first fingerprint image satisfies a first condition.

[0023] The first condition may include one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold; the quality score of the first fingerprint image is less than a second threshold; the foreground area of ​​the first fingerprint image is less than a third threshold; the similarity between the first fingerprint image and the fingerprint template is greater than a fourth threshold; and the overlap between the first fingerprint image and the fingerprint template is greater than a fifth threshold.

[0024] In this way, the first condition can be used to determine whether the collected user fingerprint image is from a wet finger. If so, the fingerprint image from the wet finger can be optimized. This allows users to make fingerprint payments using wet fingers without additional hardware support, thus reducing device cost and complexity. Furthermore, there is no need to lower the fingerprint matching threshold, ensuring device security while guaranteeing fingerprint payments using wet fingers.

[0025] In one possible implementation, the image metric of the second fingerprint image is higher than that of the first fingerprint image.

[0026] In one possible implementation, the image metrics of the second fingerprint image include one or more of the texture consistency of the second fingerprint image, the quality score of the second fingerprint image, and the foreground area of ​​the second fingerprint image; the image metrics of the first fingerprint image include one or more of the texture consistency of the first fingerprint image, the quality score of the first fingerprint image, and the foreground area of ​​the first fingerprint image.

[0027] In one possible implementation, the image metrics of the second fingerprint image being higher than those of the first fingerprint image include one or more of the following: the texture consistency of the second fingerprint image being higher than the fingerprint consistency of the first fingerprint image, the quality score of the second fingerprint image being higher than the quality score of the first fingerprint image, and the foreground area of ​​the second fingerprint image being higher than the foreground area of ​​the first fingerprint image.

[0028] In this way, optimizing the first fingerprint image can improve the image metrics of the first fingerprint image.

[0029] In one possible implementation, before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method may further include: determining that the first fingerprint image satisfies a second condition.

[0030] The second condition may include one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold, the quality score of the first fingerprint image is less than a second threshold, and the foreground area of ​​the first fingerprint image is less than a third threshold.

[0031] In this way, the electronic device can directly determine whether to optimize the first fingerprint image or directly match the first fingerprint image based on the second condition.

[0032] In one possible implementation, before the electronic device detects the user's first action, the method may further include: the electronic device displaying a payment interface, which includes payment information, including one or more of the payment amount, the payer, and the payee.

[0033] Thirdly, an electronic device is provided, comprising: a fingerprint sensor, one or more processors, one or more memories, and a transceiver; wherein the transceiver, one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the electronic device to perform the method as described in any possible implementation of any of the above aspects.

[0034] Fourthly, a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method as described in any possible implementation of any of the above aspects.

[0035] Fifthly, a chip is provided for use in an electronic device, characterized in that it includes a processing circuit and an interface circuit, the interface circuit being used to receive code instructions and transmit them to the processing circuit, the processing circuit being used to execute the code instructions to perform a method as described in any possible implementation of the first aspect above.

[0036] Sixthly, a computer program product is provided that, when run on a computer, causes the computer to perform the method in any possible implementation of any of the above aspects. Attached Figure Description

[0037] Figure 1A is a schematic diagram of fingerprint unlocking of an electronic device with the screen off according to an embodiment of this application;

[0038] Figure 1B is a schematic diagram of the main interface of the electronic device provided in an embodiment of this application;

[0039] Figure 1C is a schematic diagram of the screen lock of an electronic device provided in an embodiment of this application;

[0040] Figure 1D is a schematic diagram of the payment interface of the electronic device provided in an embodiment of this application;

[0041] Figures 1E-1F are schematic diagrams of the user interface of an electronic device provided in an embodiment of this application when a group of fingerprint verifications fails.

[0042] Figures 1G-1H are a set of fingerprint images provided in the embodiments of this application;

[0043] Figure 2 is a schematic flowchart of a fingerprint recognition method provided in an embodiment of this application;

[0044] Figure 3 is a schematic diagram of AI repair and AI repair model training provided in the embodiments of this application;

[0045] Figure 4 is a schematic flowchart of another fingerprint recognition method provided in an embodiment of this application;

[0046] Figure 5 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application;

[0047] Figure 6 is a software architecture diagram of the electronic device provided in an embodiment of this application;

[0048] Figure 7 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application;

[0049] Figure 8 is a schematic flowchart of a fingerprint recognition method provided in an embodiment of this application. Detailed Implementation

[0050] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0051] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0052] Currently, fingerprint recognition has become a common authentication method in electronic devices such as mobile phones, tablets, laptops, smartwatches, and smart bracelets, for purposes such as device unlocking and payment verification.

[0053] Figures 1A-1D exemplarily illustrate interface diagrams of an electronic device 100 using fingerprint recognition for device unlocking, payment verification, etc.

[0054] As shown in Figure 1A, when the electronic device 100 is in a screen-off state, the user can place their finger on the fingerprint collection area of ​​the electronic device 100. In response to the user's finger pressing or touching operation, the electronic device 100 can collect the user's fingerprint and perform fingerprint recognition on the collected fingerprint image containing the user's fingerprint. When fingerprint recognition is successful, the electronic device 100 can light up the display screen and display the main interface, for example, the main interface 10A of the electronic device 100 shown in Figure 1B. The main interface of the electronic device 100 can also be referred to as the desktop.

[0055] As shown in Figure 1C, when the electronic device 100 displays the lock screen interface 10B, the user can place their finger on the fingerprint collection area of ​​the electronic device 100, such as the fingerprint collection area 101 in the lock screen interface 10B. In response to the user's finger pressing or touching operation, the electronic device 100 can collect the user's fingerprint and perform fingerprint recognition on the collected fingerprint image containing the user's fingerprint. After successful fingerprint recognition, the electronic device 100 can light up the display screen and display the main interface, such as the main interface 10A of the electronic device 100 shown in Figure 1B.

[0056] As shown in Figure 1D, the electronic device 100 may display a payment interface 10C. When a user confirms a payment, the user can place their finger on the fingerprint collection area of ​​the electronic device 100, for example, the fingerprint collection area 101 shown in Figure 1D. In response to the user's finger pressing or touching operation, the electronic device 100 can collect the user's fingerprint and perform fingerprint recognition on the collected fingerprint image containing the user's fingerprint. When fingerprint recognition is successful, the electronic device 100 can execute the payment. For example, the electronic device 100 may also display a payment execution prompt.

[0057] It is understood that the main interface 10A, lock screen interface 10B, and payment interface 10C shown in Figures 1B-1D are for illustrative purposes only. The main interface 10A, lock screen interface 10B, and payment interface 10C may contain more or fewer interface elements, and this application embodiment does not limit this.

[0058] However, if a user has just washed their hands and their fingers are wet or sweaty, performing fingerprint recognition on the electronic device 100 with wet or sweaty fingers will result in fingerprint recognition failure. For example, when a user attempts to unlock the electronic device 100 shown in Figure 1A or Figure 1C with a wet or sweaty finger, the electronic device 100 cannot successfully recognize the fingerprint image of the wet or sweaty finger. Unlocking is unsuccessful, and the electronic device 100 will not enter the main interface 10A shown in Figure 1B. The electronic device 100 can display the user interface 10D shown in Figure 1E, which can be used to prompt the user that fingerprint recognition has failed. As another example, when a user attempts to verify fingerprint payment with a wet or sweaty finger on the electronic device 100 shown in Figure 1D, the electronic device 100 cannot successfully recognize the fingerprint image of the wet or sweaty finger. Payment fails, and the electronic device 100 can display a fingerprint verification failure message on the payment interface 10C. This message can be message 102 shown in Figure 1F. The message 102 could display "Fingerprint verification failed." This would negatively impact the user's fingerprint recognition experience on the electronic device 100.

[0059] When a user places a wet or sweaty finger on the fingerprint acquisition area of ​​the electronic device 100, the moisture on the finger may obscure some fingerprint features, resulting in some missing fingerprint features in the fingerprint image acquired by the electronic device 100. For example, Figure 1G shows a fingerprint image when the user's finger is dry. Figure 1H shows a fingerprint image acquired by the electronic device 100 when the user's finger is wet. The fingerprint image in Figure 1G can be referred to as fingerprint image 1. The fingerprint image in Figure 1H can be referred to as fingerprint image 2. The fingerprint features in fingerprint image 1 are more complete than those in fingerprint image 2.

[0060] Fingerprint features can include, but are not limited to, fingerprint feature points and singularities. Fingerprint feature points include, but are not limited to, endpoints, bifurcation points, isolated points, short ridges, loops, and bridges. An endpoint indicates the termination of a fingerprint ridge. A bifurcation point indicates that a fingerprint ridge splits into two or more ridges. Singularities include core points and triangular points. Core points are located at the asymptotic center of a fingerprint ridge. Triangular points are the center points of triangular ridge regions in a fingerprint image; the three fingerprint ridges closest to this point form an approximate triangle.

[0061] In some embodiments of this application, the electronic device 100 can determine whether a fingerprint image is from a wet finger based on the number of feature points per unit area in the fingerprint image. For example, when the number of feature points per unit area in the fingerprint image is less than a threshold of 1, the electronic device 100 can determine that the fingerprint image is from a wet finger. Here, a fingerprint image from a wet finger can be simply referred to as a wet finger image. When the fingerprint image acquired by the electronic device 100 is a wet finger image, the electronic device 100 can downsample the template fingerprints in the fingerprint template library. Then, the electronic device 100 matches the wet finger image with the downsampled template fingerprint. Thus, when user A has a right thumb fingerprint image pre-stored in the fingerprint template library of the electronic device 100, and user A uses a wet right thumb for fingerprint authentication on the electronic device 100, the electronic device 100 can successfully recognize the user's wet right thumb fingerprint image. Here, the fingerprint template library can pre-store one or more fingerprint template images. The fingerprints contained in one or more fingerprint templates are fingerprints that have been registered and recorded with operating permissions on the electronic device 100.

[0062] A fingerprint template can be derived from a user's original fingerprint through processing, creating one or more irreversible and unrelated fingerprint sub-templates. These sub-templates can replace the fingerprint's feature information for fingerprint matching.

[0063] In this way, although the electronic device 100 can successfully recognize a user's wet fingerprint, it may also allow fingerprints from other users that are not pre-stored in the fingerprint template library to be successfully recognized by the electronic device 100. This would reduce the security of fingerprint authentication in the electronic device 100.

[0064] In some embodiments of this application, the fingerprint acquisition area of ​​the electronic device 100 may have a press button with an absorbent film. When a user's finger presses or touches the fingerprint acquisition area, if the press button with the absorbent film absorbs water, the electronic device 100 can determine that the user's finger is wet or sweaty. Then, the electronic device 100 can lower the matching threshold. When the electronic device 100 matches the wet finger image with the template fingerprint, if the matching similarity value or overlap value is greater than or equal to the lowered matching threshold, the electronic device 100 can determine that the fingerprint recognition is successful.

[0065] Thus, while the electronic device 100 can successfully recognize a user's wet fingerprint, it may also allow fingerprints from other users not pre-stored in the fingerprint template library to be successfully recognized by the electronic device 100. This reduces the security of fingerprint authentication in the electronic device 100. Furthermore, electronic devices without an absorbent film cannot recognize wet finger images.

[0066] To improve the security of fingerprint authentication and increase the accuracy of fingerprint recognition of a user's wet finger image by the electronic device 100, this application provides a fingerprint recognition method. The method includes: First, the electronic device 100 acquires a first fingerprint image of the user. Then, the electronic device 100 optimizes the first fingerprint image to obtain an optimized fingerprint image, where the image metrics in the optimized fingerprint image are higher than those in the first fingerprint image. Next, the electronic device 100 matches the optimized fingerprint image with a fingerprint template image. If the match is successful, the electronic device performs a first action. The first action includes unlocking the electronic device 100, making a payment, or launching a privacy application.

[0067] In some embodiments of this application, the image metrics of a fingerprint image may include one or more of texture consistency, quality score, and foreground area.

[0068] Texture consistency measures the continuity of the valleys or ridges in a fingerprint. The value of texture consistency ranges from 0 to 100. A higher texture consistency value indicates a more continuous trend of the valleys or ridges in the fingerprint image. Generally, the texture consistency value of a fingerprint image with wet fingerprints is lower than that of a fingerprint image with dry fingerprints.

[0069] The quality score is used to evaluate the quality of fingerprint images. The quality score ranges from 0 to 100; a higher quality score indicates better fingerprint image quality. Generally, fingerprint images with wet fingers have lower quality scores than those with dry fingers.

[0070] The foreground area represents the usable portion of the fingerprint image acquired by the electronic device 100, relative to the area of ​​the entire fingerprint image. The usable portion of the fingerprint image includes areas containing fingerprint features, such as areas with fingerprint valleys and ridges. The foreground area can range from 0 to 1; a larger value indicates a larger usable portion of the fingerprint image. Generally, the foreground area value of a fingerprint image with a wet finger is lower than that of a fingerprint image with a dry finger.

[0071] In some embodiments of this application, the lock screen of the electronic device 100 may be a secure lock screen that requires unlocking via fingerprint, password, pattern, gesture, or other biometric identifiers. Other biometric identifiers include, but are not limited to, face and voiceprint. Passwords include, but are not limited to, numbers, letters, and a mixture of numbers and letters. Exemplarily, the lock screen of the electronic device 100 may be the lock screen interface 10B shown in FIG1C.

[0072] Figure 2 exemplarily illustrates a fingerprint recognition method provided by an embodiment of this application. As shown in Figure 2, a fingerprint recognition method provided by an embodiment of this application may include the following steps:

[0073] S201, Electronic device 100 detects the user's first operation and collects the user's fingerprint image 1.

[0074] In some embodiments of this application, the first operation may be an operation in which a user presses or touches the fingerprint collection area of ​​the electronic device 100. For example, as shown in FIG1A or FIG1C, the user presses or touches the fingerprint collection area of ​​the electronic device 100 when the screen is off or locked. In response to the user's first operation, the electronic device 100 may collect the user's fingerprint image 1. Exemplarily, the fingerprint image 1 collected by the electronic device 100 may be the fingerprint image shown in FIG1G or FIG1H.

[0075] In some embodiments of this application, the first operation can also be an operation in which a user presses or touches the fingerprint collection area on the payment interface of the electronic device 100. For example, as shown in FIG1D, a user can press or touch the fingerprint collection area 101 on the payment interface 10C shown in FIG1D.

[0076] In some embodiments of this application, the first operation may further include the user clicking the payment control on the payment page and then pressing or touching the fingerprint collection area displayed on the payment interface.

[0077] In some embodiments of this application, the first operation may further include the user clicking the icon of the privacy application and then pressing or touching the fingerprint collection area displayed on the electronic device. The privacy application is an application that requires security verification to start. Security verification includes, but is not limited to, fingerprint verification, facial verification, password verification, and voiceprint verification.

[0078] After the electronic device 100 detects the user's first operation, it can acquire the user's fingerprint image 1. The electronic device 100 can match the user's fingerprint image 1 with fingerprint templates in the fingerprint template library. If the match fails, and the fingerprint image 1 meets condition 1, the electronic device 100 optimizes the fingerprint image 1, for example, by performing AI repair, and then matches the AI-repaired image with fingerprint templates in the fingerprint template library. In this embodiment, the fingerprint image acquired by the electronic device can be called the original image, and the fingerprint image after AI repair can be called the AI-repaired image.

[0079] S202, the electronic device 100 preprocesses the fingerprint image 1 to obtain the fingerprint image 2.

[0080] In some embodiments of this application, the electronic device 100 can preprocess the fingerprint image 1 to obtain a fingerprint image 2. The purpose of preprocessing is to filter out image noise in the fingerprint image 1 and enhance the contrast of the valleys and ridges of the fingerprint in the fingerprint image 1, thereby improving the fingerprint image quality. Compared with the fingerprint image 1, the fingerprint image 2 after preprocessing has clearer fingerprint patterns and less noise.

[0081] In some embodiments of this application, the preprocessing of fingerprint image 1 includes, but is not limited to, smoothing, normalizing, filtering, image enhancement, binarization, and grayscale conversion of fingerprint image 1. The specific preprocessing method is not limited in the embodiments of this application.

[0082] S203, Electronic device 100 extracts features from fingerprint image 2 to obtain fingerprint feature 1.

[0083] The electronic device 100 can extract features from the fingerprint image 2 to obtain fingerprint feature 1. Fingerprint feature 1 may include fingerprint feature points and / or singular points in the fingerprint image 2.

[0084] In some embodiments of this application, the electronic device 100 may skip step S202 and directly extract features from the fingerprint image 1 to obtain fingerprint features 1. It is understood that the embodiments of this application do not limit the specific algorithm for feature extraction.

[0085] S204. Electronic device 100 matches fingerprint feature 1 with fingerprint template to obtain matching result 1.

[0086] Electronic device 100 can match fingerprint feature 1 with fingerprint templates in fingerprint template library to obtain matching result 1.

[0087] In some embodiments of this application, the electronic device 100 can calculate the similarity and / or overlap between fingerprint feature 1 and fingerprint templates in a fingerprint template library.

[0088] In some embodiments of this application, the electronic device 100 can determine the vector line segment pairs of the fingerprint image 2 based on the fingerprint feature 1. For example, the electronic device 100 can connect all fingerprint feature points and singular points in fingerprint feature 1 pairwise with lines until all fingerprint feature points or singular points form line segments with any other point, thereby forming the vector line segment pairs of the fingerprint image 2. The electronic device 100 can compare the vector line segment pairs of the fingerprint image 2 with the vector line segment pairs of fingerprint templates in the fingerprint template library to obtain a similarity result.

[0089] For example, if the fingerprint template library includes a fingerprint template, this fingerprint template can be called fingerprint template 1. Then the electronic device 100 can compare the vector line segment pairs of fingerprint image 2 with the vector line segment pairs of fingerprint template 1 to obtain a similarity score of 1.

[0090] In some embodiments of this application, when a user registers their fingerprint in an electronic device, the device's fingerprint template library can store three fingerprint templates. For example, if a user registers their right thumb fingerprint in an electronic device, the device's fingerprint template library can include three fingerprint templates of that right thumb. These three fingerprint templates can be partial fingerprint images of the right thumb, or one of the three fingerprint templates can be a complete fingerprint image of the right thumb. The three fingerprint templates can be the same or different.

[0091] For example, if the fingerprint template library includes three fingerprint templates, namely fingerprint template 1, fingerprint template 2, and fingerprint template 3, in one possible implementation, the electronic device 100 can compare the vector line segment pairs of fingerprint image 2 with the vector line segment pairs of fingerprint template 1, fingerprint template 2, and fingerprint template 3 respectively to obtain similarity 1, similarity 2, and similarity 3 respectively.

[0092] In some embodiments of this application, the electronic device 100 can first compare the vector line segment pairs of fingerprint image 2 with the vector line segment pairs of a first fingerprint template, for example, fingerprint template 1, to obtain a similarity score of 1. If the similarity score of 1 is greater than the similarity threshold of 1, the electronic device 100 can directly determine whether a match is successful based on the similarity score of 1. If the similarity score of 1 is less than the similarity threshold of 1, the electronic device 100 can then compare the vector line segment pairs of fingerprint image 2 with the vector line segment pairs of a second fingerprint template, for example, fingerprint template 2, to obtain a similarity score of 2. If the similarity score of 2 is greater than the similarity threshold of 1, the electronic device 100 can directly determine whether a match is successful based on the similarity score of 2. If the similarity score of 2 is less than the similarity threshold of 1, the electronic device 100 can then compare the vector line segment pairs of fingerprint image 2 with the vector line segment pairs of a third fingerprint template, for example, fingerprint template 3, to obtain a similarity score of 3. And so on, the electronic device 100 can determine whether it needs to compare with the vector line segment pairs of another fingerprint template in the fingerprint template library based on the similarity results obtained from the comparison. When the similarity obtained from the comparison is less than the similarity threshold 1, the electronic device 100 can compare the vector line segment pair of fingerprint image 2 with the vector line segment pair of another fingerprint template until the electronic device 100 compares the vector line segment pair of fingerprint image 2 with the vector line segment pair of the last fingerprint template to obtain the similarity value.

[0093] In some embodiments of this application, the first fingerprint template in the fingerprint template library may be the first fingerprint template entered into the fingerprint template library according to the order in which the user's fingerprints were entered, or it may be the fingerprint template with the highest priority level in the fingerprint template library according to priority order. The priority order of the fingerprint templates may be set by the user, or it may be determined by the electronic device 100 based on the frequency with which the user uses the fingerprint corresponding to the fingerprint template for identity authentication. Similarly, the second fingerprint template in the fingerprint template library may be the second fingerprint template entered into the fingerprint template library according to the order in which the user's fingerprints were entered, or it may be a fingerprint template with a priority level second only to the first fingerprint template in the fingerprint template library according to priority order. The third fingerprint template in the fingerprint template library may be the third fingerprint template entered into the fingerprint template library according to the order in which the user's fingerprints were entered, or it may be a fingerprint template with a priority level second only to the second fingerprint template in the fingerprint template library according to priority order.

[0094] In some embodiments of this application, the similarity mentioned above, for example, the values ​​of similarity 1, similarity 2, and similarity 3 are all in the range of 0-1, that is, natural numbers greater than or equal to 0 and less than or equal to 1.

[0095] In some embodiments of this application, the electronic device 100 can calculate the overlap between the fingerprint image 2 and the fingerprint template, that is, the ratio of the area of ​​the same part in the fingerprint image 2 and the fingerprint template to the area of ​​the fingerprint image 2. In some embodiments, this overlap can also be referred to as the overlapping area ratio. The value of the overlap ranges from 0 to 1, that is, a natural number greater than or equal to 0 and less than or equal to 1.

[0096] In some embodiments of this application, the electronic device 100 can calculate the center point of the fingerprint image 2. For example, the coordinates of the center point can be the average of the coordinates of all fingerprint feature points in the fingerprint image 2. Then, the electronic device 100 rotates the fingerprint image 2 based on the center point of the fingerprint image 2 and the center point of the fingerprint template, aligning the direction of the rotated fingerprint image 2 with the direction of the fingerprint template. The direction of the fingerprint image 2 can be the direction of the fingerprint ridges in the fingerprint image 2. The direction of the fingerprint template can be the direction of the fingerprint ridges in the fingerprint template. After the direction of the fingerprint image 2 and the direction of the fingerprint template are aligned, the electronic device 100 can calculate the overlap between the fingerprint image 2 and the fingerprint template.

[0097] For example, if the fingerprint template library includes a fingerprint template, this fingerprint template can be referred to as fingerprint template 1. Then the electronic device 100 can calculate the overlap 1 between the fingerprint image 2 and the fingerprint template 1.

[0098] If the fingerprint template library includes three fingerprint templates, namely fingerprint template 1, fingerprint template 2, and fingerprint template 3, then the electronic device 100 can calculate the overlap degree 1 between fingerprint image 2 and fingerprint template 1, the overlap degree 2 between fingerprint image 2 and fingerprint template 2, and the overlap degree 3 between fingerprint image 2 and fingerprint template 3, respectively.

[0099] In some embodiments of this application, the electronic device 100 can first calculate the overlap degree 1 between the fingerprint image 2 and the first fingerprint template, for example, fingerprint template 1. If the overlap degree 1 is greater than the overlap degree threshold 1, the electronic device 100 can directly determine whether the match is successful based on the overlap degree 1. If the overlap degree 1 is less than the overlap degree threshold 1, the electronic device 100 can calculate the overlap degree 2 between the fingerprint image 2 and the second fingerprint template, for example, fingerprint template 2. If the overlap degree 2 is greater than the overlap degree threshold 1, the electronic device 100 can directly determine whether the match is successful based on the overlap degree 2. If the overlap degree 2 is less than the overlap degree threshold 1, the electronic device 100 can calculate the overlap degree 2 between the fingerprint image 2 and the third fingerprint template, for example, fingerprint template 2. And so on, the electronic device 100 can determine whether it is necessary to calculate the overlap degree between the fingerprint image 2 and another fingerprint template in the fingerprint template library based on the calculated overlap degree result. When the calculated overlap degree is less than the overlap degree threshold 1, the electronic device 100 can calculate the overlap degree between the fingerprint image 2 and another fingerprint template, until the electronic device 100 calculates the overlap degree between the fingerprint image and the last fingerprint template.

[0100] S205. Electronic device 100 determines whether the match is successful. If yes, proceed to step S206; otherwise, proceed to step S207.

[0101] Electronic device 100 can determine whether the match is successful based on matching result 1. If the match is successful, electronic device 100 can execute step S206. If the match is unsuccessful, electronic device 100 can execute step S207.

[0102] Match result 1 may include one or more similarities and / or one or more overlaps. The one or more similarities may include similarity 1. The one or more overlaps may include overlap 1.

[0103] In some embodiments of this application, if the matching result 1 contains only similarity 1, and similarity 1 is greater than or equal to similarity threshold 2, then the electronic device 100 can determine that the match is successful. If similarity 1 is less than similarity threshold 2, then the electronic device 100 determines that the match is unsuccessful.

[0104] The similarity threshold 2 can be greater than the similarity threshold 1. For example, the similarity threshold 2 can be 0.95, and the similarity threshold 1 can be 0.8. This application does not limit the specific values ​​of the similarity threshold 2 and the similarity threshold 1.

[0105] In some embodiments of this application, when the matching result 1 contains multiple similarities, the electronic device 100 can calculate an average similarity value 1. If the average similarity value 1 is greater than or equal to a similarity threshold 2, the electronic device 100 can determine that the match is successful. If the average similarity value 1 is less than the similarity threshold 2, the electronic device 100 can determine that the match is unsuccessful. For example, the matching result 1 may contain similarity 1, similarity 2, and similarity 3. The electronic device 100 can calculate an average similarity value 1, which is equal to the sum of similarity 1, similarity 2, and similarity 3 divided by 3. If the average similarity value 1 is greater than or equal to the similarity threshold 2, the electronic device 100 can determine that the match is successful. If the average similarity value 1 is less than the similarity threshold 2, the electronic device 100 can determine that the match is unsuccessful.

[0106] In some embodiments of this application, when the matching result 1 contains multiple similarities, if one or more of the multiple similarities are greater than or equal to the similarity threshold 2, the electronic device 100 can determine that the match is successful. If all of the multiple similarities are less than the similarity threshold 2, the electronic device 100 determines that the match is unsuccessful.

[0107] In some embodiments of this application, if the matching result 1 contains only overlap 1, and the overlap 1 is greater than or equal to the overlap threshold 2, then the electronic device 100 determines that the matching is successful. If the overlap 1 is less than the overlap threshold 2, then the electronic device 100 determines that the matching is unsuccessful.

[0108] In some embodiments of this application, the overlap threshold 2 may be greater than the overlap threshold 1. For example, the overlap threshold 2 may be 0.96, and the overlap threshold 1 may be 0.8. The embodiments of this application do not limit the specific values ​​of the overlap threshold 2 and the overlap threshold 1.

[0109] In some embodiments of this application, when the matching result 1 contains multiple overlaps, the electronic device 100 can calculate an average overlap value 1. If the average overlap value 1 is greater than or equal to an overlap threshold 2, the electronic device 100 can determine that the match is successful. If the average overlap value 1 is less than the overlap threshold 2, the electronic device 100 can determine that the match is unsuccessful. For example, the matching result 1 may contain overlap 1, overlap 2, and overlap 3. The electronic device 100 can calculate an average overlap value 1, which is equal to the sum of overlap 1, overlap 2, and overlap 3 divided by 3. If the average overlap value 1 is greater than or equal to the overlap threshold 2, the electronic device 100 can determine that the match is successful. If the average overlap value 1 is less than the overlap threshold 2, the electronic device 100 can determine that the match is unsuccessful.

[0110] In some embodiments of this application, when the matching result 1 contains multiple overlaps, if one or more of the multiple overlaps are greater than or equal to the overlap threshold 2, the electronic device 100 can determine that the matching is successful. If the multiple overlaps are all less than the overlap threshold 2, the electronic device 100 can determine that the matching is unsuccessful.

[0111] In some embodiments of this application, if the matching result 1 includes similarity 1 and overlap 1, and if similarity 1 is greater than or equal to similarity threshold 2, and overlap 1 is greater than or equal to overlap threshold 2, then the electronic device 100 can determine that the match is successful. If similarity 1 is less than similarity threshold 2, or overlap is less than overlap threshold 2, then the electronic device 100 can determine that the match is unsuccessful.

[0112] S206, Electronic device 100 performs the first action.

[0113] When the user's fingerprint image 1, or fingerprint image 2 obtained by preprocessing fingerprint image 1, is successfully matched with a fingerprint template in the fingerprint template library, the electronic device 100 can perform the first action.

[0114] The first action includes, but is not limited to, unlocking the screen of the electronic device 100, or performing a payment, or launching one or more privacy applications.

[0115] S207. Electronic device 100 determines whether fingerprint image 1 meets the first condition. If yes, it executes step S208; otherwise, it executes step S212.

[0116] When the electronic device 100 determines that the user's fingerprint image 1, or fingerprint image 2 preprocessed from fingerprint image 1, fails to match a fingerprint template in the fingerprint template library, the electronic device 100 can determine whether fingerprint image 1, or fingerprint image 2 preprocessed from fingerprint image 1, satisfies a first condition. If the first condition is met, the electronic device 100 can determine that fingerprint image 1 is a wet finger fingerprint image, or is a wet finger image. Then the electronic device 100 can optimize the wet finger fingerprint image, for example, through AI repair, i.e., the electronic device 100 can execute step S208. If the first condition is not met, the electronic device 100 determines that fingerprint image 1 is not a wet finger image, and the electronic device 100 can execute step S212.

[0117] The first condition may include one or more of the following:

[0118] Condition ①: The texture consistency of fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 is less than the threshold 1.

[0119] Condition ②: The quality score of fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 is less than the threshold 2.

[0120] Condition ③: The foreground area of ​​fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 is less than the threshold 3.

[0121] Condition ④: The similarity between fingerprint image 1, or fingerprint image 2 obtained by preprocessing fingerprint image 1, and the fingerprint template is greater than the similarity threshold 3.

[0122] Condition ⑤: The overlap between fingerprint image 1, or fingerprint image 2 obtained by preprocessing fingerprint image 1, and the fingerprint template is greater than the overlap threshold 3.

[0123] The threshold 1 is greater than 0 and less than 100. For example, the value of threshold 1 can be 50. Threshold 1 can also be other values. This application embodiment does not limit the specific value of threshold 1.

[0124] The threshold 2 is greater than 0 and less than 100. For example, the value of threshold 2 can be 40. Threshold 2 can also be other values, and the specific value of threshold 2 is not limited in this embodiment.

[0125] The threshold 3 is greater than 0 and less than 1. For example, the value of threshold 3 can be 0.9. Threshold 3 can also be other values, and the specific value of threshold 3 is not limited in this embodiment.

[0126] The similarity threshold 3 can be greater than or equal to the similarity threshold 1, and less than the similarity threshold 2. For example, the similarity threshold 3 can be 0.8, and the specific value of the similarity threshold 3 is not limited in this application embodiment.

[0127] The overlap threshold 3 can be greater than or equal to the overlap threshold 1, and less than the overlap threshold 2. For example, the overlap threshold 3 can be 0.8, and the specific value of the overlap threshold 3 is not limited in this application embodiment.

[0128] Among these, texture consistency in condition ①, quality score in condition ②, and foreground area in condition ③ can be referred to as image metrics of a fingerprint image. Similarity in condition ④ and overlap in condition ⑤ can be referred to as matching metrics of a fingerprint image.

[0129] When fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 satisfies the first condition, that is, when it satisfies one or more of the above conditions ①, ②, ③, ④ and ⑤, electronic device 100 can determine that fingerprint image 1 is a user's wet finger image, and electronic device 100 can optimize fingerprint image 1, such as through AI repair.

[0130] S208, electronic device 100 optimizes fingerprint image 1 to obtain fingerprint image 3.

[0131] When fingerprint image 1 meets the first condition, electronic device 100 can optimize fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 to obtain fingerprint image 3.

[0132] In some embodiments of this application, the image metrics of fingerprint image 3 are higher than those of fingerprint image 1 or fingerprint image 2.

[0133] In some embodiments of this application, the image metrics of fingerprint image 3 being higher than those of fingerprint image 1 or fingerprint image 2 may include one or more of the following: fingerprint image 3 having higher texture consistency than fingerprint image 1 or fingerprint image 2, fingerprint image 3 having higher quality score than fingerprint image 1 or fingerprint image 2, and fingerprint image 3 having higher foreground area than fingerprint image 1 or fingerprint image 2. All three of the following—texture consistency, quality score, and foreground area—of fingerprint image 3 may be higher than those of fingerprint image 1 or fingerprint image 2. Alternatively, one or two of the following—texture consistency, quality score, and foreground area—of fingerprint image 3 may be higher than those of fingerprint image 1 or fingerprint image 2. For example, if the texture consistency of fingerprint image 3 is higher than that of fingerprint image 1 or fingerprint image 2, then the quality score and foreground area of ​​fingerprint image 3 are equal to those of fingerprint image 1 or fingerprint image 2.

[0134] In some embodiments of this application, the electronic device 100 optimizes the fingerprint image 1 to obtain the fingerprint image 3, which may include: the electronic device 100 performing AI repair on the fingerprint image 1 or the fingerprint image 2 to obtain the fingerprint image 3. Specifically, as shown in the AI ​​repair process in Figure 3(a), the electronic device 100 can input the fingerprint image 1 or the fingerprint image 2 into the AI ​​repair model, the AI ​​repair model can repair the fingerprint image 1 or the fingerprint image 2, and then the AI ​​repair model can output the fingerprint image 3.

[0135] The training data for this AI restoration model can be multiple pairs of fingerprint images, each pair containing images of wet fingers (when the fingers are wet) and dry fingers (when the fingers are dry). The training process for the AI ​​restoration model can be shown in Figure 3(b). The input to the AI ​​restoration model can be the wet finger image from training image pair 1, and the output can be the restored image 1 obtained by the AI ​​restoration model from the wet finger image. Then, the restored image 1 is compared with the dry finger image from training image pair 1 to obtain the model error. Finally, the model error can be used to correct the AI ​​restoration model.

[0136] In some embodiments of this application, the AI ​​restoration model can be trained by a model server provided by the electronic device 100 manufacturer. The electronic device 100 can download the trained AI restoration model from the model training server. In some embodiments of this application, the electronic device 100 can download an initial AI restoration model from the model server and then train a new AI restoration model based on historical data within the electronic device 100. Compared to the initial AI restoration model, the new AI restoration model can restore wet finger images more clearly and accurately.

[0137] In some embodiments of this application, the AI ​​repair model includes, but is not limited to, deep neural network models, convolutional neural network models, recurrent neural network models, etc. The specific form of the AI ​​repair model is not limited in the embodiments of this application.

[0138] S209, Electronic device 100 extracts features from fingerprint image 3 to obtain fingerprint feature 2.

[0139] Electronic device 100 can extract features from fingerprint image 3 to obtain fingerprint feature 2. Fingerprint feature 2 can be fingerprint feature points and / or singular points in fingerprint image 3.

[0140] In some embodiments of this application, the fingerprint feature points and / or singularities in fingerprint image 3 may be more accurate than those in fingerprint image 1 or fingerprint image 2. In some embodiments of this application, the coordinates of one or more fingerprint feature points extracted by electronic device 100 from fingerprint image 3 may be more accurate than the coordinates of fingerprint feature points extracted from fingerprint image 1 or fingerprint image 2.

[0141] S210, the electronic device 100 matches the fingerprint feature 2 with the fingerprint template to obtain the matching result 2.

[0142] Electronic device 100 can match fingerprint feature 2 with one or more fingerprint templates in the fingerprint template library to obtain matching result 2.

[0143] In some embodiments of this application, the electronic device 100 can calculate the similarity and / or overlap between the fingerprint feature 2 and one or more fingerprint templates in a fingerprint template library.

[0144] Matching result 2 can contain one or more similarities and / or one or more overlaps.

[0145] Step S210 can be referred to the description in step S204, and will not be repeated here.

[0146] S211. The electronic device determines whether the match is successful. If yes, proceed to step S206; otherwise, proceed to step S212.

[0147] Electronic device 100 can determine whether the match is successful based on the matching result 2. If the match is successful, electronic device 100 can execute step S206. If the match is unsuccessful, electronic device 100 can execute step S212.

[0148] Matching result 2 may include one or more similarities and / or one or more overlaps. The one or more similarities may include similarity 4. The one or more overlaps may include overlap 4.

[0149] In some embodiments of this application, if the matching result 2 only contains similarity 4, and if similarity 4 is greater than or equal to the similarity threshold 2, then the electronic device 100 can determine that the match is successful. If similarity 4 is less than the similarity threshold 2, then the electronic device 100 determines that the match is unsuccessful.

[0150] For details regarding the similarity threshold 2, please refer to the description in step S205, which will not be repeated here.

[0151] In some embodiments of this application, when the matching result 2 contains multiple similarities, the electronic device 100 can calculate an average similarity value 2. If the average similarity value 2 is greater than or equal to a similarity threshold 2, the electronic device 100 can determine that the match is successful. If the average similarity value 2 is less than the similarity threshold 2, the electronic device 100 can determine that the match is unsuccessful. For example, the matching result 2 may contain similarity values ​​4, 5, and 6. The electronic device 100 can calculate an average similarity value 2, which is equal to the sum of similarity values ​​4, 5, and 6 divided by 3. If the average similarity value 2 is greater than or equal to the similarity threshold 2, the electronic device 100 can determine that the match is successful. If the average similarity value 2 is less than the similarity threshold 2, the electronic device 100 can determine that the match is unsuccessful.

[0152] In some embodiments of this application, when the matching result 2 contains multiple similarities, if all multiple similarities are greater than or equal to the similarity threshold 2, the electronic device 100 can determine that the match is successful. If one or more of the multiple similarities are less than the similarity threshold 2, the electronic device 100 determines that the match is unsuccessful.

[0153] In some embodiments of this application, if the matching result 2 only contains overlap 4, and the overlap 4 is greater than or equal to the overlap threshold 2, then the electronic device 100 determines that the match is successful. If the overlap 4 is less than the overlap threshold 2, then the electronic device 100 determines that the match is unsuccessful.

[0154] The overlap threshold 2 can be found in the description in step S205, and will not be repeated here.

[0155] In some embodiments of this application, when the matching result 2 contains multiple overlaps, the electronic device 100 can calculate the average overlap. If the average overlap is greater than or equal to the overlap threshold 2, the electronic device 100 can determine that the match is successful. If the average overlap is less than the overlap threshold 2, the electronic device 100 can determine that the match is unsuccessful. For example, the matching result 2 may contain overlaps 4, 5, and 6. The electronic device 100 can calculate the average overlap 2, which is equal to the sum of overlaps 4, 5, and 6 divided by 3. If the average overlap 2 is greater than or equal to the overlap threshold 2, the electronic device 100 can determine that the match is successful. If the average overlap 2 is less than the overlap threshold 2, the electronic device 100 can determine that the match is unsuccessful.

[0156] In some embodiments of this application, if the matching result 1 contains multiple overlaps, and all of the multiple overlaps are greater than or equal to the overlap threshold 2, the electronic device 100 can determine that the matching is successful. If one or more of the multiple overlaps are less than the overlap threshold 2, the electronic device 100 can determine that the matching is unsuccessful.

[0157] In some embodiments of this application, if the matching result 1 includes similarity 4 and overlap 4, and if similarity 1 is greater than or equal to similarity threshold 2, and overlap 1 is greater than or equal to overlap threshold 2, then the electronic device 100 can determine that the match is successful. If similarity 1 is less than similarity threshold 2, or overlap is less than overlap threshold 2, then the electronic device 100 can determine that the match is unsuccessful.

[0158] S212, Electronic device 100 performs the second action.

[0159] When fingerprint image 1, fingerprint image 2, or fingerprint image 3 does not match a fingerprint template in the fingerprint template library, or when the match fails, electronic device 100 may perform a second action.

[0160] The second action includes, but is not limited to, any one of the following:

[0161] ①. Electronic devices 100 may display a message indicating that fingerprint matching was unsuccessful.

[0162] ②. Electronic device 100 may display prompt text, which is used to remind the user to verify fingerprint or authenticate identity through other means again.

[0163] ③. Electronic device 100 can display "payment failed".

[0164] ④. Electronic device 100 may prompt the user that the privacy application failed to start.

[0165] In some embodiments of this application, the process by which the electronic device 100 directly matches the fingerprint image 2 in steps S202-S204 can be referred to as original image matching or original image matching mode. Alternatively, the process by which the electronic device 100 directly matches the fingerprint image 1 can also be referred to as original image matching or original image matching mode. In steps S208-S210, the process by which the electronic device 100 matches the optimized fingerprint image, i.e., fingerprint image 3, can be referred to as AI repair matching or AI repair matching mode.

[0166] Thus, using the method provided in this application embodiment, the electronic device 100 can first match the acquired fingerprint image with a fingerprint template. If the matching fails, the electronic device 100 can determine that the fingerprint image is a wet finger image based on one or more of the image indicators or matching indicators of the fingerprint image. Then, the electronic device 100 can perform AI repair on the wet finger image, resulting in a fingerprint image with higher image indicators than the wet finger image. Finally, the electronic device 100 can match the repaired fingerprint image with the fingerprint template. This improves the fingerprint recognition rate of wet finger images. The method provided in this application embodiment utilizes an AI repair algorithm to repair wet or sweaty fingerprint images, improving the accuracy and stability of fingerprint recognition. Furthermore, it avoids the embarrassment and inconvenience of users being unable to unlock the device due to wet or sweaty fingers, improving the user experience. The fingerprint recognition method provided in this application embodiment does not require lowering the matching threshold or downsampling based on the quality of the collected data, thereby ensuring the security of the device.

[0167] In some embodiments of this application, the electronic device 100 may not directly match the acquired fingerprint image. The electronic device 100 can determine whether the fingerprint image meets a second condition. If the second condition is not met, the electronic device 100 can choose the original image matching mode; if the second condition is met, the electronic device 100 can choose the AI-repaired image matching mode. The original image matching mode involves the electronic device 100 matching the acquired fingerprint image with a fingerprint template. The AI-repaired image matching mode involves the electronic device 100 first performing AI repair on the acquired fingerprint image and then matching the repaired fingerprint image with the fingerprint template. Thus, when the fingerprint image is from a dry finger, the electronic device 100 can directly match the fingerprint image with the fingerprint template, improving fingerprint recognition efficiency. When the fingerprint image is from a wet finger, the electronic device 100 can perform AI repair before matching, improving the accuracy and stability of fingerprint recognition for wet finger images. Furthermore, it avoids the embarrassment and inconvenience of users being unable to unlock the device due to wet or sweaty fingers, improving the user experience.

[0168] Figure 4 illustrates a flowchart of another fingerprint recognition method provided in an embodiment of this application. As shown in Figure 4, the fingerprint recognition method provided in this application embodiment may include the following steps:

[0169] S401, Electronic device 100 detects the user's first operation and collects the user's fingerprint image 1.

[0170] Step S401 can be referred to step S201, and will not be repeated here.

[0171] S402. Electronic device 100 determines whether fingerprint image 1 meets the second condition. If yes, it executes step S409; otherwise, it executes step S403.

[0172] The second condition may include one or more of the following:

[0173] Condition ①: The texture consistency of fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 is less than the threshold 1.

[0174] Condition ②: The quality score of fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 is less than the threshold 2.

[0175] Condition ③: The foreground area of ​​fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 is less than the threshold 3.

[0176] When fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 meets the second condition, that is, when it meets one or more of the above conditions ①, ②, and ③, the electronic device 100 can select the AI ​​repair image mode, that is, execute the following steps S409-S412. When fingerprint image 1 or fingerprint image 2 obtained by preprocessing fingerprint image 1 does not meet the second condition, the electronic device 100 can select the original image matching mode, that is, execute the following steps S403-S406.

[0177] S403, Electronic device 100 preprocesses fingerprint image 1 to obtain fingerprint image 2.

[0178] S404, Electronic device 100 extracts features from fingerprint image 2 to obtain fingerprint feature 1.

[0179] S405, Electronic device 100 matches fingerprint feature 1 with fingerprint template to obtain matching result 1.

[0180] S406. Electronic device 100 determines whether the match is successful. If yes, proceed to step S407; otherwise, proceed to step S408.

[0181] Steps S403-S406 can be referred to steps S202-S205 above, and will not be repeated here.

[0182] S407, Electronic device 100 performs the first action.

[0183] Step S407 can be referred to step S206 above, and will not be repeated here.

[0184] S408, Electronic device 100 performs the second action.

[0185] Step S408 can be referred to step S212 above, and will not be repeated here.

[0186] S409, electronic device 100 optimizes fingerprint image 1 to obtain fingerprint image 3.

[0187] S410 and electronic device 100 extract features from fingerprint image 3 to obtain fingerprint feature 2.

[0188] S411, Electronic device 100 matches fingerprint feature 2 with fingerprint template to obtain matching result 2.

[0189] S412. Electronic device 100 determines whether the match is successful. If yes, proceed to step S407; otherwise, proceed to step S408.

[0190] Steps S409-S412 can be referred to in steps S208-S211, and will not be repeated here.

[0191] In some embodiments of this application, the process by which the electronic device 100 directly matches the fingerprint image 2 in steps S403-S406 can be referred to as original image matching or original image matching mode. Alternatively, the process by which the electronic device 100 directly matches the fingerprint image 1 can also be referred to as original image matching or original image matching mode. In steps S409-S412, the process by which the electronic device 100 matches the optimized fingerprint image, i.e., fingerprint image 3, can be referred to as AI repair matching or AI repair matching mode.

[0192] In the fingerprint identification method provided in this application, the electronic device 100 can first calculate the image indicators of the fingerprint image and select the corresponding matching mode based on the indicators. The image indicators may include: calculating one or more of the following fingerprint image indicators: texture consistency, quality score, and foreground area. Then, the electronic device can select a matching mode based on the image indicators. Only when a second condition is met is the AI-repaired matching mode selected; otherwise, the original image matching mode is selected. During AI image repair matching, the electronic device 100 first performs AI repair on the fingerprint image, and then compares the repaired image with the fingerprint template stored in the device. If the comparison is successful, the device unlocks.

[0193] In this embodiment, the user unlocks the device using both a dry finger and a wet finger. The wet finger unlocks successfully, and the unlocking time is longer than the dry finger unlocking time. When the user uses a specially textured prosthetic finger and a normal finger (i.e., the user's real finger) wet to unlock, only the normal finger wet can unlock.

[0194] Furthermore, after a user successfully unlocks the device multiple times with a wet finger, the electronic device 100 can store the image of the user's wet finger in a fingerprint template library. If the number of fingerprint templates stored in the library reaches its limit, the electronic device 100 can first delete some infrequently used fingerprint templates before storing the user's wet finger image in the library. This way, when the user subsequently uses the wet finger to unlock, the electronic device 100 can directly use the original image matching mode to achieve a successful match, without needing to perform AI-based matching repair. Thus, after a user has successfully unlocked the device multiple times with a wet finger, the unlocking time for subsequent attempts to unlock with a wet finger can be significantly reduced.

[0195] In some embodiments of this application, the first threshold can be threshold 1, the second threshold can be threshold 2, the third threshold can be threshold 3, the fourth threshold can be similarity threshold 3, and the fifth threshold can be overlap threshold 3. The first fingerprint image can be fingerprint image 1 or fingerprint image 2, and the second fingerprint image can be fingerprint image 3.

[0196] The exemplary electronic device 100 provided in the embodiments of this application will be introduced first below.

[0197] Figure 5 is a schematic diagram of the structure of the electronic device 100 provided in the embodiment of this application.

[0198] The following detailed description uses electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0199] Electronic device 100 can be a mobile phone, tablet computer, laptop computer, smartwatch, smart bracelet, or other device that supports fingerprint unlocking, fingerprint authentication, fingerprint payment, etc.

[0200] Electronic device 100 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0201] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0202] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0203] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0204] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0205] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0206] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0207] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0208] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0209] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0210] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0211] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0212] The SIM interface can be used to communicate with the SIM card interface 195 to transmit data to or read data from the SIM card.

[0213] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0214] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0215] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0216] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0217] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0218] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0219] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0220] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0221] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0222] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0223] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0224] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0225] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0226] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0227] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0228] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0229] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0230] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0231] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0232] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as facial recognition, fingerprint recognition, mobile payment, etc.). The data storage area may store data created during the use of electronic device 100 (such as facial information template data, fingerprint information templates, etc.). Furthermore, internal memory 121 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0233] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0234] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0235] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0236] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0237] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0238] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0239] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0240] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0241] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0242] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0243] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0244] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0245] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 may use the proximity sensor 180G to detect when a user holds the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0246] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0247] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0248] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0249] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0250] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0251] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0252] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0253] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and detach from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, and other SIM cards. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication.

[0254] Figure 6 is a software structure block diagram of an electronic device 100 according to an embodiment of this application.

[0255] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers, from top to bottom: the application layer, the application framework layer, the runtime and system libraries, and the kernel layer.

[0256] The application layer can include a series of application packages.

[0257] As shown in Figure 6, the application package may include applications (also known as apps) such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0258] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0259] As shown in Figure 6, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0260] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0261] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0262] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0263] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0264] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0265] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog-style notifications on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0266] The runtime consists of the core libraries and the virtual machine. The runtime is responsible for system scheduling and management.

[0267] The core library consists of two parts: one part is the functionalities that the programming language (e.g., Java) needs to call, and the other part is the system's core library.

[0268] The application layer and application framework layer run in a virtual machine. The virtual machine executes the programming files (e.g., .jave files) of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0269] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0270] The Surface Manager is used to manage the display subsystem and provides the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0271] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0272] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0273] A 2D graphics engine is a graphics engine for 2D drawing.

[0274] The kernel layer is the layer between hardware and software. The kernel layer includes at least the display driver, camera driver, audio driver, sensor driver, and virtual card driver.

[0275] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the electronic device 100.

[0276] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a touch click as an example, where the corresponding control is the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through camera 193.

[0277] Figure 7 shows a schematic diagram of one structure of electronic device 100.

[0278] As shown in Figure 7, this application provides an electronic device 100, which includes a fingerprint sensor, a processor, and a memory. The fingerprint sensor, processor, and memory can be directly or indirectly connected via circuit wiring.

[0279] The fingerprint sensor can be used to acquire a first fingerprint; the processor can be used to perform fingerprint recognition on the first fingerprint acquired by the fingerprint sensor; the processor can be used to determine whether the first fingerprint meets the first condition if it is determined that the first fingerprint does not match the fingerprint template. The processor can also be used to optimize the first fingerprint if it meets the first condition, for example, by inputting the first fingerprint into an AI repair model to obtain a second fingerprint, where the image index of the second fingerprint is higher than that of the first fingerprint. The processor can also be used to match the second fingerprint with the fingerprint template. Finally, the processor can be used to unlock the electronic device 100 if it is determined that either the first fingerprint or the second fingerprint matches the fingerprint template.

[0280] The memory can be used to store a fingerprint template library, which contains fingerprint templates entered by the user.

[0281] In some embodiments of this application, the fingerprint sensor may be disposed below the display screen of the electronic device 100, on the side of the electronic device 100, or on the back of the electronic device 100. The specific location of the fingerprint sensor in the electronic device 100 is not limited in the embodiments of this application.

[0282] In some embodiments of this application, the fingerprint sensor may be an optical fingerprint sensor, a capacitive (i.e., semiconductor) fingerprint sensor, or a radio frequency fingerprint sensor. The specific type of fingerprint sensor is not limited in the embodiments of this application.

[0283] In some embodiments of this application, the first fingerprint may be fingerprint image 1 or fingerprint image 2 mentioned above. The second fingerprint may be fingerprint image 3 mentioned above.

[0284] In some embodiments of this application, the specific process by which the processor performs fingerprint recognition on the first fingerprint can be referred to the description in steps S202-S204 above.

[0285] In some embodiments of this application, the first condition can be referred to the description in step S207 above, and will not be repeated here.

[0286] In some embodiments of this application, the processor optimizes the first fingerprint to obtain a second fingerprint and identifies the second fingerprint, as described in steps S208-S210 above.

[0287] In some embodiments of this application, the processor can also be used to perform payment when it is determined that the first fingerprint matches the fingerprint template.

[0288] In some embodiments of this application, the processor can also be used to perform payment if it is determined that the second fingerprint matches the fingerprint template.

[0289] In some embodiments of this application, the processor can also be used to identify the first fingerprint if the first fingerprint does not meet the second condition. The processor can also be used to optimize the first fingerprint if the first fingerprint meets the second condition, for example, by inputting the first fingerprint into an AI restoration model to obtain a second fingerprint, where the image index of the second fingerprint is higher than that of the first fingerprint. The processor can also be used to match the second fingerprint with a fingerprint template; the processor can also be used to unlock the electronic device 100 if it is determined that either the first fingerprint or the second fingerprint matches the fingerprint template.

[0290] Figure 8 shows a flowchart of a fingerprint recognition method. As shown in Figure 8, a fingerprint recognition method provided in this application embodiment may include the following steps:

[0291] S801, Electronic device 100 detects the user's first operation and collects the user's first fingerprint image.

[0292] Step S801 can be referred to step S201, and will not be repeated here.

[0293] S802, the electronic device 100 optimizes the first fingerprint image to obtain the second fingerprint image.

[0294] Step S802 can be referred to step S208, and will not be repeated here.

[0295] S803: Confirm that the second fingerprint image matches the fingerprint template stored in the electronic device 100, and unlock the electronic device 100.

[0296] Step S803 can be referred to steps S211 and S206, and will not be repeated here.

[0297] The fingerprint recognition method provided in this application allows the electronic device 100 to optimize fingerprint recognition when the fingerprint image is from a wet finger. For example, it can perform AI-based repair before matching, improving the accuracy and stability of fingerprint recognition for wet finger images. Furthermore, it avoids the embarrassment and inconvenience of users being unable to unlock the device due to wet or sweaty fingers, thus enhancing the user experience.

[0298] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0299] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0300] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0301] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A fingerprint recognition method, characterized in that, include: The electronic device detects the user's first action and captures the user's first fingerprint image; The electronic device optimizes the first fingerprint image to obtain a second fingerprint image; Once it is confirmed that the second fingerprint image successfully matches the fingerprint template stored in the electronic device, the electronic device is unlocked.

2. The method according to claim 1, characterized in that, Before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method further includes: It is determined that the first fingerprint image fails to match the fingerprint template and that the first fingerprint image satisfies the first condition.

3. The method according to claim 2, characterized in that, The first condition includes one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold, the quality score of the first fingerprint image is less than a second threshold, the foreground area of ​​the first fingerprint image is less than a third threshold, the similarity between the first fingerprint image and the fingerprint template is greater than a fourth threshold, and the overlap between the first fingerprint image and the fingerprint template is greater than a fifth threshold.

4. The method according to any one of claims 1-3, characterized in that, The image index of the second fingerprint image is higher than that of the first fingerprint image.

5. The method according to claim 4, characterized in that, The image metrics of the second fingerprint image include one or more of the following: texture consistency of the second fingerprint image, quality score of the second fingerprint image, and foreground area of ​​the second fingerprint image. The image metrics of the first fingerprint image include one or more of the following: texture consistency of the first fingerprint image, quality score of the first fingerprint image, and foreground area of ​​the first fingerprint image.

6. The method according to claim 5, characterized in that, The image metrics of the second fingerprint image being higher than those of the first fingerprint image include one or more of the following: the texture consistency of the second fingerprint image being higher than the fingerprint consistency of the first fingerprint image, the quality score of the second fingerprint image being higher than the quality score of the first fingerprint image, and the foreground area of ​​the second fingerprint image being higher than the foreground area of ​​the first fingerprint image.

7. The method according to claim 1, characterized in that, Before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method further includes: The first fingerprint image is determined to satisfy the second condition.

8. The method according to claim 7, characterized in that, The second condition includes one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold, the quality score of the first fingerprint image is less than a second threshold, and the foreground area of ​​the first fingerprint image is less than a third threshold.

9. A fingerprint recognition method, characterized in that, include: The electronic device detects the user's first action and captures the user's first fingerprint image; The electronic device optimizes the first fingerprint image to obtain a second fingerprint image; Once it is determined that the second fingerprint image successfully matches the fingerprint template stored in the electronic device, the electronic device executes the payment.

10. The method according to claim 9, characterized in that, Before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method further includes: It is determined that the first fingerprint image fails to match the fingerprint template and that the first fingerprint image satisfies the first condition.

11. The method according to claim 10, characterized in that, The first condition includes one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold, the quality score of the first fingerprint image is less than a second threshold, the foreground area of ​​the first fingerprint image is less than a third threshold, the similarity between the first fingerprint image and the fingerprint template is greater than a fourth threshold, and the overlap between the first fingerprint image and the fingerprint template is greater than a fifth threshold.

12. The method according to any one of claims 9-11, characterized in that, The image index of the second fingerprint image is higher than that of the first fingerprint image.

13. The method according to claim 12, characterized in that, The image metrics of the second fingerprint image include one or more of the following: texture consistency of the second fingerprint image, quality score of the second fingerprint image, and foreground area of ​​the second fingerprint image. The image metrics of the first fingerprint image include one or more of the following: texture consistency of the first fingerprint image, quality score of the first fingerprint image, and foreground area of ​​the first fingerprint image.

14. The method according to claim 13, characterized in that, The image metrics of the second fingerprint image being higher than those of the first fingerprint image include one or more of the following: the texture consistency of the second fingerprint image being higher than the fingerprint consistency of the first fingerprint image, the quality score of the second fingerprint image being higher than the quality score of the first fingerprint image, and the foreground area of ​​the second fingerprint image being higher than the foreground area of ​​the first fingerprint image.

15. The method according to claim 9, characterized in that, Before the electronic device optimizes the first fingerprint image to obtain the second fingerprint image, the method further includes: The first fingerprint image is determined to satisfy the second condition.

16. The method according to claim 15, characterized in that, The second condition includes one or more of the following: the texture consistency of the first fingerprint image is less than a first threshold, the quality score of the first fingerprint image is less than a second threshold, and the foreground area of ​​the first fingerprint image is less than a third threshold.

17. An electronic device, characterized in that, include: A fingerprint sensor, one or more processors, and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-8 or 9-16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-8 or 9-16.

Images