Fingerprint stain removal model construction method and fingerprint recognition sensor

Abstract

The present invention provides a method for constructing a fingerprint stain removal model. The method comprises: using a simulated stain algorithm to simulate a sample fingerprint image to construct multiple different simulated stained fingerprint images; and inputting the multiple different simulated stained fingerprint images into a convolutional neural network training model for training to obtain a fingerprint stain removal model. Furthermore, the present invention provides a fingerprint recognition method, a fingerprint recognition sensor, and an electronic device. When recognizing a stained fingerprint image, the fingerprint stain removal model can remove the stains and restore the fingerprint image to a stained-free state, significantly reducing the impact of stains on fingerprint recognition and thereby improving the recognition rate of stained fingerprints.

Description

Technical Field

[0001] The present invention relates to the field of fingerprint recognition technology, and in particular to a fingerprint stain removal model construction method, a fingerprint recognition method, a fingerprint recognition sensor, and electronic equipment. Background Art

[0002] With the popularization of full-screen technology and fingerprint recognition technology in mobile terminals, the demand for optical fingerprint sensors has increased rapidly, and under-screen optical fingerprint technology has gradually become the mainstream technology of biometric recognition technology for electronic devices.

[0003] In optical fingerprint technology, image quality significantly impacts fingerprint recognition rates. Water or oil stains on the finger can significantly impact the clarity and integrity of the fingerprint. For example, if fingerprint ridges are contiguous or a large area of ​​fingerprint information is covered, these stains can negatively impact recognition rates. Summary of the Invention

[0004] In view of this, it is necessary to provide a fingerprint stain removal model construction method, recognition method, sensor and electronic device that improve the recognition rate of stained fingerprints.

[0005] In a first aspect, this embodiment provides a method for constructing a fingerprint stain removal model, the method comprising:

[0006] Using a simulated stain algorithm to simulate the same sample fingerprint image to construct multiple different simulated stain fingerprint images; and

[0007] The multiple different simulated stained fingerprint images are input into a convolutional neural network training model for training to obtain a fingerprint stain removal model.

[0008] In a second aspect, an embodiment of the present invention further provides a fingerprint recognition method, the fingerprint recognition method comprising:

[0009] Obtaining the fingerprint image to be verified;

[0010] Determining whether the fingerprint image to be verified has stains;

[0011] If there is stain on the fingerprint image to be verified, input the fingerprint image to be verified into a fingerprint stain removal model to perform calculation to obtain a fingerprint image to be verified after the stain is removed, the fingerprint stain removal model being constructed by the fingerprint stain removal model construction method described above; and

[0012] Verifying the fingerprint image to be verified after stain removal with the sample fingerprint image to obtain a verification result; or

[0013] If there is no stain on the fingerprint image to be verified, the fingerprint image to be verified is verified with the sample fingerprint image to obtain a verification result.

[0014] In a third aspect, an embodiment of the present invention further provides a fingerprint recognition sensor, the fingerprint recognition sensor comprising:

[0015] a storage medium for storing fingerprint recognition program instructions; and

[0016] The processor is used to execute fingerprint recognition program instructions to implement the above fingerprint recognition method.

[0017] In a fourth aspect, an embodiment of the present invention further provides an electronic device, which includes a main body and the above-mentioned fingerprint recognition sensor arranged on the main body.

[0018] The above-mentioned fingerprint stain removal model construction method can create a fingerprint stain removal model. When identifying a fingerprint image with stains, the stains can be removed to restore the fingerprint image when the fingerprint is free of stains, thereby greatly reducing the impact of stains on fingerprint recognition, thereby improving the recognition rate of fingerprints with stains. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0020] Figure 1 A schematic flow chart of a method for constructing a fingerprint stain removal model according to an embodiment.

[0021] Figure 2 This is a schematic diagram of the first sub-process of a fingerprint stain removal model construction method provided by an embodiment.

[0022] Figure 3-5 A schematic diagram of a simulated stained fingerprint image provided by an embodiment.

[0023] Figure 6 This is a schematic diagram of the second sub-process of the fingerprint stain removal model construction method provided by one embodiment.

[0024] Figure 7 A schematic diagram of the internal program modules of a convolutional neural network model provided in one embodiment.

[0025] Figure 8 A schematic diagram of a fingerprint recognition method according to an embodiment of the present invention.

[0026] Figure 9 A schematic diagram of an electronic device applying the fingerprint recognition method provided by an embodiment.

[0027] Figure 10 A schematic diagram of the internal structure of a fingerprint recognition sensor to which the fingerprint recognition method is applied is provided in an embodiment.

[0028] Component number description

[0029] Electronic device 100 fingerprint recognition sensor 1

[0030] Memory 11 Processor 12

[0031] Fingerprint recognition program instructions 110 main body 101

[0032] Sample Fingerprint Image Library 112 Fingerprint Stain Removal Model 113

[0033] Convolutional Neural Network Model 20 Image Feature Processing Model 21

[0034] Adversarial Network Model 22 Downsampling Model 210

[0035] Upsampling Model 212 Generative Model 220

[0036] Discriminant model 222 Sample fingerprint image S

[0037] First simulated stained fingerprint image S1 Second simulated stained fingerprint image S2

[0038] The third simulated stained fingerprint image S3

[0039] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. The drawings in the specification illustrate examples of embodiments of the present invention. It will be understood that the scales shown in the drawings in the specification are not the scales of the actual implementation of the present invention. They are only for illustrative purposes and are not drawn according to the original size. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0041] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate. In other words, the embodiments described are implemented according to an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, may also encompass other content. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or apparatus.

[0042] It should be noted that the descriptions of "first", "second", etc. in the present invention are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include one or more of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0043] The present invention provides a method for constructing a fingerprint stain removal model, a recognition method, a sensor, and an electronic device for improving the recognition rate of stained fingerprints. When a fingerprint has stains, such as water stains or oil stains, the fingerprint image sensed by the fingerprint recognition sensor is often covered by the stains or the ridges or valleys cannot be accurately distinguished. Therefore, when recognizing a fingerprint with stains, the recognition rate is greatly affected by the stains. The fingerprint stain removal model created by the method provided by the present invention can first remove the stains when recognizing a fingerprint image with stains, thereby restoring the fingerprint image when the fingerprint is stain-free, greatly reducing the impact of stains on fingerprint recognition, thereby improving the recognition rate of fingerprints with stains.

[0044] Please see Figure 1 The method for constructing a fingerprint stain removal model provided by the first embodiment includes the following steps.

[0045] Step S101, using the simulated stain algorithm to simulate the same sample fingerprint image to construct multiple different simulated stain fingerprint images. It can be understood that when the user uses the electronic device 100 (such as Figure 9When adding a fingerprint password, you need to enter your fingerprint. The sample fingerprint image is obtained when the user sets the fingerprint password using the fingerprint recognition sensor on the electronic device. The sample fingerprint image is stored in the sample fingerprint image library 112, which is stored in the local memory of the electronic device (such as Figure 10 As shown), it can also be stored in a cloud server. The fingerprint recognition sensor adopts an optical fingerprint sensor or a capacitive fingerprint sensor. The electronic device 100 can be, but is not limited to, a mobile phone, a tablet computer or a laptop computer. In some feasible embodiments, the stain fingerprint image is a square fingerprint image. Specifically, step S101 first converts the sample fingerprint image into a square fingerprint image training sample; then uses the simulated stain algorithm to perform simulation operations on the square sample fingerprint image to construct a plurality of different simulated stain fingerprint images. Specifically, using the simulated stain algorithm to perform stain simulation operations on the sample fingerprint image mainly adds noise to the sample fingerprint image to simulate the fingerprint image morphology when stains exist in the sample fingerprint image. How to use the simulated stain algorithm to construct a plurality of different simulated stain fingerprint images will be introduced in detail below.

[0046] Step S103: Input multiple simulated stained fingerprint images into the convolutional neural network training model for training to obtain a fingerprint stain removal model. Figure 7 In this embodiment, the convolutional neural network training model 20 includes an image feature processing model 21 and an adversarial network model 22. The adversarial network model 22 is a generative adversarial network, such as a standard GAN network, or variant GAN networks such as WGAN and DCGAN. The adversarial network model 22 includes a generative model 220 and a discriminative model 222. The image feature processing model 21 is an encoding and decoding model. The image feature processing model 21 includes a downsampling model 210 and an upsampling model 212. The downsampling model 210 can also be called an encoding model; the upsampling model 212 can also be called a decoding model. Specifically, step S103 specifically includes: inputting multiple different simulated stained fingerprint images one by one into the image feature processing model 21 to obtain multiple feature images; and inputting the multiple feature images one by one into the adversarial network model 22 to obtain a fingerprint stain removal model. How the image feature processing model 21 and the adversarial network model 22 form different stained fingerprint recognition models will be described in detail below.

[0047] In the above embodiment, multiple different simulated stained fingerprint images can be trained to form a fingerprint stain removal model. This stain removal model can then be used to remove stains from stained fingerprint images, restoring the fingerprint image to a clean state. That is, when verifying a fingerprint, the fingerprint image can be restored regardless of whether stains are present, thereby improving the fingerprint recognition rate. Furthermore, when the simulated stain algorithm simulates the same sample fingerprint image to construct multiple different simulated stained fingerprint images, the sample fingerprint image is first converted to a square sample fingerprint image. This ensures that the converted sample fingerprint image has the same input image size as required by the convolutional neural network training model.

[0048] Please refer to Figure 2 , which is a schematic diagram of a sub-flow of an embodiment of step S101. The simulated stained fingerprint image includes a first simulated stained fingerprint image, a second simulated stained fingerprint image, and a third simulated stained fingerprint image. Specifically, step S101 includes the following steps.

[0049] Step S201: Calculate the sample fingerprint image using the morphological dilation algorithm to obtain a plurality of first simulated stained fingerprint images. Figure 3 As shown, morphological dilation is performed on the sample fingerprint image S to generate multiple blurred first simulated stained fingerprint images S1. This simulates the morphology of the blurred fingerprint image captured by the optical sensor when the thickness of the ridges and valleys is inconsistent due to water or oil stains on the user's finger and inconsistent pressure.

[0050] Step S203: Calculate the sample fingerprint image using the elliptical mask algorithm to obtain multiple second simulated stained fingerprint images. Figure 4 As shown in the figure, an elliptical mask noise is added to the sample fingerprint image S to generate multiple second simulated stained fingerprint images S2 with multiple regions blocked. In other words, this simulates the situation where multiple regions of the fingerprint image captured by the optical sensor are blocked when the elliptical concave area in the fingerprint is blocked by water or oil stains on the user's finger.

[0051] Step S205: Calculate the sample fingerprint image using the random Perlin noise algorithm to obtain multiple third simulated stained fingerprint images. Figure 5 As shown in the figure, Perlin noise is added to the sample fingerprint image S to generate multiple third simulated stained fingerprint images S3 with noise appearing in different locations. In other words, this simulates the situation where the fingerprint ridges and valleys are obscured by water or oil stains on the user's finger, resulting in multiple blurred areas in the fingerprint image captured by the optical sensor.

[0052] It is understandable that the above embodiment uses three different stain simulation algorithms to obtain multiple simulated stained fingerprint images. In some feasible embodiments, one or two of these algorithms may be selected to perform stain simulation operations on sample fingerprint images. In some feasible embodiments, other image noise addition methods may be used to perform stain simulation operations on sample fingerprint images. That is, the above stain simulation algorithm may be one, three, or more stain simulation algorithms, which is not limited here.

[0053] Please refer to Figure 6 and Figure 7 , Figure 6 It is a schematic diagram of a flow chart of an embodiment of step S103. Step S103 includes the following steps.

[0054] Step S601: Downsample multiple different simulated stained fingerprint images to obtain multiple downsampled feature images corresponding to the multiple simulated stained fingerprint images. Specifically, the simulated stained fingerprint images are downsampled to form feature images with a smaller data volume than the simulated stained fingerprint images. More specifically, each simulated stained fingerprint image is divided into blocks of size s*s. The image within each block is converted into a pixel, and the value of this pixel is the mean value of all pixels within the corresponding block.

[0055] Step S603 converts the multiple downsampled feature images into multiple upsampled feature images. This means restoring the downsampled feature images into multiple upsampled feature images of the same size as the simulated stained fingerprint images. More specifically, interpolation is performed on the multiple downsampled images, resulting in a nearly clear image.

[0056] Step S605: input the up-sampled feature images one by one into the generation model to generate corresponding multiple expected fingerprint images.

[0057] Step S607: using a discriminant model, the plurality of expected fingerprint images are compared with the sample fingerprint image to obtain a comparison result.

[0058] Step S609: When the comparison result meets the preset conditions, the current parameters of the convolutional neural network training model are determined to form a fingerprint stain removal model.

[0059] It can be understood that the above steps S601 and S603 describe the process of inputting multiple different simulated stained fingerprint images into the image feature processing model one by one to obtain multiple feature images, that is, the process of encoding and decoding the simulated stained fingerprint images.

[0060] In the above embodiment, the encoding and decoding algorithm and the adversarial network algorithm are used to complete the training of the simulated stained fingerprint image, thereby obtaining a better fingerprint stain removal model.

[0061] Please see Figure 8 , which is a flow chart of a fingerprint recognition method provided by an embodiment. The fingerprint recognition method includes the following steps.

[0062] Step S801: Acquire a fingerprint image to be verified. Specifically, the fingerprint image to be verified is generated when a user presses a finger on a sensing area of ​​a fingerprint sensor in an electronic device to unlock a locked electronic device or to unlock a function executed in the electronic device.

[0063] Step S803: Determine whether the fingerprint image to be verified is contaminated. Specifically, the fingerprint image to be verified is analyzed to determine whether it is clear and whether there are any obscured areas. If contaminated, proceed to step S805; if not, proceed to step S809.

[0064] Step S805: Input the fingerprint image to be verified into the fingerprint contamination removal model to calculate and obtain the fingerprint image to be verified after the contamination is removed. The fingerprint contamination removal model 113 is stored in the fingerprint recognition sensor 1 (eg Figure 10 As shown), the fingerprint stain removal model 113 is created due to the above-mentioned fingerprint stain removal model construction method.

[0065] Step S807: Verify the fingerprint image to be verified after stain removal with the sample fingerprint image to obtain a verification result.

[0066] Step S809: Verify the fingerprint image to be verified with the sample fingerprint image to obtain a verification result. The sample fingerprint image is stored in the fingerprint recognition sensor.

[0067] In the above embodiment, during fingerprint recognition, the stain removal model is used to perform stain recognition and stain removal on the fingerprint image to be verified, thereby restoring the stained fingerprint image to a clear image, so that the stained fingerprint image can be verified, thereby improving the fingerprint recognition rate.

[0068] In the above embodiment, the fingerprint image to be verified is judged for stains, and the stained fingerprint image to be verified is identified and then input into the fingerprint stain removal model for verification. This eliminates the need for stain removal operations on unstained fingerprint images, significantly reducing the amount of computation. It is understood that in some feasible embodiments, where the amount of computation is not a concern, the fingerprint image to be verified may be directly input into the fingerprint stain removal model, thereby similarly achieving verification of the stained fingerprint image in the fingerprint image to be verified.

[0069] Please refer to Figure 10, which is a schematic diagram of the internal structure of the fingerprint recognition sensor 1. The fingerprint recognition sensor 1 includes a storage medium 11 and a processor 12. The storage medium 11 is used to store fingerprint recognition program instructions 110, a sample fingerprint image library 112, a fingerprint stain removal model 113, etc. The processor 12 is used to execute the fingerprint recognition program instructions to implement the above-mentioned fingerprint recognition method. The fingerprint recognition method is the same as the above-mentioned fingerprint recognition methods and will not be repeated here.

[0070] Figure 10 Only the fingerprint recognition sensor 1 having components 11 and 12 is shown. It can be understood by those skilled in the art that Figure 10 The structure shown does not constitute a limitation on the fingerprint recognition sensor 1 , and the fingerprint recognition sensor 1 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0071] Please refer to Figure 9 The fingerprint recognition sensor 1 is applied to an electronic device 100. The electronic device 100 may be a mobile phone, a tablet computer, a laptop computer, an access control system, or the like. In this embodiment, the specific application of the fingerprint recognition sensor 1 is described using the mobile phone as an example. The electronic device 100 includes a main body 101 and a fingerprint recognition sensor 1 disposed on the main body 101. The sensing area 10 of the fingerprint recognition sensor 1 is disposed on the outer surface of the electronic device 100. In this embodiment, the fingerprint recognition sensor 1 is an optical fingerprint sensor.

[0072] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for constructing a fingerprint stain removal model, characterized in that: The fingerprint stain removal model construction method comprises: Performing simulation operations on the same sample fingerprint image using a simulated stain algorithm to construct multiple different simulated stained fingerprint images, wherein the simulated stain algorithm adds noise to the sample fingerprint image to simulate the fingerprint image morphology when stains are present in the sample fingerprint image, wherein the simulated fingerprint image morphology when stains are present in the sample fingerprint image is an image simulating water stains or oil stains on the user's finger; and Inputting the plurality of different simulated stained fingerprint images into a convolutional neural network training model for training to obtain a fingerprint stain removal model; the stain removal model is used to remove stains from the stained fingerprint image when identifying the stained fingerprint image, thereby restoring the fingerprint image without stains; The convolutional neural network training model includes an image feature processing model and an adversarial network model. The inputting of the plurality of different simulated stained fingerprint images into the convolutional neural network training model for training to obtain a fingerprint stain removal model specifically includes: Inputting the multiple different simulated stained fingerprint images one by one into the image feature processing model to obtain multiple feature images; and The multiple feature images are input into the adversarial network model one by one to obtain the fingerprint stain removal model.

2. The fingerprint stain removal model construction method according to claim 1, characterized in that: The feature image is an upsampled feature image, and the multiple simulated stained fingerprint images are input one by one into the feature processing model to obtain multiple feature images, specifically including: Downsampling the multiple different simulated stained fingerprint images to obtain multiple downsampled feature images corresponding to the multiple simulated stained fingerprint images; and The multiple down-sampled feature images are converted into the up-sampled feature image.

3. The method for constructing a fingerprint stain removal model according to claim 1, wherein: The adversarial network model is a generative adversarial network, which includes a generative model and a discriminative model. Inputting the multiple feature images one by one into the adversarial network model to obtain the fingerprint stain removal model specifically includes: Inputting the plurality of feature images one by one into the generation model to generate a corresponding plurality of expected fingerprint images; The discriminant model is used to compare the plurality of expected fingerprint images with the sample fingerprint image to obtain a comparison result; and When the comparison result meets the preset conditions, the current parameters of the convolutional neural network training model are determined to form the fingerprint stain removal model.

4. The method for constructing a fingerprint stain removal model according to claim 1, wherein: The simulated stained fingerprint image includes a first simulated stained fingerprint image, a second simulated stained fingerprint image, or a third simulated stained fingerprint image. The simulated stained fingerprint image is simulated by the simulated stained fingerprint algorithm to construct multiple different simulated stained fingerprint images. Specifically, the simulated stained fingerprint image includes: Using a morphological dilation algorithm to calculate the sample fingerprint image to obtain a plurality of the first simulated stained fingerprint images; or Calculating the sample fingerprint image using an elliptical mask algorithm to obtain a plurality of second simulated stained fingerprint images; or The sample fingerprint image is operated by using a random Perlin noise algorithm to obtain a plurality of third simulated stained fingerprint images.

5. The method for constructing a fingerprint stain removal model according to claim 1, wherein: The simulated stained fingerprint image includes a first simulated stained fingerprint image, a second simulated stained fingerprint image, and a third simulated stained fingerprint image. The simulated stained fingerprint image is simulated by the simulated stained fingerprint algorithm to construct multiple different simulated stained fingerprint images. Specifically, Using a morphological dilation algorithm to calculate the sample fingerprint image to obtain a plurality of the first simulated stained fingerprint images; Calculating the sample fingerprint image using an elliptical mask algorithm to obtain a plurality of second simulated stained fingerprint images; and The sample fingerprint image is operated by using a random Perlin noise algorithm to obtain a plurality of third simulated stained fingerprint images.

6. The method for constructing a fingerprint stain removal model according to claim 4 or 5, wherein: The sample fingerprint image is a square sample fingerprint image.

7. The fingerprint stain removal model construction method according to claim 1, characterized in that: The simulated stain algorithm is used to simulate the same sample fingerprint image to construct multiple different simulated stain fingerprint images, specifically including: Converting the sample fingerprint image into a square sample fingerprint image; and The simulated stain algorithm is used to perform simulation operations on the square sample fingerprint image to construct a plurality of different simulated stain fingerprint images.

8. A fingerprint recognition method, characterized in that: The fingerprint recognition method further includes: Obtaining the fingerprint image to be verified; Determining whether the fingerprint image to be verified has stains; If stains are present in the fingerprint image to be verified, inputting the fingerprint image to be verified into a fingerprint stain removal model for calculation to obtain a fingerprint image to be verified after stains are removed, wherein the fingerprint stain removal model is constructed by the fingerprint stain removal model construction method according to any one of claims 1 to 7; and Verifying the fingerprint image to be verified after stain removal with the sample fingerprint image to obtain a verification result; or If there is no stain on the fingerprint image to be verified, the fingerprint image to be verified is verified with the sample fingerprint image to obtain a verification result.

9. A fingerprint recognition sensor, characterized in that: The fingerprint recognition sensor includes: a storage medium for storing fingerprint recognition program instructions; and A processor, configured to execute the fingerprint recognition program instructions to implement the fingerprint recognition method according to claim 8.

10. An electronic device, characterized in that: The electronic device includes a main body and the fingerprint recognition sensor according to claim 9 arranged on the main body.

Images