Fingerprint recognition method and electronic device

Abstract

This application discloses a fingerprint recognition method and electronic device, relating to the field of fingerprint recognition. This method addresses the problem of low recognition rates caused by poor fingerprint image quality during fingerprint recognition. Specifically, the electronic device, in response to a user's fingerprint recognition operation, acquires a fingerprint image of the user through a fingerprint recognition module. The acquired fingerprint image is then quality-assessed and, based on the quality assessment results, repaired and recognized.

Description

Technical Field

[0001] The present application relates to the field of fingerprint recognition, and in particular to a fingerprint recognition method and electronic device. Background Art

[0002] To protect user privacy and data security, and to prevent accidental touches (misoperations), mobile phones typically have a lock screen feature. Users can lock their phones (e.g., by pressing the lock screen button) when not in use to prevent others from accessing their privacy and stealing their data. Users can unlock their phones before using them.

[0003] Generally, the unlocking operations of a mobile phone include password unlocking, fingerprint unlocking and face unlocking. Among them, fingerprint unlocking is widely used because of its high stability and security. In addition, as mobile phone screens develop towards full screens, the fingerprint unlocking of mobile phones currently generally adopts the under-screen fingerprint solution to avoid the fingerprint unlocking area occupying the screen display area and increase the screen-to-body ratio. The implementation methods of under-screen fingerprint usually include optical fingerprint recognition and ultrasonic fingerprint recognition. Among them, ultrasonic fingerprint recognition has stronger penetration ability and can be adapted to a variety of screens without screen burn-in. In addition, ultrasonic fingerprint recognition can have a larger recognition area, higher security and better user experience, and therefore has become the development trend of under-screen fingerprint.

[0004] However, the current imaging quality of ultrasonic fingerprint recognition is poor, resulting in a low recognition rate of ultrasonic fingerprint recognition. Summary of the Invention

[0005] The present application provides a fingerprint recognition method and electronic device, which solves the problem of low recognition rate caused by poor fingerprint image imaging quality during fingerprint recognition.

[0006] In order to achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, embodiments of the present application provide a fingerprint recognition method applicable to electronic devices. The method comprises: obtaining a first fingerprint image of a user; performing a quality assessment on the first fingerprint image; determining that the first fingerprint image is of a first quality level, wherein the first quality level is used to identify the quality of the first fingerprint image; determining a first repair model based on the first quality level, the first repair model including first repair parameters, the first repair model corresponding to the first quality level; processing the first fingerprint image based on the first repair model to obtain a second fingerprint image; and performing fingerprint recognition using the second fingerprint image.

[0008] The above technical solution can be used to perform a corresponding degree of repair based on the quality of the fingerprint image collected by the fingerprint recognition module before fingerprint recognition. This can improve the quality of the fingerprint image and the fingerprint recognition rate by repairing the fingerprint image, and can also avoid false fingerprint recognition caused by excessive fingerprint image repair.

[0009] In one possible implementation, the method further includes: obtaining a third fingerprint image of the user; performing a quality assessment on the third fingerprint image; determining that the third fingerprint image is at a second quality level, wherein the second quality level is used to identify the quality of the third fingerprint image; determining a second repair model based on the second quality level, the second repair model including a second repair parameter, and the second repair model corresponding to the second quality level; processing the third fingerprint image based on the second repair model to obtain a fourth fingerprint image; and performing fingerprint recognition using the fourth fingerprint image.

[0010] That is, when a user re-performs fingerprint recognition, if the quality level of the newly captured fingerprint image is different from the previous one, the phone will re-determine the corresponding fingerprint restoration parameters based on the quality level of the newly captured fingerprint image to restore the fingerprint image. In other words, after each fingerprint image is captured, the phone can perform a quality assessment on the fingerprint image and restore the fingerprint image based on the determined quality level using the restoration parameters. This prevents the phone from performing excessive restoration on a high-quality fingerprint image, which could lead to false positives in subsequent fingerprint recognitions.

[0011] In another possible implementation, the first quality level is greater than the second quality level, and the degree of change between the second fingerprint image and the first fingerprint image is less than the degree of change between the fourth fingerprint image and the third fingerprint image.

[0012] That is, the higher the quality level of the fingerprint image, the lower the degree of repair the mobile phone can perform. In this way, it can avoid the mobile phone performing a high degree of repair on a high-quality fingerprint image, which may lead to false recognition in subsequent fingerprint recognition.

[0013] In another possible implementation, a first repair model is determined based on a first quality level, including: determining first repair parameters according to the first quality level and a mapping relationship between preset quality levels and repair parameters; determining a repair model including the first repair parameters as the first repair model; and determining a second repair model based on a second quality level, including: determining second repair parameters according to the second quality level and a mapping relationship between preset quality levels and repair parameters; determining a repair model including the second repair parameters as the second repair model.

[0014] In this way, the mobile phone can determine the corresponding repair model or repair parameter according to the quality level according to the preset mapping relationship, which is easier to implement.

[0015] In another possible implementation, the first repair model and the second repair model are the same as or different from each other.

[0016] That is, if the repair models are the same, it means that the phone uses the same repair model to repair the fingerprint image, but the repair model has different repair parameters corresponding to different quality levels. If the repair models are different, it means that the phone uses different repair models for different quality levels to repair fingerprint images of corresponding quality levels.

[0017] In another possible implementation, the first repair model and the second repair model are different; determining the first repair model based on the first quality level includes: determining the first repair model according to the first quality level and a mapping relationship between a preset quality level and the repair model; determining the second repair model based on the second quality level includes: determining the second repair model according to the second quality level and a mapping relationship between a preset quality level and the repair model.

[0018] In this way, the mobile phone can determine the corresponding repair model or repair parameter according to the quality level according to the preset mapping relationship, which is easier to implement.

[0019] In another possible implementation, obtaining the first fingerprint image of the user includes: collecting a first original image of the user's fingerprint; and preprocessing the first original image to obtain the first fingerprint image.

[0020] In another possible implementation, preprocessing the first original image to obtain the first fingerprint image includes: preprocessing the first original image according to preset calibration data to obtain the first fingerprint image, where the calibration data includes noise data when the original image is collected.

[0021] By preprocessing the original image to obtain a fingerprint image, the noise in the fingerprint image can be reduced and the quality of the fingerprint image can be improved.

[0022] In another possible implementation, performing quality assessment on the first fingerprint image includes: fusing the first fingerprint image with the first original image; and performing quality assessment on the first fingerprint image according to a fusion result of the first fingerprint image and the first original image.

[0023] In this way, performing quality assessment on fingerprint images by fusing images can improve the accuracy of quality assessment on fingerprint images.

[0024] In another possible implementation, before fusing the first fingerprint image and the first original image, the method further includes: processing the first original image according to the calibration data; fusing the first fingerprint image and the first original image includes: fusing a result obtained by processing the first original image according to the calibration data with the first fingerprint image.

[0025] In this way, the original image can be denoised based on the calibration data before fusion, thereby further improving the quality of the fused data and improving the accuracy of the final quality assessment of the fingerprint image.

[0026] In a second aspect, embodiments of the present application provide a fingerprint recognition device that can be applied to an electronic device to implement the method described in the first aspect. The functions of the device can be implemented through hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, such as a processing module and a fingerprint recognition module.

[0027] Among them, the fingerprint recognition module can be used to obtain the user's first fingerprint image; the processing module can be used to perform quality assessment on the first fingerprint image; determine that the first fingerprint image is of a first quality level, wherein the first quality level is used to identify the quality of the first fingerprint image; determine a first repair model based on the first quality level, the first repair model includes a first repair parameter, and the first repair model corresponds to the first quality level; process the first fingerprint image based on the first repair model to obtain a second fingerprint image; the fingerprint recognition module can also be used to use the second fingerprint image for fingerprint recognition.

[0028] In one possible implementation, the fingerprint recognition module can also be used to obtain a third fingerprint image of the user; the processing module can also be used to perform quality assessment on the third fingerprint image; determine that the third fingerprint image is at a second quality level, where the second quality level is used to identify the quality of the third fingerprint image; determine a second repair model based on the second quality level, the second repair model includes second repair parameters, and the second repair model corresponds to the second quality level; process the third fingerprint image based on the second repair model to obtain a fourth fingerprint image; the fingerprint recognition module can also be used to perform fingerprint recognition using the fourth fingerprint image.

[0029] In another possible implementation, the first quality level is greater than the second quality level, and the degree of change between the second fingerprint image and the first fingerprint image is less than the degree of change between the fourth fingerprint image and the third fingerprint image.

[0030] In another possible implementation, the processing module is specifically used to determine the first repair parameter according to the first quality level and the mapping relationship between the preset quality level and the repair parameter; determine the repair model including the first repair parameter as the first repair model; the processing module is specifically used to determine the second repair parameter according to the second quality level and the mapping relationship between the preset quality level and the repair parameter; determine the repair model including the second repair parameter as the second repair model.

[0031] In another possible implementation, the first repair model and the second repair model are the same as or different from each other.

[0032] In another possible implementation, the first repair model and the second repair model are different; the processing module is specifically used to determine the first repair model according to the first quality level and the mapping relationship between the preset quality level and the repair model; the processing module is specifically used to determine the second repair model according to the second quality level and the mapping relationship between the preset quality level and the repair model.

[0033] In another possible implementation, the fingerprint recognition module is specifically configured to collect a first original image of the user's fingerprint; and pre-process the first original image to obtain a first fingerprint image.

[0034] In another possible implementation, the fingerprint recognition module is specifically configured to preprocess the first original image according to preset calibration data to obtain the first fingerprint image, where the calibration data includes noise data when the original image is collected.

[0035] In another possible implementation, the processing module is specifically configured to fuse the first fingerprint image with the first original image; and perform quality assessment on the first fingerprint image according to a fusion result of the first fingerprint image and the first original image.

[0036] In another possible implementation, the processing module is further configured to process the first original image according to the calibration data; the processing module is specifically configured to fuse a result obtained by processing the first original image according to the calibration data with the first fingerprint image.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor. When the processor is configured to execute the instructions, the electronic device implements the fingerprint recognition method as described in the first aspect or any possible implementation of the first aspect.

[0038] In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by an electronic device, the electronic device implements the fingerprint recognition method as described in the first aspect or any possible implementation of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer-readable code. When the computer-readable code is executed in an electronic device, the electronic device implements the fingerprint recognition method as described in the first aspect or any possible implementation manner of the first aspect.

[0040] It should be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0041] In a sixth aspect, embodiments of the present application provide a fingerprint recognition method that can be applied to electronic devices. The method includes: obtaining a first fingerprint image of a user; performing a quality assessment on the first fingerprint image; determining that the first fingerprint image is at a first quality level, wherein the first quality level is used to identify the quality of the first fingerprint image; determining whether the first quality level is a preset quality level; if so, determining a first repair model based on the first quality level, the first repair model including a first repair parameter, the first repair model corresponding to the first quality level; processing the first fingerprint image based on the first repair model to obtain a second fingerprint image; and performing fingerprint recognition using the second fingerprint image.

[0042] The above technical solution can be used to perform a corresponding degree of repair based on the quality of the fingerprint image collected by the fingerprint recognition module before fingerprint recognition. This can improve the quality of the fingerprint image and the fingerprint recognition rate by repairing the fingerprint image, and can also avoid false fingerprint recognition caused by excessive fingerprint image repair.

[0043] In a possible implementation, after determining whether the first quality level is a preset quality level, the method further includes: if not, performing fingerprint recognition on the first fingerprint image.

[0044] In this way, when the fingerprint image does not meet the preset quality level, fingerprint recognition can be performed directly on the fingerprint image, so that only the image that needs fingerprint repair can be repaired, thereby improving fingerprint recognition efficiency.

[0045] In another possible implementation, the preset quality level includes one or more quality levels.

[0046] In another possible implementation, obtaining the first fingerprint image of the user includes: collecting a first original image of the user's fingerprint; and preprocessing the first original image to obtain the first fingerprint image.

[0047] By preprocessing the original image to obtain a fingerprint image, the noise in the fingerprint image can be reduced and the quality of the fingerprint image can be improved.

[0048] In a seventh aspect, embodiments of the present application provide a fingerprint recognition device that can be applied to an electronic device to implement the method described in the sixth aspect. The functions of the device can be implemented through hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, such as a processing module and a fingerprint recognition module.

[0049] Among them, the fingerprint recognition module can be used to obtain the user's first fingerprint image; the processing module can be used to perform quality assessment on the first fingerprint image; determine that the first fingerprint image is of a first quality level, wherein the first quality level is used to identify the quality of the first fingerprint image; judge whether the first quality level is a preset quality level; if so, determine a first repair model based on the first quality level, the first repair model includes a first repair parameter, and the first repair model corresponds to the first quality level; process the first fingerprint image based on the first repair model to obtain a second fingerprint image; the fingerprint recognition module can also be used to use the second fingerprint image for fingerprint recognition.

[0050] In a possible implementation, the fingerprint recognition module may be further configured to, if not, perform fingerprint recognition on the first fingerprint image.

[0051] In another possible implementation, the preset quality level includes one or more quality levels.

[0052] In another possible implementation, the fingerprint recognition module is specifically configured to collect a first original image of the user's fingerprint; and pre-process the first original image to obtain a first fingerprint image.

[0053] In an eighth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor. When the processor is configured to execute the instructions, the electronic device implements the fingerprint recognition method as described in the sixth aspect or any possible implementation of the sixth aspect.

[0054] In a ninth aspect, embodiments of the present application provide a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by an electronic device, the electronic device implements the fingerprint recognition method as described in the sixth aspect or any possible implementation of the sixth aspect.

[0055] In a tenth aspect, an embodiment of the present application provides a computer program product, comprising a computer-readable code. When the computer-readable code runs in an electronic device, the electronic device implements the fingerprint recognition method as described in the sixth aspect or any possible implementation of the sixth aspect.

[0056] It should be understood that the beneficial effects of the seventh to tenth aspects mentioned above can be found in the relevant description of the sixth aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the principle of an ultrasonic fingerprint recognition module provided in an embodiment of the present application;

[0058] Figure 2A schematic diagram of an application scenario of a fingerprint recognition method provided in an embodiment of the present application;

[0059] Figure 3a A schematic diagram of a low-quality fingerprint image provided in an embodiment of the present application;

[0060] Figure 3b A schematic diagram of a medium-quality fingerprint image provided in an embodiment of the present application;

[0061] Figure 3c A schematic diagram of a high-quality fingerprint image provided in an embodiment of the present application;

[0062] Figure 3d A schematic diagram of a high-quality fingerprint image after over-repair provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0064] Figure 5 A schematic diagram of a fingerprint recognition method according to an embodiment of the present invention;

[0065] Figure 6 A schematic diagram of a fingerprint quality assessment process provided in an embodiment of the present application;

[0066] Figure 7 A schematic diagram of another fingerprint quality assessment process provided in an embodiment of the present application;

[0067] Figure 8 A schematic diagram of a fingerprint repair process provided in an embodiment of the present application;

[0068] Figure 9 A schematic diagram of another fingerprint recognition method provided in an embodiment of the present application;

[0069] Figure 10 A schematic diagram of another fingerprint recognition method provided in an embodiment of the present application;

[0070] Figure 11 A flowchart of another fingerprint recognition method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] To protect user privacy and data security, and to prevent accidental operation when not in use, mobile phones generally have a lock screen feature. This allows users to lock their phones (e.g., by pressing the lock screen button) when not in use, preventing others from accessing their phones, snooping on their privacy, or stealing their data. When the user needs to use their phone again, they can unlock it before using it.

[0072] Typically, mobile phone unlocking methods include password unlocking, fingerprint unlocking, and facial unlocking. Fingerprint unlocking is widely used due to its high stability and security. As mobile phone screens move towards full-screen displays, fingerprint unlocking currently generally uses an under-screen fingerprint solution.

[0073] The implementation methods of under-screen fingerprint recognition usually include optical fingerprint recognition and ultrasonic fingerprint recognition. Among them, optical fingerprint recognition uses light reflection to obtain fingerprint images. That is, the finger area is illuminated by light emitted by a light source (such as the OLED in the organic light-emitting diode (OLED) screen as a light source), and the illuminated finger area can reflect light back to the sensor, so that the sensor can obtain the fingerprint image based on the reflected light. Ultrasonic fingerprint recognition uses ultrasonic reflection to obtain fingerprint images. For example, Figure 1 As shown, the ultrasonic fingerprint recognition module includes at least a transducer 101. The ultrasonic waves emitted by the transducer 101 can scan the finger area through the screen 102, and the scanned finger area can reflect the ultrasonic waves back to the transducer 101. The reflected ultrasonic waves can make the transducer 101 vibrate and convert them into electrical signals, so that the fingerprint image can be obtained according to the converted electrical signals. Since ultrasonic waves have stronger penetrating power than light, ultrasonic fingerprint recognition has stronger penetrating power and can be adapted to a variety of screens, and the fingerprint image collected by ultrasonic fingerprint recognition is a three-dimensional image, so ultrasonic fingerprint recognition has higher security, so ultrasonic fingerprint recognition has become the development trend of under-screen fingerprints. However, the current imaging quality of ultrasonic fingerprint recognition is poor, resulting in a low recognition rate of ultrasonic fingerprint recognition.

[0074] To solve the above problems, an embodiment of the present application provides a fingerprint recognition method. This method can be applied to scenarios where a user unlocks an electronic device that uses fingerprint recognition (i.e., an electronic device that includes a fingerprint recognition module) with a fingerprint. For example, the electronic device is a mobile phone, and the mobile phone is provided with an under-screen fingerprint recognition. Figure 2 As shown, when the mobile phone is in the lock screen, the area on the mobile phone screen corresponding to the fingerprint recognition module can display an indication screen 201 for indicating the fingerprint unlocking area. The user can use the finger used for fingerprint unlocking to touch (or press) the position corresponding to the indication screen 201. In response to the user's touch operation at the corresponding position of the indication screen 201, the mobile phone can obtain the fingerprint image of the user's corresponding finger through the fingerprint recognition module, and use the fingerprint recognition method provided in this embodiment to perform fingerprint recognition based on the obtained fingerprint image, thereby facilitating the unlocking of the mobile phone when the fingerprint recognition is successful.

[0075] The fingerprint recognition method may include: the electronic device, in response to a user's fingerprint recognition operation, obtaining a user's fingerprint image through a fingerprint recognition module, performing a quality assessment on the obtained fingerprint image, and repairing and recognizing the fingerprint image based on the quality assessment result.

[0076] The term "fingerprint image" generally refers to a pre-processed image of the fingerprint (i.e., an image used for fingerprint recognition) obtained by pre-processing the image information after acquiring the fingerprint image information. Typically, the fingerprint image information collected by the fingerprint recognition module includes the original fingerprint image (also known as the Raw image or Raw data, i.e., the fingerprint image originally collected by the fingerprint recognition module). The fingerprint recognition module also typically includes preset calibration data (or Base data, i.e., calibration data generated during production calibration of the fingerprint recognition module, which primarily includes noise data present during fingerprint recognition). Therefore, the pre-processed fingerprint image can be obtained based on the original image and calibration data. For example, the calibration data can be removed (i.e., denoising, such as subtracting the value of each pixel in the original image from the value of the corresponding pixel in the calibration data) from the original fingerprint image collected by the fingerprint recognition module to obtain a pre-processed image (i.e., a fingerprint image). Of course, in some possible implementations, after the original fingerprint image is denoised based on the calibration data, pre-processing operations such as Gaussian filtering and moiré removal can also be performed to obtain the pre-processed image (i.e., a fingerprint image).

[0077] Optionally, the quality assessment is to evaluate the quality of the fingerprint image and determine the quality level of the fingerprint image. The quality assessment of the fingerprint image can be based on whether the texture is continuous, whether the texture has discontinuities, whether the image is clear, whether the image has noise or white noise or Gaussian noise, and / or whether the image has foreign matter, etc. One or more items can be selected for evaluation, and this application does not limit this. When performing quality assessment on the acquired fingerprint image, the fingerprint image can be divided into different quality levels according to its quality. The quality of the fingerprint image can refer to the degree of similarity between the fingerprint texture in the fingerprint image and the fingerprint texture of the user. That is, the higher the similarity between the fingerprint texture in the fingerprint image and the fingerprint texture of the user, the higher the quality of the fingerprint image.

[0078] Optionally, a quality level can be used to identify the quality of a fingerprint image. Fingerprint image restoration and identification can be performed based on the quality assessment result. The fingerprint image can be restored to a corresponding degree based on the quality level corresponding to the quality assessment result. That is, the higher the quality level, the lower the corresponding restoration degree (the restoration degree is generally the degree of change in the fingerprint image before and after restoration). (In other words, the higher the quality level, the lower the restoration degree for the fingerprint image). For example, fingerprint images can be classified into three quality levels: low, medium, and high. When the quality assessment result is low, the fingerprint image is restored to a higher restoration degree (e.g., a high restoration degree) corresponding to a low-quality image. When the quality assessment result is medium, the fingerprint image is restored to a medium restoration degree (e.g., a medium restoration degree) corresponding to a medium-quality image. When the quality assessment result is high, the fingerprint image is restored to a lower restoration degree (e.g., a low restoration degree) corresponding to a high-quality image. Of course, the quality of fingerprint images can also be classified into other different quality levels, such as low and high, and the restoration degree corresponding to each quality level will be adjusted accordingly. This is not a limitation here. After the fingerprint image has been repaired accordingly, fingerprint recognition can be performed on the repaired fingerprint image.

[0079] For example, fingerprint image restoration and recognition can be performed based on the quality assessment result. Alternatively, when the quality level corresponding to the quality assessment result is consistent with the repairable quality level (or when the quality level corresponding to the quality assessment result is a preset quality level), the fingerprint image is restored to a corresponding degree. For example, fingerprint images can be divided into two quality levels, low quality and high quality, and the repairable quality level (or the preset quality level) is set to high quality. Then, when the quality assessment result is low quality, the fingerprint image is not restored and fingerprint recognition is performed directly based on the fingerprint image. When the quality assessment result is high quality, the fingerprint image is restored to the degree of restoration corresponding to the high quality image (e.g., low restoration degree), and fingerprint recognition is then performed based on the restored fingerprint image. Of course, the quality of the fingerprint image can also be divided into other different quality levels, such as low quality, medium quality, and high quality. Accordingly, the preset quality level can be set to one or two (i.e., the upper limit of the number of preset quality levels that can be set is N-1, where N is the total number of quality levels). When the assessed quality level of the fingerprint image is consistent with the preset quality level, the fingerprint image is repaired to a corresponding degree. If the assessed quality level of the fingerprint image is inconsistent with the preset quality level (i.e., it is not the preset quality level), the fingerprint image is not repaired and fingerprint recognition is performed directly. This can avoid over-repairing caused by repairing fingerprint images whose quality is inconsistent with the preset quality, thereby eliminating the risk of false fingerprint recognition during fingerprint recognition due to over-repairing of the fingerprint image.

[0080] By adopting this fingerprint recognition method, it is possible to perform corresponding repairs according to the quality of the acquired fingerprint image, and then perform fingerprint recognition. Therefore, the quality of the fingerprint image can be improved by repairing the fingerprint image, thereby improving the fingerprint recognition rate, and it is possible to avoid false recognition of fingerprints due to excessive repair of the fingerprint image. For example, during ultrasonic fingerprint recognition, the quality of the acquired fingerprint image can be divided into three quality levels: low quality (also referred to as the third quality level in this application), medium quality (also referred to as the second quality level in this application), and high quality (also referred to as the first quality level in this application); optionally, the fingerprint image can be collected by an ultrasonic recognition module. Among them, if Figure 3a As shown in , low-quality fingerprint images are images with unclear fingerprint textures, where the fingerprint textures are almost indistinguishable. The proportion of low-quality fingerprint images is usually around 10%. Figure 3b As shown in the figure, a medium-quality fingerprint image is one with clear fingerprint texture but many obvious discontinuities in the fingerprint texture (usually called a dry fingerprint image). The proportion of medium-quality fingerprint images obtained in ultrasonic fingerprint recognition is usually about 40%. Figure 3c As shown, a high-quality fingerprint image has clear fingerprint texture and a small number of slight discontinuities (e.g. Figure 3c The image shown in the area 301 in the figure is taken as an example, and the proportion of high-quality fingerprint images obtained is generally about 50%. Therefore, by adopting this method, the quality of the acquired fingerprint image can be evaluated, and the evaluated fingerprint images of different qualities can be repaired to corresponding degrees (e.g., the low-quality fingerprint image is repaired to a high degree (such as it can also be called the third repair degree in this application), the medium-quality fingerprint image is repaired to a medium degree (such as it can also be called the second repair degree in this application), and the high-quality fingerprint image is repaired to a low degree (such as it can also be called the first repair degree in this application)) to improve the overall quality of the acquired fingerprint image, thereby improving the recognition rate of ultrasonic fingerprint recognition. Among them, by performing corresponding different degrees of repair on the three different qualities of fingerprint images, the risk of false recognition caused by excessive repair of the acquired fingerprint image can be avoided. For example, since the quality of the high-quality fingerprint image is still poorer than that of the fingerprint images collected by the optical fingerprint recognition module and the capacitive fingerprint recognition module, the high-quality fingerprint image needs to be further repaired. If the same degree of repair as the medium-quality fingerprint image is used when repairing the high-quality fingerprint image, the high-quality fingerprint image will be over-repaired, resulting in the following: Figure 3d The high-quality fingerprint image after restoration is compared with the Figure 3cThe high-quality fingerprint image shown (i.e., the high-quality fingerprint image before restoration) in which the fingerprint textures that were originally not connected are connected together due to over-restoration (e.g., Figure 3c The area shown in 302 in FIG. 1 originally had unconnected parts, but Figure 3d The area shown in 303 in the figure connects previously unconnected parts due to over-repairing, causing distortion in the repaired high-quality fingerprint image and thus increasing the risk of false recognition during fingerprint recognition. However, the above fingerprint recognition method allows high-quality fingerprint images to be repaired with a lower degree of repair than that corresponding to medium-quality fingerprint images, thereby avoiding the risk of false recognition caused by over-repairing high-quality fingerprint images.

[0081] The fingerprint recognition method provided in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0082] In the embodiments of the present application, the electronic device with a camera function may be a mobile phone, a tablet computer, a handheld computer, a PC, a cellular phone, a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet), a smart home device (such as a television), a car computer (such as an onboard computer), a smart screen, a game console, and an augmented reality (AR) / virtual reality (VR) device. The embodiments of the present application do not impose any special restrictions on the specific device form of the electronic device.

[0083] For example, taking the electronic device as a mobile phone, Figure 4 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown. That is, for example, Figure 4 The electronic device shown may be a mobile phone.

[0084] like Figure 4As shown, the electronic device may include a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, an earphone interface 470D, a sensor module 480, a button 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc. Among them, the sensor module 480 can include a pressure sensor 480A, a gyroscope sensor 480B, an air pressure sensor 480C, a magnetic sensor 480D, an acceleration sensor 480E, a distance sensor 480F, a proximity light sensor 480G, a fingerprint sensor 480H (such as an ultrasonic fingerprint recognition module, an optical fingerprint recognition module, and a capacitive fingerprint recognition module, etc.), a temperature sensor 480J, a touch sensor 480K, an ambient light sensor 480L, a bone conduction sensor 480M, etc.

[0085] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0086] The processor 410 may include one or more processing units. For example, the processor 410 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0087] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0088] Processor 410 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 410 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 410. If processor 410 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 410 latency, and thus improves system efficiency.

[0089] In some embodiments, the processor 410 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0090] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 450, wireless communication module 460, modem processor and baseband processor.

[0091] The electronic device implements display functionality through a GPU, display screen 494, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 410 may include one or more GPUs that execute program instructions to generate or modify display information.

[0092] Display screen 494 is used to display images, videos, and the like. Display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLED, or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device can include one or N display screens 494, where N is a positive integer greater than one.

[0093] The electronic device can implement a shooting function through an ISP, a camera 493, a video codec, a GPU, a display 494, and an application processor. In some embodiments, the electronic device may include one or N cameras 493, where N is a positive integer greater than 1. For example, the electronic device may include three cameras, one of which is a main camera, one is a telephoto camera, and one is an ultra-wide-angle camera.

[0094] The internal memory 421 can be used to store computer executable program codes, which include instructions. The processor 410 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 421. The internal memory 421 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0095] Of course, it is understandable that the above Figure 4The following is only an example of an electronic device in the form of a mobile phone. If the electronic device is a tablet computer, handheld computer, PC, PDA, wearable device (such as smart watch, smart bracelet), smart home device (such as TV), car computer (such as car computer), smart screen, game console and AR / VR device, the structure of the electronic device may include more than Figure 4 The structure shown in the figure is less than Figure 4 More structures are shown in the figure, which are not limited here.

[0096] The methods in the following embodiments can all be implemented in an electronic device having the above hardware structure.

[0097] Taking the electronic device as a mobile phone, the fingerprint image obtained by the fingerprint recognition module (or fingerprint sensor) of the electronic device can be divided into the following categories: Figure 3a The low quality shown, such as Figure 3b The medium quality shown and Figure 3c The three quality levels shown are as an example. Figure 5 A fingerprint recognition method provided by an embodiment of the present application is shown. Figure 5 As shown, the method may include the following S501-S504.

[0098] S501. The mobile phone obtains a fingerprint image (eg, the first fingerprint image and the third fingerprint image in this application) through a fingerprint recognition module.

[0099] For example, the fingerprint recognition module can be an ultrasonic fingerprint recognition module, an optical fingerprint recognition module, a capacitive fingerprint recognition module, etc., among which the optical fingerprint recognition module and the ultrasonic fingerprint recognition module can also be combined with the under-screen fingerprint technology and set under the screen (i.e., display screen) of the mobile phone.

[0100] Among them, the fingerprint image is an image used for fingerprint recognition. For example, the information collected by the fingerprint recognition module usually includes the original image of the fingerprint (such as the first original image in this application, etc., the original image can also be called Raw image, Raw data, that is, the image of the fingerprint originally collected by the fingerprint recognition module). The fingerprint recognition module (or mobile phone) is generally also preset with calibration data (or called Base data, that is, the calibration data generated when the fingerprint recognition module is produced and calibrated, which mainly includes the noise data present when the fingerprint recognition collects the original image). Therefore, the fingerprint image collected by the fingerprint recognition module can be obtained based on the original image (that is, Raw image) and the calibration data (that is, Base data), that is, the fingerprint image can be a pre-processed image obtained by pre-processing the original image based on the calibration data. For example, based on the original image of the fingerprint collected by the fingerprint recognition module, the calibration data can be removed (that is, denoising, such as subtracting the value of each pixel in the original image from the value corresponding to the corresponding pixel in the calibration data) to obtain a pre-processed image, and the pre-processed image is used as the fingerprint image). Of course, in some possible implementations, after the original fingerprint image is denoised according to the calibration data, preprocessing operations such as Gaussian filtering and moiré removal may be performed to obtain a preprocessed image (i.e., fingerprint image), which is not limited here.

[0101] S502: The mobile phone performs quality assessment on the fingerprint image.

[0102] As an example, a mobile phone can use a fingerprint quality assessment model to perform quality assessment on fingerprint images. The fingerprint quality assessment model can be based on the ResNet network structure and trained using regression techniques. That is, the fingerprint quality assessment model can be used to output a quality assessment result (such as a quality score, quality level, etc.) corresponding to the fingerprint image when inputting data including the fingerprint image. In the embodiments of the present application, the model can be trained based on the specific data settings for the model input and output.

[0103] For example, the fingerprint quality assessment model can be trained using the fused fingerprint data as model input and the quality score corresponding to the input fused fingerprint data as model output.

[0104] For example, the fused fingerprint data can be the data obtained by multimodal data fusion (for example, multimodal data fusion is performed using a stage-based fusion algorithm, a feature-based fusion algorithm, and a semantic-based fusion algorithm) of the original image (i.e., Raw image) of the fingerprint collected by the fingerprint recognition module and the preprocessed image (i.e., fingerprint image) obtained by preprocessing the original image. The quality score corresponding to the fused fingerprint data can be the quality score of the preprocessed image in the fused fingerprint data. The data obtained by fusion of the above-mentioned original image and preprocessed image can be used as model input, and the quality score corresponding to the preprocessed image can be used as model output to train the fingerprint quality assessment model. It can be understood that multimodal data fusion can adopt fusion algorithms in the prior art, such as CONCAT and interpolation fusion, which are not limited in this application.

[0105] like Figure 6As shown, when using a fingerprint quality assessment model trained based on the above example, the mobile phone can perform a quality assessment on a fingerprint image as follows: when the fingerprint recognition module captures an original fingerprint image (e.g., a first original image) and generates a preprocessed image (i.e., a fingerprint image, such as the first fingerprint image) based on the original image, the mobile phone can fuse the original image and the preprocessed image to obtain fused data (i.e., fused fingerprint data). The mobile phone can then input the fused data into the fingerprint quality assessment model, causing the fingerprint quality assessment model to output a corresponding quality assessment result (i.e., a quality score). Different quality levels of fingerprint images can correspond to different quality score ranges. For example, the quality score can be set to 0-100, with the range corresponding to a low quality level being [0, 20], the range corresponding to a medium quality level being (20, 40), and the range corresponding to a high quality level being (40, 100). That is, when the quality score output by the fingerprint quality assessment model is in the range [0, 20], the quality assessment result of the fingerprint image included in the input fused data can be determined to be low quality. When the quality score value output by the fingerprint quality assessment model is between (20, 40), the quality assessment result of the fingerprint image included in the input fused data can be determined to be of medium quality. When the quality score value output by the fingerprint quality assessment model is between (40, 100], the quality assessment result of the fingerprint image included in the input fused data can be determined to be of high quality. Of course, when the quality score value output by the fingerprint quality assessment model is 20, the quality assessment result of the fingerprint image included in the input fused data can be determined to be of low quality or medium quality, which can be set according to actual needs and is not limited here. When the quality score value output by the fingerprint quality assessment model is 40, the quality assessment result of the fingerprint image included in the input fused data can be determined to be of medium quality or high quality, which can be set according to actual needs and is not limited here. In this way, the quality assessment of the fingerprint image is performed based on the fused data. Since the fused data can better represent the characteristics of the fingerprint image, the accuracy of the quality assessment of the fingerprint image can be improved.

[0106] For another example, the fused fingerprint data can also be data obtained by fusing the original image (i.e., the raw image, such as the first original image) of the fingerprint collected by the fingerprint recognition module, a preprocessed image (i.e., the fingerprint image, such as the first fingerprint image) obtained by preprocessing the original image, and calibration data (i.e., base data) preset by the fingerprint recognition module. For example, the original image can be normalized after removing the calibration data (i.e., the pixel value of each pixel in the original image minus the calibration data value of the corresponding pixel) (because data calculations are performed, normalization can flatten all values ​​to the same range to eliminate the impact of outliers) to obtain processed data. The processed data is then multimodally fused with the preprocessed image to obtain fused fingerprint data. In this way, the fused fingerprint data can be denoised, thereby improving the training effect of the fingerprint quality assessment model and making its quality assessment more accurate. The quality score corresponding to the fused fingerprint data can be the quality score of the preprocessed image in the fused fingerprint data. In particular, the normalization of the intermediate image after removing the calibration data from the original image can be maximum-minimum normalization. That is, the pixel value of each pixel in the normalized processed data satisfies the following formula:

[0107]

[0108] Where I is the normalized pixel value, i is the pixel value of the corresponding pixel before normalization, min is the minimum pixel value of each pixel before normalization, and max is the maximum pixel value of each pixel before normalization.

[0109] The data obtained by fusing the above-mentioned original image, pre-processed image and calibration data can be used as model input, and the quality score corresponding to the pre-processed image can be used as model output to train the fingerprint quality assessment model.

[0110] like Figure 7As shown, when using the fingerprint quality assessment model trained based on the above example, the mobile phone can perform quality assessment on the fingerprint image as follows: when the fingerprint recognition module collects the original image of the fingerprint and obtains a preprocessed image (i.e., fingerprint image) based on the original image, the mobile phone can remove the calibration data from the original image (e.g., the pixel value of each pixel in the original image minus the calibration data value of the corresponding pixel) and perform normalization processing to obtain processed data, and then fuse the processed data with the preprocessed image to obtain fused data (i.e., fused fingerprint data). The mobile phone can then input the fused data into the fingerprint quality assessment model, so that the fingerprint quality assessment model outputs the corresponding quality assessment result (i.e., quality score value). Different quality levels of the fingerprint image can correspond to different quality score value ranges. For example, the quality score can be set to 0-100, the interval corresponding to the low quality level can be set to [0, 20), the interval corresponding to the medium quality level can be set to (20, 40), and the interval corresponding to the high quality level can be set to (40, 100). That is, when the quality score output by the fingerprint quality assessment model is in the range of [0, 20), it can be determined that the quality assessment result of the fingerprint image included in the input fusion data is of low quality. When the quality score output by the fingerprint quality assessment model is in the range of (20, 40), it can be determined that the quality assessment result of the fingerprint image included in the input fusion data is of medium quality. When the quality score value output by the fingerprint quality assessment model is between (40, 100], the quality assessment result of the fingerprint image included in the input fusion data can be determined to be high quality. Of course, when the quality score value output by the fingerprint quality assessment model is 20, the quality assessment result of the fingerprint image included in the input fusion data can be determined to be low quality or medium quality, which can be set according to actual needs and is not limited here. When the quality score value output by the fingerprint quality assessment model is 40, the quality assessment result of the fingerprint image included in the input fusion data can be determined to be medium quality or high quality, which can be set according to actual needs and is not limited here.

[0111] It should be noted that in the embodiment of the present application, the model can also be trained using only the fingerprint image as the model input and the quality score corresponding to the fingerprint image as the model output, so as to directly determine the quality of the fingerprint image based on the fingerprint image. For example, the mobile phone can perform quality assessment on the fingerprint image as follows: when the fingerprint recognition module captures the original image of the fingerprint (such as the first original image) and obtains a pre-processed image (i.e., the fingerprint image, such as the first fingerprint image) based on the original image, the mobile phone can input the pre-processed image into the fingerprint quality assessment model, so that the fingerprint quality assessment model outputs the corresponding quality assessment result (i.e., the quality score), and the corresponding quality level can be determined according to the interval of the quality score.

[0112] Alternatively, the model can be trained using the original image captured by the fingerprint recognition module as the model input and the quality score corresponding to the original image as the model output, thereby directly determining the quality of the corresponding fingerprint image based on the original image. For example, a mobile phone can perform quality assessment on a fingerprint image by: when the fingerprint recognition model captures the original image of the fingerprint, the mobile phone can input the original image into the fingerprint quality assessment model, so that the fingerprint quality assessment model outputs the corresponding quality assessment result (i.e., the quality score), and then the corresponding quality level can be determined based on the quality score range.

[0113] S503: The mobile phone repairs the fingerprint image to a corresponding degree according to the quality assessment result of the fingerprint image.

[0114] The mobile phone can perform a lower degree of repair on fingerprint images with higher quality (such as quality level) based on the quality evaluation result of the fingerprint image, thereby avoiding excessive repair of high-quality fingerprint images.

[0115] For example, based on the quality assessment result obtained in S502, the mobile phone can perform a corresponding degree of repair on the fingerprint image according to the quality level of the fingerprint image. That is, the mobile phone can use a corresponding degree of repair to repair the fingerprint image based on a preset mapping relationship between the quality level and the degree of repair (or repair parameters, repair model).

[0116] For example, if the mobile phone determines that the fingerprint image quality level is low, it can perform a high degree of restoration on the fingerprint image. If the mobile phone determines that the fingerprint image quality level is medium, it can perform a medium degree of restoration on the fingerprint image. If the mobile phone determines that the fingerprint image quality level is high, it can perform a low degree of restoration on the fingerprint image. In other words, the higher the quality of the fingerprint image, the lower the degree of restoration is performed on the fingerprint image, thereby avoiding distortion of the fingerprint image caused by excessive restoration.

[0117] As an example, a mobile phone can use a fingerprint restoration model to restore a fingerprint image. The fingerprint restoration model can be trained based on a Unet network structure (e.g., a two-level Unet network structure). That is, the fingerprint restoration model can be used to input a fingerprint image and output a restored image of the fingerprint image.

[0118] For example, the fingerprint restoration model can use the same model structure and fingerprint restoration parameters for fingerprint images of different quality levels to achieve different degrees of fingerprint restoration for fingerprint images of different quality levels. When training the model, fingerprint restoration parameters for fingerprint images of different quality levels can be obtained by using fingerprint images of different degradation levels as training data.

[0119] For example, a degradation model can be used to perform low-level degradation on a fingerprint image with a high recognition rate (such as a fingerprint image with clear fingerprint texture and no discontinuity), thereby obtaining a fingerprint image with a high quality (such as a fingerprint image with a high quality) Figure 3c A training image with similar quality to the fingerprint image shown in FIG. This training image can be used as model input, and the corresponding pre-degraded image (i.e., a fingerprint image with a high recognition rate before degradation corresponding to the training image) can be used as model output to train the fingerprint restoration model. This can yield fingerprint restoration parameters for high-quality fingerprint images (e.g., high-quality fingerprint restoration parameters that enable a low degree of restoration of the fingerprint image).

[0120] For example, a degradation model can be used to perform a moderate degree of degradation on a fingerprint image with a high recognition rate (such as a fingerprint image with clear fingerprint texture and no discontinuities), thereby obtaining a fingerprint image with a similar quality to the above-mentioned medium quality (such as Figure 3b A training image with similar quality to the fingerprint image shown in FIG. This training image can be used as the model input, and the corresponding pre-degraded image (i.e., the fingerprint image with a high recognition rate before degradation corresponding to the training image) can be used as the model output to train the fingerprint restoration model. This can yield fingerprint restoration parameters for medium-quality fingerprint images (e.g., medium-quality fingerprint restoration parameters, which enable a medium degree of restoration of the fingerprint image).

[0121] For another example, a degradation model can be used to perform a high degree of degradation on a fingerprint image with a low recognition rate (such as a fingerprint image with a fuzzy fingerprint texture), thereby obtaining a fingerprint image with a low quality (such as a fingerprint image with a fuzzy fingerprint texture). Figure 3a A training image with similar quality to the fingerprint image shown in FIG. This training image can be used as model input, and the corresponding pre-degraded image (i.e., the fingerprint image with a low recognition rate before degradation corresponding to the training image) can be used as model output to train the fingerprint restoration model. This can yield fingerprint restoration parameters for low-quality fingerprint images (e.g., low-quality fingerprint restoration parameters that enable a high degree of restoration of the fingerprint image).

[0122] like Figure 8As shown, a fingerprint restoration model trained based on the above example is used, having the same model structure, with different restoration parameters for fingerprint images of different quality levels (e.g., low-quality fingerprint restoration parameters for low-quality fingerprint images, medium-quality fingerprint restoration parameters for medium-quality fingerprint images, and high-quality fingerprint restoration parameters for high-quality fingerprint images). When using this fingerprint restoration model, the mobile phone can perform corresponding restoration of the fingerprint image based on the quality assessment result of the fingerprint image by using the corresponding restoration parameters according to the mapping relationship between the preset quality level and the restoration parameters. For example, when the mobile phone determines that the fingerprint image (e.g., the fifth fingerprint image) is of low quality, the low-quality fingerprint image is input into the fingerprint restoration model and restored with the low-quality fingerprint restoration parameters to obtain a highly restored image (e.g., the sixth fingerprint image). When the mobile phone determines that the fingerprint image (e.g., the third fingerprint image) is of medium quality, the medium-quality fingerprint image is input into the fingerprint restoration model and restored with the medium-quality fingerprint restoration parameters to obtain a medium-restored image (e.g., the fourth fingerprint image). When the mobile phone determines that the fingerprint image (such as the first fingerprint image) is a high-quality fingerprint image, the high-quality fingerprint image is input into the fingerprint repair model and the fingerprint image is repaired with high-quality fingerprint repair parameters to obtain a low-level repaired image (such as the second fingerprint image).

[0123] Optionally, in an embodiment of the present application, based on the above-mentioned method for training a fingerprint restoration model, fingerprint restoration models (such as the first restoration model, the second restoration model, the third restoration model, etc. in the present application) for fingerprint images of different quality levels can be trained separately and independently (i.e., having independent model structures), which is not limited here. Accordingly, when using the fingerprint restoration model, the mobile phone can perform corresponding degree of restoration on the fingerprint image according to the quality assessment result of the fingerprint image: according to the mapping relationship between the preset quality level and the restoration model, the corresponding restoration model is used to restore the fingerprint image. For example, when the mobile phone determines that the fingerprint image (such as the fifth fingerprint image) is a low-quality fingerprint image, the fingerprint image is input into the fingerprint restoration model (such as the third restoration model) for low-quality fingerprint images to restore the fingerprint image to obtain a highly restored image (such as the sixth fingerprint image). When the mobile phone determines that the fingerprint image (such as the third fingerprint image) is a medium-quality fingerprint image, the fingerprint image is input into the fingerprint restoration model (such as the second restoration model) for medium-quality fingerprint images to restore the fingerprint image to obtain a medium-restored image (such as the fourth fingerprint image). When the mobile phone determines that the fingerprint image is a high-quality fingerprint image (such as the first fingerprint image), the fingerprint image is input into a fingerprint repair model for high-quality fingerprint images (such as the first repair model) to repair the fingerprint image to obtain a low-level repaired image (such as the second fingerprint image).

[0124] S504: The mobile phone performs fingerprint recognition based on the repaired fingerprint image.

[0125] Optionally, the mobile phone can perform fingerprint recognition based on the repaired fingerprint image using a fingerprint recognition algorithm known in the art, without limitation. For example, the repaired fingerprint image can be compared with a stored fingerprint image entered by the user, and recognition is successful when the similarity between the two (e.g., structural similarity (SSIM) value) is greater than a threshold.

[0126] Based on Figure 5 The fingerprint recognition method shown in Figure 9 As shown, in an embodiment of the present application, after a mobile phone collects a user's fingerprint information (e.g., fingerprint information including a fingerprint image, an original fingerprint image, and calibration data) through a fingerprint recognition module, the fingerprint recognition process performed by the mobile phone may be: inputting the fingerprint information into a fingerprint quality assessment model so that the fingerprint quality assessment model outputs a corresponding quality assessment result (e.g., an assessment result with a quality level of low quality, medium quality, high quality, etc.). When the quality assessment result output by the fingerprint quality assessment model is low quality, the mobile phone may input the fingerprint image into a fingerprint restoration model and restore the fingerprint image using low-quality fingerprint restoration parameters to obtain a highly restored image. The restored image is then subjected to fingerprint recognition according to a fingerprint recognition algorithm, and user unlocking is performed when recognition passes. When the quality assessment result output by the fingerprint quality assessment model is medium quality, the mobile phone may input the fingerprint image into a fingerprint restoration model and restore the fingerprint image using medium-quality fingerprint restoration parameters to obtain a medium-restored image. The restored image is then subjected to fingerprint recognition according to a fingerprint recognition algorithm, and user unlocking is performed when recognition passes. When the fingerprint quality assessment model outputs a high-quality assessment result, the phone can input the fingerprint image into the fingerprint restoration model and restore it using high-quality fingerprint restoration parameters to obtain a low-quality restored image. The restored image is then subjected to fingerprint recognition using the fingerprint recognition algorithm. If the recognition passes, the user is unlocked.

[0127] Optionally, in the embodiment of the present application, the mobile phone can also repair only the fingerprint images of the quality level that can be repaired (or only the fingerprint images of which the quality level corresponding to the quality assessment result is a preset quality level). Continuing with the electronic device as a mobile phone, the fingerprint images obtained by the fingerprint recognition module (or fingerprint sensor) of the electronic device can be divided into the following categories: Figure 3a The low quality shown, such as Figure 3b The medium quality shown and Figure 3c The three quality levels of High Quality are shown as examples. Figure 10Another fingerprint recognition method provided by the embodiment of the present application is shown. Figure 10 As shown, the method may include the following S1001-S1005.

[0128] S1001. The mobile phone obtains a fingerprint image through a fingerprint recognition module.

[0129] S1002. The mobile phone performs quality assessment on the fingerprint image.

[0130] S1003: The mobile phone determines whether the quality assessment result of the fingerprint image is consistent with the preset quality. If so, the process proceeds to S1004; otherwise, the process proceeds to S1005.

[0131] For example, the preset quality can be a preset quality level of a fingerprint image that can be repaired. For example, if the preset quality level is medium quality, then when the mobile phone determines that the quality level is medium quality after performing a quality assessment on the fingerprint image, the mobile phone can determine that the quality assessment result is consistent with the preset quality. For example, after the mobile phone collects the original image of the user's fingerprint (such as the first original image) through the fingerprint recognition model, it can preprocess the original image based on the preset calibration data to obtain a preprocessed image (i.e., a fingerprint image, such as the first fingerprint image). Then, the mobile phone can perform a quality assessment on the preprocessed image. If it is determined that the quality level of the preprocessed image is medium quality, it can be determined that the quality assessment result of the preprocessed image is consistent with the preset quality level, that is, the quality level of the preprocessed image is the preset quality level. If it is determined that the quality level of the preprocessed image is not medium quality (such as high quality or low quality), it can be determined that the quality assessment result of the preprocessed image is inconsistent with the preset quality level, that is, the quality level of the preprocessed image is not the preset quality level.

[0132] S1004: The mobile phone repairs the fingerprint image and performs fingerprint recognition on the repaired fingerprint image.

[0133] The mobile phone can repair the fingerprint image whose quality assessment result is consistent with the preset quality through the fingerprint repair model. The fingerprint repair model can repair the fingerprint image corresponding to the preset quality to a corresponding degree. Therefore, the fingerprint repair model can refer to Figure 5 The difference between the implementation of the fingerprint restoration model described in S503 is that the fingerprint restoration model only trains fingerprint restoration parameters corresponding to the preset quality.

[0134] S1005. The mobile phone performs fingerprint recognition on the fingerprint image.

[0135] It should be noted that the specific implementations of S1001 and S1002 are respectively as follows: Figure 5The specific implementation of fingerprint recognition in S1004 and S1005 is the same as that in S501 and S502. Figure 5 The fingerprint identification method in S504 is the same or similar and will not be described in detail here.

[0136] It should also be noted that in Figure 10 In the illustrated method, the preset quality levels can also be two, such as medium quality and high quality. Then, when the mobile phone determines that the quality level of the fingerprint image is medium after performing a quality assessment, the mobile phone can determine that the quality assessment result is consistent with the preset quality and repair the fingerprint image using the medium-quality fingerprint repair parameters in the fingerprint repair model to obtain a repaired image. When the mobile phone determines that the quality level of the fingerprint image is high after performing a quality assessment, the mobile phone can determine that the quality assessment result is consistent with the preset quality and repair the fingerprint image using the high-quality fingerprint repair parameters in the fingerprint repair model to obtain a repaired image. When the mobile phone determines that the quality level of the fingerprint image is low after performing a quality assessment, the mobile phone can determine that the quality assessment result is inconsistent with the preset quality and proceed directly to fingerprint recognition without repairing the fingerprint image.

[0137] Based on Figure 10 The fingerprint recognition method shown in Figure 11 As shown, taking the preset quality as medium quality as an example, in an embodiment of the present application, after the mobile phone obtains the user's fingerprint information (e.g., fingerprint information including fingerprint image, original fingerprint image, and calibration data) through the fingerprint recognition module, the fingerprint recognition process performed by the mobile phone may be: inputting the fingerprint information into the fingerprint quality assessment model so that the fingerprint quality assessment model outputs a corresponding quality assessment result (e.g., the assessment result is a quality level of low quality, medium quality, high quality, etc.). When the quality assessment result output by the fingerprint quality assessment model is low quality, the mobile phone can directly perform fingerprint recognition on the fingerprint image according to the fingerprint recognition algorithm, and when the recognition passes, the user is unlocked. When the quality assessment result output by the fingerprint quality assessment model is medium quality, the mobile phone can input the fingerprint image into the fingerprint restoration model and restore the fingerprint image using medium-quality fingerprint restoration parameters to obtain a medium-restored image. The restored image is then fingerprinted according to the fingerprint recognition algorithm, and when the recognition passes, the user is unlocked. When the quality assessment result output by the fingerprint quality assessment model is high quality, the mobile phone can directly perform fingerprint recognition on the fingerprint image according to the fingerprint recognition algorithm, and when the recognition passes, the user is unlocked.

[0138] Optionally, based on the above Figure 5 He Ru Figure 10According to the method shown, in the embodiment of the present application, a fingerprint repair switch can also be set in the setting list of the mobile phone to allow the user to choose whether to perform fingerprint repair when fingerprint recognition is turned on.

[0139] This fingerprint recognition method can be used to perform a corresponding degree of repair based on the quality of the fingerprint image obtained by the fingerprint recognition module before fingerprint recognition. This can improve the quality of the fingerprint image and the fingerprint recognition rate by repairing the fingerprint image, and can also avoid false fingerprint recognition caused by excessive fingerprint image repair.

[0140] Corresponding to the methods in the aforementioned embodiments, embodiments of the present application also provide a fingerprint recognition device. This device can be applied to the aforementioned electronic devices to implement the methods in the aforementioned embodiments. The functions of this device can be implemented through hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. For example, the device includes: a processing module and a fingerprint recognition module. The processing module and fingerprint recognition module can be used together to implement the relevant methods in the aforementioned embodiments.

[0141] It should be understood that the division of units or modules (hereinafter referred to as units) in the above devices is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or physically separated. Furthermore, the units in the device may be implemented entirely in the form of software called through processing elements; entirely in the form of hardware; or partially in the form of software called through processing elements, and partially in the form of hardware.

[0142] For example, each unit can be a separately established processing element, or it can be integrated into a certain chip of the device for implementation. In addition, it can also be stored in a memory in the form of a program, and called by a certain processing element of the device to execute the function of the unit. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above units can be implemented by the integrated logic circuit of the hardware in the processor element or in the form of software called by the processing element.

[0143] In one example, the units in the above apparatus may be one or more integrated circuits configured to implement the above method, such as one or more ASICs, or one or more DSPs, or one or more FPGAs, or a combination of at least two of these integrated circuit forms.

[0144] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a CPU or other processor that can call programs. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0145] In one implementation, the units implementing the corresponding steps of the above methods in the apparatus described above may be implemented in the form of a processing element scheduling program. For example, the apparatus may include a processing element and a storage element, with the processing element invoking a program stored in the storage element to execute the method described in the above method embodiments. The storage element may be a storage element on the same chip as the processing element, i.e., an on-chip storage element.

[0146] In another implementation, the program for executing the above method may be stored in a memory element on a different chip from the processing element, i.e., an off-chip memory element. In this case, the processing element calls or loads the program from the off-chip memory element onto the on-chip memory element to call and execute the method described in the above method embodiment.

[0147] For example, embodiments of the present application may also provide a device, such as an electronic device, which may include a processor and a memory for storing instructions executable by the processor. When the processor is configured to execute the instructions, the electronic device implements the fingerprint recognition method implemented by the electronic device in the aforementioned embodiment. The memory may be located within the electronic device or outside the electronic device. The processor may include one or more.

[0148] In another implementation, the unit of the apparatus implementing each step of the above method may be configured as one or more processing elements, which may be provided on the corresponding electronic device. The processing elements may be integrated circuits, such as one or more ASICs, one or more DSPs, one or more FPGAs, or a combination of these integrated circuits. These integrated circuits may be integrated together to form a chip.

[0149] For example, embodiments of the present application further provide a chip system that can be applied to the aforementioned electronic device. The chip system includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via circuits; the processors receive and execute computer instructions from the electronic device's memory via the interface circuits to implement the electronic device-related methods described in the aforementioned method embodiments.

[0150] An embodiment of the present application also provides a computer program product, including computer instructions executed by an electronic device, such as the electronic device described above.

[0151] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, such as a program. The software product is stored in a program product, such as a computer-readable storage medium, and includes a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0156] For example, embodiments of the present application may further provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by an electronic device, enables the electronic device to implement the fingerprint recognition method described in the aforementioned method embodiment.

[0157] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A fingerprint recognition method, characterized in that: include: Obtaining a first fingerprint image of the user; wherein the first fingerprint image is obtained by removing calibration data from a first original image of the user's fingerprint; the calibration data is preset data of the fingerprint sensor and is generated during production and calibration of the fingerprint sensor; Multimodal data fusion is performed on the first fingerprint image and the first original image to obtain fused data; alternatively, the pixel value of each pixel in the first original image is subtracted from the calibration data value of the corresponding pixel, and then maximum and minimum normalization processing is performed to flatten the calculated pixel values ​​into the same interval to obtain an intermediate image, and multimodal data fusion is performed on the first fingerprint image and the intermediate image to obtain fused data; the expression for the maximum and minimum normalization processing is: Where I is the normalized pixel value, i is the pixel value of the corresponding pixel before normalization, min is the minimum pixel value of each pixel before normalization, and max is the maximum pixel value of each pixel before normalization; Inputting the fused data into a fingerprint quality assessment model, the fingerprint quality assessment model performs a quality assessment on the first fingerprint image based on the fused data to obtain a quality score; wherein the fingerprint quality assessment model is based on a Resent network structure and is trained using a regression approach, the model input during training of the fingerprint quality assessment model is the fused fingerprint data, and the model assessment target during training is the quality score corresponding to the fused fingerprint data; Determining that the first fingerprint image is of a first quality level based on the quality score, calling a first restoration model corresponding to the first quality level, and having the first restoration model perform fingerprint restoration processing on the first fingerprint image using first restoration parameters to obtain a second fingerprint image; wherein the first restoration parameters are restoration parameters obtained by training the first restoration model based on a Unet network structure and using regression, using the first degraded image as a model input and the first training image as a restoration target; the first training image is a fingerprint image having a quality level higher than the first quality level, and the first degraded image is obtained by performing image degradation on the first training image, and the quality level of the first degraded image corresponds to the first quality level; Determining that the first fingerprint image is of a second quality level according to the quality score, calling a second restoration model corresponding to the second quality level, and having the second restoration model perform fingerprint restoration processing on the first fingerprint image using second restoration parameters to obtain a second fingerprint image; wherein the second restoration parameters are restoration parameters obtained by training the second restoration model based on a Unet network structure and adopting the idea of ​​regression with the second degraded image as model input and the second training image as restoration target; the second training image is a fingerprint image having a quality level higher than the second quality level, the second degraded image is obtained by performing image degradation on the second training image, and the quality level of the second degraded image corresponds to the second quality level; the first quality level is better than the second quality level; the degradation degree of the second degraded image is higher than the degradation degree of the first degraded image; the restoration degree of the second restoration parameter is higher than the restoration degree of the first restoration parameter; Determining that the first fingerprint image is of a third quality level according to the quality score, calling a third restoration model corresponding to the third quality level, and having the third restoration model perform fingerprint restoration processing on the first fingerprint image using third restoration parameters to obtain a second fingerprint image; wherein the third restoration parameters are restoration parameters obtained by training the third restoration model based on a Unet network structure and adopting the idea of ​​regression, using a third degraded image as a model input and a third training image as a restoration target; the third training image is a fingerprint image having a quality level higher than the third quality level, and the quality level of the third training image is lower than the quality levels of the first training image and the second training image; the third degraded image is obtained by performing image degradation on the third training image, and the degradation degree of the third degraded image is higher than the degradation degree of the second degraded image; the quality level of the third degraded image corresponds to the third quality level; the restoration degree of the third restoration parameter is higher than that of the second restoration parameter; The second fingerprint image is used to perform fingerprint recognition.

2. The method according to claim 1, characterized in that The degree of change between the second fingerprint image obtained by the first restoration model and the first restoration parameter and the first fingerprint image is smaller than the degree of change between the second fingerprint image obtained by the second restoration model and the second restoration parameter and the first fingerprint image.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Determining the first repair parameter according to the first quality level and a preset mapping relationship between the quality level and the repair parameter; determining a restoration model including the first restoration parameter as the first restoration model; Determining the second repair parameter according to the second quality level and a preset mapping relationship between the quality level and the repair parameter; A restoration model including the second restoration parameters is determined as the second restoration model.

4. The method according to claim 1 or 2, characterized in that The first repair model is the same as or different from the second repair model.

5. The method according to claim 1 or 2, characterized in that The first repair model and the second repair model are different; and the method further comprises: Determining the first repair model according to the first quality level and a preset mapping relationship between the quality level and the repair model; According to the second quality level, the second repair model is determined according to a preset mapping relationship between the quality level and the repair model.

6. The method according to claim 1 or 2, characterized in that The obtaining of the first fingerprint image of the user includes: collecting the first original image of the user's fingerprint; The first original image is preprocessed according to preset calibration data to obtain the first fingerprint image; the preprocessing includes removing the calibration data from the first original image.

7. A fingerprint recognition method, characterized in that: include: Obtaining a first fingerprint image of the user; wherein the first fingerprint image is obtained by removing calibration data from a first original image of the user's fingerprint; the calibration data is preset data of the fingerprint sensor and is generated during production and calibration of the fingerprint sensor; Multimodal data fusion is performed on the first fingerprint image and the first original image to obtain fused data; alternatively, the pixel value of each pixel in the first original image is subtracted from the calibration data value of the corresponding pixel, and then maximum and minimum normalization processing is performed to flatten the calculated pixel values ​​into the same interval to obtain an intermediate image, and multimodal data fusion is performed on the first fingerprint image and the intermediate image to obtain fused data; the expression for the maximum and minimum normalization processing is: Where I is the normalized pixel value, i is the pixel value of the corresponding pixel before normalization, min is the minimum pixel value of each pixel before normalization, and max is the maximum pixel value of each pixel before normalization; Inputting the fused data into a fingerprint quality assessment model, the fingerprint quality assessment model performs a quality assessment on the first fingerprint image based on the fused data to obtain a quality score; wherein the fingerprint quality assessment model is based on a Resent network structure and is trained using a regression approach, the model input during training of the fingerprint quality assessment model is the fused fingerprint data, and the model assessment target during training is the quality score corresponding to the fused fingerprint data; Determining, according to the quality score, that the first fingerprint image is of a first quality level, a second quality level, or a third quality level; wherein the fingerprint image of the first quality level has the highest quality, and the fingerprint image of the third quality level has the lowest quality; When the first fingerprint image is of the first quality level and the first quality level is a preset quality level, a first restoration model corresponding to the first quality level is called, and the first restoration model performs fingerprint restoration processing on the first fingerprint image using first restoration parameters to obtain a second fingerprint image; wherein the first restoration parameters are restoration parameters obtained by training the first restoration model based on a Unet network structure and using regression ideas, using the first degraded image as a model input and the first training image as a restoration target; the first training image is a fingerprint image having a quality level higher than the first quality level, and the first degraded image is obtained by performing image degradation on the first training image, and the quality level of the first degraded image corresponds to the first quality level; When the first fingerprint image is of the second quality level and the second quality level is a preset quality level, a second restoration model corresponding to the second quality level is called, and the second restoration model performs fingerprint restoration processing on the first fingerprint image using second restoration parameters to obtain a second fingerprint image; wherein the second restoration parameters are restoration parameters obtained by training the second restoration model based on a Unet network structure and adopting a regression idea with a second degraded image as a model input and a second training image as a restoration target; the second training image is a fingerprint image with a quality level higher than the second quality level, the second degraded image is obtained by performing image degradation on the second training image, and the quality level of the second degraded image corresponds to the second quality level; the first quality level is better than the second quality level; the degradation degree of the second degraded image is higher than the degradation degree of the first degraded image; and the restoration degree of the second restoration parameter is higher than that of the first restoration parameter; When the first fingerprint image is of the third quality level and the third quality level is a preset quality level, a third restoration model corresponding to the third quality level is called, and the third restoration model performs fingerprint restoration processing on the first fingerprint image using third restoration parameters to obtain a second fingerprint image; wherein the third restoration parameters are restoration parameters obtained by training the third restoration model based on a Unet network structure and using a regression idea with a third degraded image as a model input and a third training image as a restoration target; the third training image is a fingerprint image with a quality level higher than the third quality level, and the quality level of the third training image is lower than the quality levels of the first training image and the second training image; the third degraded image is obtained by performing image degradation on the third training image, and the degradation degree of the third degraded image is higher than the degradation degree of the second degraded image; the quality level of the third degraded image corresponds to the third quality level; the restoration degree of the third restoration parameter is higher than the restoration degree of the second restoration parameter; The second fingerprint image is used to perform fingerprint recognition.

8. The method according to claim 7, characterized in that The method further comprises: When the first fingerprint image is at the first quality level and the first quality level is not the preset quality level, or when the first fingerprint image is at the second quality level and the second quality level is not the preset quality level, or when the first fingerprint image is at the third quality level and the third quality level is not the preset quality level, the first fingerprint image is used for fingerprint recognition.

9. The method according to claim 7 or 8, characterized in that The preset quality level includes any one or more of the first quality level, the second quality level, and the third quality level.

10. The method according to claim 7 or 8, characterized in that The obtaining of the first fingerprint image of the user includes: collecting the first original image of the user's fingerprint; The first original image is preprocessed according to preset calibration data to obtain the first fingerprint image; the preprocessing includes removing the calibration data from the first original image.

11. An electronic device, characterized in that: include: A processor, a memory for storing instructions executable by the processor, wherein when the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 10.

12. A computer-readable storage medium having computer program instructions stored thereon; characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 10.

13. A computer program product, characterized in that The method comprises a computer-readable code, which, when executed in an electronic device, enables the electronic device to implement the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 10.

Images