Fingerprint image processing method and terminal equipment

Abstract

The invention discloses a fingerprint image processing method and terminal equipment, and relates to the technical field of terminals, and the method comprises the steps that the terminal equipment responds to a fingerprint registration operation of a user, obtains a first fingerprint image and a second fingerprint image, carries out the splicing of the first fingerprint image and the second fingerprint image through the fingerprint shape, obtains a spliced fingerprint image, and sends the spliced fingerprint image to the terminal equipment; and obtaining a spliced fingerprint image, removing a blurred image in the spliced fingerprint image, obtaining a third fingerprint image, and generating a registered fingerprint template according to the third fingerprint image. As the blurred image in the third fingerprint image is removed, the confidence coefficient of the registered fingerprint template generated according to the third fingerprint image is high, and the problem that the security of fingerprint identification is reduced due to the use of the fingerprint image with a blurred partial region is solved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a fingerprint image processing method and a terminal device. Background Art

[0002] Fingerprint recognition technology is a type of biometric technology. Due to the uniqueness, immutability and ease of collection of fingerprints, fingerprint recognition technology is widely used in terminal devices.

[0003] Fingerprint recognition is usually divided into two processes: registration and authentication. For example, for a terminal device to successfully perform fingerprint recognition, the user needs to first register the fingerprint on the terminal device; that is, the user pre-enters the fingerprint on the terminal device, and the pre-entered fingerprint can be called a registered fingerprint. When the terminal device performs identity authentication, it can compare the user's input fingerprint for identity authentication with the registered fingerprint to determine the identity of the user who input the fingerprint.

[0004] Due to factors such as the setting form of the fingerprint sensor on the terminal device and the capabilities of the fingerprint sensor, the fingerprint image collected by the fingerprint sensor may be incomplete or unclear, resulting in reduced security of fingerprint recognition. Summary of the invention

[0005] The embodiment of the present application provides a fingerprint image processing method and terminal device, which can remove blurred images in fingerprint images, improve the confidence of registered fingerprint templates, and solve the problem that the security of fingerprint recognition is reduced due to the use of fingerprint images with blurred areas.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a fingerprint image processing method is provided, the method comprising:

[0008] The terminal device obtains a first fingerprint image and a second fingerprint image in response to a fingerprint registration operation of the user; the image in the first area of ​​the first fingerprint image and the second area of ​​the second fingerprint image is blurred, and the images are spliced ​​according to the fingerprint shapes of the first fingerprint image and the second fingerprint image to obtain a spliced ​​fingerprint image, and then the blurred image in the spliced ​​fingerprint image is removed to obtain a third fingerprint image, and the third fingerprint image is used to generate a registered fingerprint template.

[0009] In this embodiment, since the blurred image in the third fingerprint image is removed, the confidence of the registered fingerprint template generated according to the third fingerprint image is higher, which solves the problem of reduced security of fingerprint recognition caused by using a fingerprint image with blurred areas.

[0010] In a possible implementation, the first area of ​​the first fingerprint image (also referred to as the blurred area of ​​the first fingerprint image) and the second area of ​​the second fingerprint image (also referred to as the blurred area of ​​the second fingerprint image) are blurred, which is reflected in that the grayscale level of the image is less than or equal to the first judgment threshold, or the light-dark contrast of the image is less than the second judgment threshold.

[0011] In a possible implementation manner, the first judgment threshold includes: a grayscale level judgment threshold, and the second judgment threshold includes: a brightness judgment threshold.

[0012] In one possible implementation, the first fingerprint image and the second fingerprint image are obtained by a fingerprint sensor disposed in a terminal device. The fingerprint sensor adopts a curved surface package, and the middle area of ​​the packaging layer is far away from the fingerprint chip of the fingerprint sensor, and the two end areas are close to the fingerprint chip of the fingerprint sensor. This makes the images of the first area of ​​the first fingerprint image and the second area of ​​the second fingerprint image obtained by the terminal device blurred.

[0013] In this embodiment, when there are blurred areas in the first fingerprint image and the second fingerprint image obtained, a third fingerprint image is obtained by removing the blurred image in the spliced ​​fingerprint image, and then the registered fingerprint template is generated using the third fingerprint image. This method can be applied to a terminal device with a fingerprint sensor packaged with a curved surface, thereby enriching the application scenarios of fingerprint recognition.

[0014] In a possible implementation manner, the first fingerprint image and the second fingerprint image are valid fingerprint images obtained by deduplication screening in multiple frames of fingerprint images.

[0015] In a possible implementation, the terminal device acquires multiple frames of fingerprint images in response to a fingerprint registration operation of the user, and determines fingerprint images whose similar feature point pairs satisfy a preset first screening condition in the multiple frames of fingerprint images.

[0016] In a possible implementation, satisfying the preset first screening condition includes: the number of similar feature point pairs between the first fingerprint image and the second fingerprint image is less than or equal to the maximum value of the preset judgment threshold interval, and greater than or equal to the minimum value of the preset judgment threshold interval.

[0017] In a possible implementation, the number of first fingerprint images and second fingerprint images can be multiple frames according to multiple frames of fingerprint images, the first fingerprint image can be a reference fingerprint image in the multiple frames of fingerprint images, and the second fingerprint image is a fingerprint image that satisfies a preset first screening condition in the multiple frames of fingerprint images when the first fingerprint image is used as the reference fingerprint image. Both the first fingerprint image and the second fingerprint image are valid fingerprint images.

[0018] In a possible implementation, when the first fingerprint image and the second fingerprint image are spliced ​​according to the fingerprint shape, feature points in the first fingerprint image and the second fingerprint image are respectively obtained, and the first fingerprint image and the second fingerprint image are spliced ​​using similar feature points.

[0019] In this embodiment, feature points in the first fingerprint image and the second fingerprint image are obtained, and the first fingerprint image and the second fingerprint image can be aligned according to similar feature points of the first fingerprint image and the second fingerprint image, which is beneficial to the stitching of the fingerprint images.

[0020] In one possible implementation, the first fingerprint image includes a third area, in which the image is clear, and after the terminal device splices the first fingerprint image and the second fingerprint image, the first sub-area in the third area overlaps with the second sub-area in the second area, and in the spliced ​​fingerprint image, the area where the third area overlaps with the second area uses the image of the first sub-area.

[0021] In this embodiment, the image in the third area of ​​the first fingerprint image is clear. When the first sub-area of ​​the third area overlaps with the second sub-area of ​​the second area, the first sub-area is used in the overlapping area. Since the first sub-area is part of the third area, the image is clear. This can make the area of ​​the clear area of ​​the stitched fingerprint image larger, which is beneficial to subsequent processing.

[0022] In a possible implementation, the first fingerprint image may be a fingerprint image of the area where the entire third area or the first sub-area of ​​the third area overlaps with the second fingerprint image in the two fingerprint image frames involved in stitching, that is, the clear area overlaps with the second fingerprint image in whole or in part, and the second fingerprint image may be a fingerprint image of the area where the entire second area or the second sub-area of ​​the second area overlaps with the first fingerprint image in the two fingerprint image frames involved in stitching, that is, the blurred area overlaps with the first fingerprint image in whole or in part.

[0023] In a possible implementation, the fingerprint registration operation includes a pressing operation or a sliding operation on the fingerprint collection area.

[0024] In a possible implementation manner, the fingerprint collection area includes a packaging surface area of ​​the fingerprint sensor.

[0025] In a possible implementation manner, when removing the blurred image in the spliced ​​fingerprint image, the terminal device sets the grayscale value of the pixel points in the blurred area in the spliced ​​fingerprint image to a preset value.

[0026] In this way, by setting the grayscale value of the pixel point in the fuzzy area in the spliced ​​fingerprint image to a preset value, the influence of the fuzzy area in the spliced ​​fingerprint image on fingerprint recognition can be avoided.

[0027] In a possible implementation manner, the preset value may be 255.

[0028] In one possible implementation, the terminal device obtains a fourth fingerprint image in response to a fingerprint authentication operation of the user, the image of the fourth area in the fourth fingerprint image is clear, and the image of the fifth area is blurred, and the terminal device performs similarity matching with the registered fingerprint template based on the feature points of the fourth area of ​​the fourth fingerprint image to obtain a first similarity score. If the first similarity score meets a preset matching condition, it is determined that the fingerprint authentication is successful.

[0029] In this embodiment, the feature points of the fourth area of ​​the fourth fingerprint image are preferentially used to perform similarity matching with the registered fingerprint template, that is, the feature points of the clear area are preferentially used to perform similarity matching with the registered fingerprint template. When the first similarity score meets the preset matching condition, it is determined that the fingerprint verification is successful, and the feature points of the fifth area of ​​the fourth fingerprint image are no longer used to perform similarity matching with the registered fingerprint template, which can ensure the security of fingerprint recognition while taking into account the efficiency of fingerprint recognition.

[0030] In one possible implementation, if the first similarity score does not meet the preset matching conditions, the terminal device performs similarity matching with the registered fingerprint template based on the feature points of the fifth area of ​​the fourth fingerprint image to obtain a second similarity score, and weightedly sums the first similarity score and the second similarity score to obtain a target similarity score, the weight of the first similarity score is greater than the weight of the second similarity score, and if the target similarity score does not meet the preset matching conditions, it is determined that the fingerprint authentication has failed, and if the target similarity score meets the preset matching conditions, it is determined that the fingerprint authentication has succeeded.

[0031] In this embodiment, the target similarity score is obtained by weighted summing the first similarity score and the second similarity score. By setting different weights for the first similarity score and the second similarity score, and the weight of the first similarity score is higher than the weight of the second similarity score, the influence of the second similarity score on the target similarity score can be reduced, thereby reducing the influence on the fingerprint recognition security when the feature points of the blurred area of ​​the fourth fingerprint image are matched with the registered fingerprint template for similarity.

[0032] In a possible implementation, after the fingerprint authentication is successful, the terminal device updates the registered fingerprint template according to the feature points of the fourth area of ​​the fourth fingerprint image.

[0033] In a possible implementation manner, the fourth fingerprint image includes an input fingerprint image.

[0034] In a second aspect, a terminal device is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, and when the terminal device is running, the processor executes the computer execution instructions stored in the memory to enable the terminal device to perform a method as described in any one of the above-mentioned first aspects.

[0035] In a third aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on a computer, the computer can execute any of the methods described in the first aspect.

[0036] According to a fourth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any one of the methods according to the first aspect.

[0037] Among them, the technical effects brought about by any design method in the second to fourth aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of the structure of a fingerprint sensor;

[0039] Figure 2 A schematic diagram of a fingerprint image applicable to the method provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of a fingerprint sensor using curved surface packaging;

[0041] Figure 4 A schematic diagram of a fingerprint image applicable to the method provided in the embodiment of the present application;

[0042] Figure 5 A schematic diagram of the structure of a fingerprint sensor provided on a watch;

[0043] Figure 6 A schematic diagram of the structure of a fingerprint sensor disposed on a watch applicable to the method provided in an embodiment of the present application;

[0044] Figure 7 A method flow chart applicable to the method provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of a scenario applicable to the method provided in an embodiment of the present application;

[0046] Fig. 9 A schematic diagram of a fingerprint image applicable to the method provided in an embodiment of the present application;

[0047] Fig.10A schematic diagram of a spliced ​​fingerprint image applicable to the method provided in an embodiment of the present application;

[0048] Fig.11 A schematic diagram of a target fingerprint image applicable to the method provided in an embodiment of the present application;

[0049] Fig.12 A schematic diagram of fingerprint feature extraction applicable to the method provided in an embodiment of the present application;

[0050] Fig.13 A method flow chart applicable to a method provided in another embodiment of the present application;

[0051] Fig.14 A method flow chart applicable to a method provided in another embodiment of the present application;

[0052] Fig.15 A schematic diagram of the structural composition of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include such expressions as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are a kind of "or" relationship.

[0054] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0055] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0056] Fingerprint recognition technology is a type of biometric technology. Fingerprint recognition technology collects fingerprint images, obtains the feature points of the fingerprint images, matches the input fingerprint template generated based on the feature points of the fingerprint images with the saved registered fingerprint template, and obtains the fingerprint recognition result. Due to the uniqueness, immutability, and ease of collection of fingerprints, fingerprint recognition technology is widely used in identity authentication scenarios such as unlocking, payment, and login.

[0057] With the development of smart terminal devices, more and more terminal devices have fingerprint recognition function. The complete fingerprint recognition function includes fingerprint registration process and fingerprint authentication process. The fingerprint registration process includes: fingerprint image acquisition, fingerprint image processing, fingerprint feature extraction, saving registered fingerprint template, etc.; the fingerprint authentication process includes: fingerprint image acquisition, fingerprint image processing, fingerprint feature extraction, matching with registered fingerprint template, etc.

[0058] The terminal device is provided with a fingerprint sensor for collecting the user's fingerprint image. After obtaining the fingerprint image, the terminal device implements fingerprint image stitching, fingerprint feature extraction, fingerprint feature matching (also known as similarity matching) and other processes through fingerprint algorithms such as image processing, feature extraction, and feature matching, and obtains the fingerprint recognition result. If the recognition result is passed, the terminal device can respond to the user's operation and perform the corresponding action. For example, if the user's operation is to unlock, then after the fingerprint recognition result is passed, the terminal device performs the action of unlocking the screen lock and entering the system operation interface. For another example, if the user's operation is to pay, then after the fingerprint recognition result is passed, the terminal device performs the payment action.

[0059] Among them, fingerprint sensors are divided into capacitive fingerprint sensors, optical fingerprint sensors, ultrasonic fingerprint sensors, etc. according to different recognition principles. Optical fingerprint sensors and ultrasonic fingerprint sensors are usually set under the screen of the terminal device, and capacitive fingerprint sensors are usually set in the non-screen area of ​​the terminal device, such as the front cover, side frame, back cover and other areas. The capacitive fingerprint sensor set on the side frame of the terminal device is limited by the size of the side frame, and usually adopts a small-sized sliding capacitive fingerprint sensor. The embodiment of this application takes a small-sized sliding capacitive fingerprint sensor as an example. For the convenience of description, the fingerprint sensors introduced below are all small-sized sliding capacitive fingerprint sensors unless otherwise specified.

[0060] For example, the structure of the fingerprint sensor is as follows: Figure 1 As shown, it includes a fingerprint chip, a substrate, a packaging material layer, a coating layer, and an electrical connector. The packaging material covers the fingerprint chip to form a packaging material layer, which is used to protect the fingerprint chip and the electrical connector. The surface of the common packaging material layer is a plane, and a coating layer is also provided above the packaging material layer. The surface of the coating layer is also a plane, which is used to further protect the fingerprint sensor. The coating layer is in direct contact with the finger, and the fingerprint chip collects capacitance signals through the packaging material layer and the coating layer. The substrate is welded to the fingerprint chip through the electrical connector, which is used to transmit the capacitance signals collected by the fingerprint chip to other electronic modules on the substrate, or other electronic modules electrically connected to the substrate for processing. The other electronic modules include but are not limited to processing modules or storage modules.

[0061] It is known that the penetration ability of the fingerprint chip is limited. If the thickness of the packaging material layer and the coating layer (hereinafter referred to as the packaging layer) exceeds the rated thickness corresponding to the penetration ability of the fingerprint chip, it will affect the imaging quality of the fingerprint image. When the thickness of the packaging layer does not exceed the rated thickness, the clarity of the fingerprint image can meet the requirements of the fingerprint algorithm. A clear fingerprint image is conducive to the fingerprint algorithm to identify more fingerprint feature points with higher confidence, which will improve the security of the fingerprint and the efficiency of fingerprint recognition. The feature points of the fingerprint include: the end point of the ridge line, the bifurcation point, etc. Figure 2 As shown, the end point of the ridge line and the bifurcation point of the ridge line are displayed.

[0062] The imaging quality of the fingerprint image is related to the penetration ability of the fingerprint chip and the thickness of the packaging layer. For example, if the thickness of the fingerprint sensor's packaging layer exceeds the penetration ability of the fingerprint chip, the overall quality of the fingerprint image will be poor, the fingerprint image will be blurred, and the features of the fingerprint feature points will not be obvious. If the thickness of the fingerprint sensor's packaging layer is uneven, the fingerprint image will appear to be clear in some areas and blurred in others. Figure 3 As shown in (a), a fingerprint sensor using curved surface packaging includes: a 3D steel patch (also called a structural support member), a substrate, a fingerprint chip and a packaging layer, such as Figure 3 As shown in (b), the fingerprint sensor is horizontally represented, the thickness of the packaging layer in the middle area is d1, and the thickness of the packaging layer in the edge area is d2, d2 is less than d1, and d1 is greater than the rated thickness d3 that the fingerprint chip can penetrate. The terminal device obtains the fingerprint image according to the capacitance signal detected by the fingerprint sensor using the curved surface packaging as shown in Figure 4 As shown, the upper and lower ends of the fingerprint image are clear, while the middle is blurred.

[0063] In some embodiments, some terminal devices adopt a rectangular frame design so that a flat-packaged fingerprint sensor is set on the device frame. These terminal devices are, for example, mobile phones, watches, tablet computers, etc.

[0064] In other embodiments, some terminal devices adopt a curved frame design. If a flat-package fingerprint sensor is set on the terminal device adopting the curved frame design, the protruding frames on both sides will be higher in height while ensuring that the height of the protruding frame in the middle is normal to facilitate the fingerprint sensor to collect the capacitive signal, making the overall appearance of the terminal device less beautiful. Figure 5 In the smart watch shown, the vertical distance between the middle area of ​​the fingerprint sensor's flat package and the fingerprint chip is h1, and the vertical distance between the two side areas and the fingerprint chip is h2, where h2 is equal to h1. That is to say, the vertical distance between each point on the package surface (coating layer) of the fingerprint sensor and the fingerprint chip is equal, so that a uniform and clear fingerprint image can be obtained. However, since the frame of the watch is curved, the two sides of the surface package of the fingerprint sensor will protrude too much from the device frame, resulting in an unsightly appearance.

[0065] If a fingerprint sensor with curved surface packaging is set on a terminal device with curved frame design, such as Figure 6In the smart watch shown in the figure, the vertical distance between the middle area of ​​the arc package of the fingerprint sensor and the fingerprint chip is h3, and the vertical distance between the two side areas and the fingerprint chip is h4. h3 is greater than h4. In other words, the packaging layer of the fingerprint sensor is not uniform in thickness relative to the fingerprint chip. The middle area is thicker and the two side areas are thinner. The obtained fingerprint image will appear as follows Figure 4 The fingerprint image shown is blurred in the middle and clear on both sides. When extracting image features from a fingerprint image with blurred areas, the confidence of the feature points in the blurred areas is low. Using these low-confidence feature points as fingerprint templates will reduce the security of fingerprint recognition in the fingerprint authentication process.

[0066] The embodiment of the present application provides a fingerprint image processing method, which can be applied to a terminal device provided with a fingerprint sensor with an arc surface package. When the user's finger presses the fingerprint sensor (also referred to as a pressing operation) or slides on the fingerprint sensor (also referred to as a sliding operation), the fingerprint sensor obtains multiple sets of capacitance signals in multiple scanning cycles, and the terminal device obtains multiple frames of fingerprint images according to the multiple sets of capacitance signals. The terminal device performs image data analysis on the multiple frames of fingerprint images, determines the clear area and the blurred area of ​​each frame of the fingerprint image, and uses the clear area and the blurred area of ​​the fingerprint image to splice the multiple frames of fingerprint images. When the clear area overlaps with the blurred area, the clear area is retained in the overlapping area. After obtaining the spliced ​​fingerprint image, the blurred area of ​​the spliced ​​fingerprint image is discarded to obtain a clear target fingerprint image. Since there is no blurred area in the target fingerprint image, the fingerprint algorithm extracts features from the target fingerprint image to obtain feature points with high confidence. Using the feature points with high confidence as the registered fingerprint template can solve the problem of reduced security of fingerprint recognition caused by using a fingerprint image with blurred areas in some areas.

[0067] Furthermore, the fingerprint image processing method provided in the embodiment of the present application can be applied to the fingerprint registration process. After obtaining the target fingerprint image, the existing fingerprint algorithm can be used to extract features of the target fingerprint image and save the registered fingerprint template. There is no need to make additional changes to the fingerprint algorithm, and it has better applicability.

[0068] Furthermore, the fingerprint image processing method provided in the embodiment of the present application can process the fingerprint image collected by the fingerprint sensor using curved surface packaging at the software level without changing the internal structure of the fingerprint sensor, especially the shape of the fingerprint chip and the substrate, without increasing the hardware cost, and has better economy.

[0069] In some implementations, the fingerprint image processing method provided in the embodiments of the present application can be applied to the fingerprint registration process, such as Figure 7As shown, the terminal device is a watch, and the fingerprint registration process on the watch is introduced as an example. Exemplarily, the fingerprint registration process includes S701 to S708, wherein:

[0070] S701: In response to receiving a fingerprint registration operation from a user, the watch starts fingerprint registration.

[0071] The user's fingerprint registration operation includes, for example: the user triggers the "Register Fingerprint" control on the setting page of the watch; or the user inputs a voice command to register the fingerprint through the voice assistant of the watch.

[0072] In some embodiments, after the watch detects the user's fingerprint registration operation, it can activate the fingerprint sensor in a dormant state, and can also display auxiliary prompt information for registering the fingerprint in the settings page of the watch. The auxiliary prompt information is used to guide the user to press or slide the finger to touch the fingerprint sensor, prompt the user to lift the finger to prepare to press or slide again, prompt the user to complete the fingerprint registration, etc.

[0073] S702: The watch collects multiple frames of fingerprint images.

[0074] In some embodiments, the fingerprint sensor periodically scans the package surface to collect capacitance signals, and collects capacitance signals generated when the user's finger presses or slides. Among them, the scanning cycle of the fingerprint sensor is very fast. From the time the user's finger presses the fingerprint sensor to the time the finger moves away from the fingerprint sensor, or when the finger completes a slide on the fingerprint sensor, it will go through multiple scanning cycles. In each scanning cycle, the fingerprint sensor collects a set of capacitance signals. Exemplarily, the user's finger sliding on the fingerprint sensor includes, for example, Figure 8 (a) to Figure 8 As shown in (b), sliding from bottom to top, it can also be as follows Figure 8 (c) to Figure 8 (d) Slide from left to right. In addition, it can also be slid from the lower left to the upper right, or slid in other directions. The embodiment of the present application does not impose any limitation on the sliding direction.

[0075] In some embodiments, the watch obtains the capacitance signal collected by the fingerprint sensor, performs analog-to-digital conversion (ADC), digital signal processor (DSP), etc. on the capacitance signal to obtain a fingerprint image. Each set of capacitance signals obtains a frame of fingerprint image, and the watch obtains multiple frames of fingerprint images according to multiple sets of signals from the user's finger pressing the fingerprint sensor to the finger moving away from the fingerprint sensor. Among them, the fingerprint image can be a Raw image, which means an image directly converted from a digital signal without calibration processing.

[0076] Exemplarily, the fingerprint image can also be obtained by image preprocessing based on the Raw image, for example, the Raw image is preprocessed with preset calibration (Base) data to obtain a preprocessed fingerprint image. Preprocessing the Raw image using Base data includes but is not limited to: image denoising, binarization, Gaussian filtering, de-moiré, etc.

[0077] S703: The watch performs image data analysis on multiple fingerprint image frames to determine the clear area and blurred area of ​​each fingerprint image frame.

[0078] In some embodiments, the watch can determine the clear area and the blurred area of ​​the fingerprint image according to the grayscale (hereinafter referred to as grayscale) change of the fingerprint image. The grayscale of the image represents the different brightness levels of the fingerprint image from the darkest to the brightest. The more levels, the clearer the fingerprint image. Taking an 8-bit fingerprint image as an example, the fingerprint image has a maximum of 28 brightness levels, that is, 0 to 255 brightness levels. The grayscale levels of the image include: 256, 128, 64, 32, 16, 8, 4, and 2. The more grayscale levels (the larger the number), the clearer the fingerprint image and the more obvious the image features. The lower the grayscale level (the smaller the number), the more blurred the fingerprint image and the more blurred the image features. In the embodiment of the present application, the fingerprint image includes a clear area and a blurred area. When the grayscale changes, it means that the clarity of the fingerprint image has changed. The grayscale change includes: an increase in the grayscale level and a decrease in the grayscale level. An increase in the grayscale level indicates that the fingerprint image changes from blurry to clear, and a decrease in the grayscale level indicates that the fingerprint image changes from clear to blurry.

[0079] For example, Fig. 9In a fingerprint image, the watch analyzes the grayscale changes of the fingerprint image to determine the clear area and the blurred area of ​​the fingerprint image. The L1 area is the transition area where the fingerprint image changes from clear to blurred. In the L1 area, the grayscale level of the fingerprint image continues to decrease. The L2 area is the transition area where the fingerprint image changes from blurred to clear. In the L2 area, the grayscale level of the fingerprint image continues to increase. In some specific implementations, a grayscale level judgment threshold can be set. When the grayscale level of the fingerprint image is less than or equal to the grayscale level judgment threshold, it means that the fingerprint features in the area can no longer be accurately extracted by the fingerprint image, or the confidence of the extracted feature points is low. In one example, the L1 area and the L2 area have a certain pixel width, which is greater than 1 pixel. The watch divides the fingerprint image into a clear area and a blurred area according to the pixel positions of the L1 area and the L2 area in the fingerprint image and the grayscale level judgment threshold. It can be known that, taking the transition area where the fingerprint image changes from clear to blurred in the L1 area as an example, if the grayscale of the fingerprint image is constantly decreasing, and at a certain grayscale level the number of grayscales is less than or equal to the grayscale level judgment threshold, then the boundary from clear to blurred is in the L1 area. Similarly, in the L2 area, if the grayscale of the fingerprint image is constantly increasing, and at a certain grayscale level the number of grayscales is greater than the grayscale level judgment threshold, then the boundary from blurred to clear is in the L2 area.

[0080] In another example, the fingerprint image can be divided into several areas, each area includes a certain number of pixels, and the watch determines the grayscale level of each area. If the grayscale level of a certain area is less than or equal to the preset grayscale level judgment threshold, the area is determined as a blurred area. If the grayscale level of a certain area is greater than the preset grayscale level judgment threshold, the area is determined as a clear area.

[0081] In some embodiments, the watch can also determine the clear area and the blurred area of ​​the fingerprint image according to the light and dark contrast of the fingerprint image. Since the fingerprint image is a grayscale image, the grayscale value of the pixel points in the ridge part is low, which constitutes the dark part of the fingerprint image, and the grayscale value of the pixel points in the valley part is high, which constitutes the bright part of the fingerprint image. The light and dark contrast represents the difference between the grayscale value of the pixel points belonging to the bright part and the grayscale value of the pixel points belonging to the dark part in the fingerprint image. The smaller the difference, the more blurred the boundary between the ridge and the valley in the fingerprint image, and the larger the difference, the clearer the boundary between the ridge and the valley in the fingerprint image. Exemplarily, the clear area and the blurred area in the fingerprint image can be determined according to the light and dark contrast of the pixels in each area of ​​the fingerprint image. When the light and dark contrast of a certain area is less than or equal to the preset light and dark judgment threshold, the area is determined to be a blurred area. On the contrary, if the light and dark contrast is greater than the preset light and dark judgment threshold, the area is determined to be a clear area. Among them, the preset light and dark judgment threshold can be set according to whether the fingerprint algorithm can accurately extract the feature points in the fingerprint image. The fingerprint algorithm in the blurred area is difficult to extract the feature points, and the confidence of the extracted feature points is low.

[0082] It is known that not all fingerprint images appear like this. Fig. 9 The fingerprint image shown is blurred in the middle and clear at both ends. In some embodiments, the fingerprint image may also be clear at the top and blurred at the bottom, or clear at the left and blurred at the right. The method of the embodiments of the present application can be used for screening, and the embodiments of the present application will not be described one by one.

[0083] In some embodiments, after the clear area and the blurred area of ​​the fingerprint image are determined, step S705 is executed to screen the fingerprint images.

[0084] In some embodiments, after the clear area and the blurred area of ​​the fingerprint image are determined, the image quality of multiple frames of fingerprint images can be screened according to the ratio of the clear area to the total area to obtain multiple frames of candidate fingerprint images.

[0085] Exemplarily, an image quality screening condition may be set. If the ratio of the clear area to the total area of ​​the fingerprint image meets the image quality screening condition, the fingerprint image is determined as a candidate fingerprint image. Otherwise, the fingerprint image is ignored. After comparing all fingerprint images, multiple frames of candidate fingerprint images are obtained. Step S704 performs screening based on the multiple frames of candidate fingerprint images.

[0086] In some embodiments, the image quality screening condition includes a quality judgment threshold. When the ratio of the clear area of ​​the fingerprint image to the total area is greater than the quality judgment threshold, the fingerprint image is determined as a candidate fingerprint image, otherwise the fingerprint image is ignored. Exemplarily, according to the application scenario of the embodiment of the present application, when the fingerprint image obtained by the fingerprint sensor with curved surface packaging includes a part of the fuzzy area, if the image quality screening threshold is set too low, such as 10%, the clear area of ​​the screened part of the fingerprint image is small, and the feature points with high confidence are also small. In order to obtain enough feature points with high confidence, more fingerprint images are required to participate in the splicing, which is inefficient. If the image quality screening threshold is set too high, such as 90%, the number of fingerprint images that meet the conditions is small. It may result in an insufficient number of fingerprint images that meet the conditions, and it is necessary to obtain more fingerprint images. It can be known that the quality judgment threshold can be set in combination with actual needs, the penetration ability of the fingerprint chip, the packaging thickness, the algorithm requirements, etc. The embodiment of the present application does not impose any restrictions on specific values.

[0087] S704: The watch performs deduplication screening on multiple frames of fingerprint images to obtain multiple frames of valid fingerprint images.

[0088] In some embodiments, when the watch performs deduplication screening on multiple frames of fingerprint images, if the above step S703 performs image quality screening on multiple frames of fingerprint images to obtain multiple frames of candidate fingerprint images, then the watch performs deduplication screening on the multiple frames of candidate fingerprint images. If the above step S703 does not perform image quality screening on multiple frames of fingerprint images, then the watch performs deduplication screening on multiple frames of fingerprint images with clear areas and fuzzy areas determined. Generally, when the user's finger is in good contact with the fingerprint sensor, and the finger surface and the packaging surface of the fingerprint sensor are kept clean, the ratio of the clear area of ​​the obtained fingerprint image to the total area of ​​the fingerprint image is between 50% and 60%. The process of the watch performing deduplication screening on multiple frames of fingerprint images in the following example is also applicable to the process of the watch performing deduplication screening on multiple frames of candidate fingerprint images, and the embodiments of the present application will not be described in detail.

[0089] Exemplarily, in the embodiment of the present application, the deduplication screening of the fingerprint image can be to remove the fingerprint image with higher similarity in multiple frames of fingerprint images. The similarity can be represented by the number of similar feature point pairs between the fingerprint images participating in the similarity matching, taking the determination of the similarity between two frames of fingerprint images as an example, the more similar feature point pairs between the two frames of fingerprint images, the higher the similarity of the fingerprint images, or the higher the ratio of the number of similar feature point pairs to the total number of feature points, the higher the similarity of the fingerprint images. The similar feature point pairs are, for example: if the similarity between feature point A in one frame of fingerprint image and feature point B in another frame of fingerprint image is higher when similarity matching is performed on the feature points of the two frames of fingerprint images, then feature point A and feature point B are formed into a similar feature point pair. In addition, the similarity matching of the feature points of the two frames of fingerprint images can be implemented using a common similarity matching algorithm, and the embodiment of the present application does not provide a detailed introduction to the specific similarity matching algorithm used to determine the similarity of the feature points.

[0090] Optionally, when the obtained fingerprint image has clear areas and blurred areas, when judging similar feature point pairs of two frames of fingerprint images, similar feature point pairs can be determined only in the clear areas of the two frames of fingerprint images, and the blurred areas of the fingerprint images do not participate in the judgment of similar feature point pairs.

[0091] In some embodiments, a preset deduplication screening condition is set, and if the number of feature point pairs between the comparison fingerprint image and the reference fingerprint image meets the preset deduplication screening condition, the comparison fingerprint image is determined to be a valid fingerprint image. The preset deduplication screening condition includes: if the number of similar feature point pairs between the comparison fingerprint image and the reference fingerprint image is less than or equal to the maximum value of the preset judgment threshold interval, and greater than or equal to the minimum value of the preset judgment threshold interval, the comparison fingerprint image is determined to be a valid fingerprint image. If the number of similar feature point pairs between the comparison fingerprint image and the reference fingerprint image is greater than the maximum value of the judgment threshold interval, or less than the minimum value of the judgment threshold interval, the comparison fingerprint image is ignored.

[0092] It can be known that the preset de-duplication screening conditions are set on the basis of minimizing the number of similar feature point pairs while retaining a portion of similar feature point pairs so that image stitching can be performed, because too many similar feature point pairs will cause most of the two frames of fingerprint images to overlap when stitching images, so more fingerprint images are required to participate in the stitching to obtain a stitched fingerprint image that meets the rated size. If the two frames of fingerprint images have no similar feature point pairs at all, they cannot be stitched, or the stitched fingerprint image cannot guarantee the continuity of the ridge lines, which brings difficulties to the subsequent fingerprint feature matching.

[0093] In some embodiments, if multiple frames of fingerprint images are obtained by sliding the user's finger on the fingerprint sensor multiple times, the following fingerprint image deduplication screening method is adopted: the multiple frames of fingerprint images can be regarded as continuous fingerprint images. In the continuous fingerprint images, the number of similar feature point pairs of two frames of fingerprint images with adjacent acquisition times is relatively large. When deduplication screening is performed on the multiple frames of continuous fingerprint images, the first frame of fingerprint image obtained first can be selected as the reference fingerprint image according to the order of acquisition time of the multiple frames of continuous fingerprint images, and the fingerprint image to be compared with the reference fingerprint image can be selected in turn from other fingerprint images except the reference fingerprint image according to the acquisition time. The fingerprint images are matched for similarity, and the number of similar feature point pairs between the reference fingerprint image and the comparison fingerprint image is obtained. If the number of similar feature point pairs between the comparison fingerprint image of the nth frame and the reference fingerprint image meets the preset deduplication screening condition, the comparison fingerprint image of the nth frame is determined as a new reference fingerprint image, and the new reference fingerprint image is sequentially feature-compared with other fingerprint images that have not participated in the similarity matching. The above process is repeated until there are no fingerprint images that have not participated in the similarity matching in the multiple-frame fingerprint images, or the number of reference fingerprint images determined in the multiple-frame fingerprint images is greater than or equal to the preset number, and the reference fingerprint image is determined as a valid fingerprint image. The preset number is set according to the rated size of the spliced ​​fingerprint image.

[0094] In some embodiments, if multiple frames of fingerprint images are obtained by the user's finger pressing the fingerprint sensor multiple times, the following fingerprint image deduplication screening method is adopted: the multiple frames of fingerprint images can be regarded as discrete fingerprint images, that is, there may be no similar feature point pairs between two frames of fingerprint images acquired at adjacent times. When deduplication screening is performed on multiple frames of discrete fingerprint images, one frame of fingerprint image can be arbitrarily selected from the multiple frames of fingerprint images as a reference fingerprint image, and then feature comparison is performed with other frames of fingerprint images to determine the number of similar feature point pairs between the comparison fingerprint image and the reference fingerprint image. If the number of similar feature point pairs between the n-th frame of comparison fingerprint image and the reference fingerprint image meets the preset deduplication screening condition, the n-th frame of comparison fingerprint image is determined as a new reference fingerprint image, and feature comparison is performed between the new reference fingerprint image and other fingerprint images. The above process is repeated until there is no fingerprint image that meets the preset deduplication screening condition in the multiple frames of fingerprint images, or the number of reference fingerprint images determined in the multiple frames of fingerprint images is greater than or equal to the preset number of frames, and the reference fingerprint image is determined as a valid fingerprint image.

[0095] In some embodiments, if the number of valid fingerprint images is less than a preset number, the process returns to step S701, and the watch pops up a message in the setting interface prompting the user to press or slide the fingerprint sensor again, and repeats the above steps S701 to S704 until a preset number of valid fingerprint images are obtained, or the number of repetitions reaches a repetition threshold and the fingerprint registration process is terminated. The number of repetitions indicates the number of times the user's finger presses or slides on the fingerprint sensor in the fingerprint registration process. When it is greater than the repetition threshold, in order to ensure the user's usage experience, the fingerprint registration process is terminated, and a prompt message is sent to the user to clean the packaging surface of the fingerprint sensor and keep his fingers clean.

[0096] S705: The watch performs fingerprint splicing on multiple frames of valid fingerprint images to obtain a spliced ​​fingerprint image.

[0097] In some embodiments, the watch selects two frames of valid fingerprint images with similar feature point pairs from multiple frames of valid fingerprint images for fingerprint image stitching, and determines the translation parameters and rotation parameters of the other frame of valid fingerprint image based on the coordinate positions of the similar feature point pairs in the respective valid fingerprint images and taking one of the frames of valid fingerprint images as a reference, and stitches the two frames of valid fingerprint images based on the translation parameters and rotation parameters. If there are clear areas and blurred areas in the valid fingerprint image, and the size and position of the clear areas and blurred areas of the two frames of valid fingerprint images involved in the stitching are different, when the fingerprint images are stitched, the clear area of ​​one frame of the valid fingerprint image may overlap with the blurred area of ​​the other frame of the fingerprint image. In the overlapping area, if one frame of the fingerprint image is clear and the other frame of the fingerprint image is blurred, the clear fingerprint image in the overlapping area will be covered on the blurred fingerprint image. Fig.10 As shown, fingerprint images 11, 12, 13, and 14 are stitched to obtain a stitched fingerprint image. In the stitched fingerprint image, the overlapping area of ​​fingerprint image 11 and fingerprint image 12 is covered by fingerprint image 12, the overlapping area of ​​fingerprint image 12 and fingerprint image 13 is covered by fingerprint image 13, and the overlapping area of ​​fingerprint image 13 and fingerprint image 14 is covered by fingerprint image 14. It can be known that the operation of covering one frame of fingerprint image on another frame of fingerprint image is a common image processing method, for example, it can be to remove the texture features of blurred fingerprint images in the overlapping area. The embodiment of the present application does not provide a detailed introduction to the method of covering one frame of fingerprint image on another frame of fingerprint image.

[0098] In some embodiments, the overlapping area is divided into several overlapping sub-areas, and the fingerprint image superimposed on each overlapping sub-area is judged respectively. If there is a situation in a certain overlapping sub-area where one frame of fingerprint image is clear and another frame of fingerprint image is blurred, then in the overlapping sub-area, the clear fingerprint image covers the blurred fingerprint image, that is, the covering relationship of the fingerprint images participating in image stitching is determined only in the overlapping sub-area (determining which frame of fingerprint image covers and which frame of fingerprint image is covered). Exemplarily, if the first frame of fingerprint image is clear and the second frame of fingerprint image is blurred in the first sub-area of ​​the overlapping area, and the first frame of fingerprint image is blurred and the second frame of fingerprint image is clear in the second sub-area of ​​the overlapping area, then when performing fingerprint image stitching, the first frame of fingerprint image covers the second frame of fingerprint image in the first sub-area, and the second frame of fingerprint image covers the first frame of fingerprint image in the second sub-area.

[0099] S706: The watch performs a deblurring operation on the spliced ​​fingerprint image to obtain a target fingerprint image.

[0100] Exemplarily, the deblurring process is to remove the fingerprint features of the blurred area of ​​the spliced ​​fingerprint image, so that when the fingerprint feature is extracted from the target fingerprint image, the fingerprint algorithm cannot recognize any fingerprint features in the area. The deblurring process includes: setting the grayscale value of the pixel points in the blurred area of ​​the spliced ​​fingerprint image to a preset value, such as 255, that is, removing the fingerprint features of the blurred area of ​​the spliced ​​fingerprint image and turning it into a blank area, such as Fig.11 As shown, after the spliced ​​fingerprint image is deblurred, the blurred area of ​​the target fingerprint image becomes a blank area. In another example, the deblurring process can also be to set the pixel points of the blurred area of ​​the fingerprint image to grayscale values ​​of other values ​​to remove the fingerprint features, which is not limited in the embodiments of the present application.

[0101] S707: The watch extracts fingerprint features from the target fingerprint image to obtain feature points of the target fingerprint image.

[0102] Exemplarily, the watch extracts features from the target fingerprint image, including extracting feature points from the binary image. In some embodiments, in order to extract feature points from the binary image, the target fingerprint image is binarized and thinned to obtain a binary image in which the ridge line is one pixel wide. Then, a pixel template of size 3×3 is used to scan the pixels of the binary image one by one, and the crossing number (cn) is calculated. The crossing number is used to determine whether it is a feature point and the type of feature point (such as an end point or a bifurcation point). The crossing number is calculated, for example, as follows: except for the center pixel, the number of white pixels adjacent to other black pixels in the pixel template above, below, left, and right, such as Fig.12As shown, in Fig.12 In (a), there is a black pixel P1 around the black pixel P, and two white pixels P2 and P8 are adjacent to P1. The number of intersections is 2. Fig.12 In (b), there are three black pixels around the black pixel P, namely P1, P5 and P7. Among them, the white pixels adjacent to P1 are P2 and P8, the white pixels adjacent to P5 are P4 and P6, and the white pixels adjacent to P7 are P6 and P8, so the number of intersections is 6. According to the different number of intersections, it can be determined whether the pixel is a feature point. If so, the type of feature point is determined. When the number of intersections is 2 and 6, the pixel is determined to be a feature point. When the number of intersections is other, the pixel is determined to be an ordinary point. When the number of intersections is 2, the type of feature point is an end point. When the number of intersections is 6, the type of feature point is a bifurcation point. In this way, it can be determined Fig.12 The black pixel point P in (a) is a feature point, and the type of the feature point is an end point. Fig.12 The black pixel point P in (b) is a feature point, and the type of the feature point is a bifurcation point. All the feature points of the target fingerprint image are obtained by traversing all the pixel points in the binary image using the pixel template.

[0103] S708: The watch generates a registered fingerprint template according to the fingerprint feature points of the target fingerprint image.

[0104] Exemplarily, after the fingerprint feature points of the target fingerprint image are obtained, these feature points are saved in a feature point set, which may also be called a registered fingerprint template, and the registered fingerprint template is saved in a database.

[0105] The embodiment of the present application provides a fingerprint image processing method. In the fingerprint registration process, the clear area and the blurred area of ​​each frame of the fingerprint image are determined by performing image data analysis on the fingerprint image, and multiple frames of fingerprint images are spliced ​​using the clear area and the blurred area of ​​the fingerprint image. When the clear area overlaps with the blurred area, the clear area is retained. After obtaining the spliced ​​fingerprint image, the blurred area of ​​the spliced ​​fingerprint image is discarded to obtain a clear target fingerprint image. Feature extraction is performed on the clear target fingerprint image to obtain feature points with high confidence. Using the feature points with high confidence for fingerprint recognition can solve the problem of reduced security of fingerprint recognition caused by using fingerprint images with blurred areas.

[0106] In some embodiments, Fig.13As shown, in the fingerprint registration process, when the user's finger operates on the fingerprint sensor (including pressing operations or sliding operations), the watch obtains multiple frames of fingerprint images. When the watch detects that the user's finger leaves the fingerprint sensor, the watch records the user's finger operation on the fingerprint sensor once. Accordingly, the number of operations is increased by 1. The watch judges the number of operations based on the preset number. The preset number represents the maximum number of operations of the user's finger on the fingerprint sensor in the fingerprint registration process.

[0107] If the number of operations is less than the preset number, the watch determines the registration type based on the time from when the user's finger touches the fingerprint sensor to when it leaves the fingerprint sensor. If the time from when the finger touches the fingerprint sensor to when it leaves the fingerprint sensor is less than or equal to the preset time, the watch determines that the registration type is press registration. When the time from when the finger touches the fingerprint sensor to when it leaves the fingerprint sensor is greater than or equal to the preset time, the watch determines that the registration type is slide registration. The watch generates registered fingerprint templates based on press registration and slide registration respectively. If the number of registered fingerprint templates is greater than or equal to the preset number, the fingerprint registration process ends. If the number of registered fingerprint templates is less than the preset number, the watch displays a prompt message on the screen prompting the user to collect fingerprints again. When the fingerprint sensor detects that the user's finger is operating on the fingerprint sensor, it repeats the above process. If the number of operations is equal to the preset number, the fingerprint registration process ends.

[0108] A fingerprint image processing method provided in an embodiment of the present application, in the fingerprint registration process, whether the user presses the fingerprint sensor with a finger or slides the fingerprint sensor with a finger, a registered fingerprint template can be generated, and when the number of registered fingerprint templates reaches a preset number or the number of user operations reaches a preset number, the fingerprint registration process is terminated. By limiting the number of registered fingerprint templates and the number of operations, a larger fingerprint recognition area can be provided in the fingerprint recognition process while avoiding a decrease in user experience due to excessive user operations (pressing or sliding).

[0109] like Fig.14 As shown, in some implementations, a fingerprint image processing method provided in an embodiment of the present application can also be applied to a fingerprint recognition process. Taking the terminal device as a watch, the fingerprint registration process on the watch is introduced as an example. Exemplarily, the fingerprint authentication process includes S801 to S804, wherein:

[0110] S801. The watch obtains a fingerprint image.

[0111] In some implementations, after the fingerprint sensor is activated, the process of acquiring the fingerprint image in step S702 is performed, which will not be described in detail here.

[0112] S802: The watch performs image data analysis on the fingerprint image to determine the clear area and the blurred area of ​​the fingerprint image.

[0113] In some implementations, the watch performs image data analysis on the fingerprint image and executes the image data analysis process as described in step S703 above to determine the clear area and the blurred area of ​​the fingerprint image.

[0114] S803: The watch extracts fingerprint features from the fingerprint image, obtains feature points, and sets confidence levels for the feature points.

[0115] In some embodiments, the watch extracts features from the fingerprint image to obtain feature points, and sets confidence levels for the feature points based on the areas where the feature points are located. For example, if the feature points are located in a clear area, the confidence level is high, and if the feature points are located in a fuzzy area, the confidence level is low. When performing fingerprint matching, feature points with high confidence levels are matched first.

[0116] To extract features from the fingerprint image, a feature extraction process similar to the above step S708 may be performed.

[0117] S804: The watch generates an input fingerprint template based on the fingerprint feature points, and performs similarity matching between the input fingerprint template and the registered fingerprint template to obtain a fingerprint recognition result.

[0118] Exemplarily, when similarity matching is performed, feature points with high confidence are preferentially matched with feature points of the registered fingerprint template for similarity matching, and a similarity score between the input fingerprint template and the registered fingerprint template is obtained. A preset matching condition can be set, and the preset matching condition includes: if the similarity score is greater than or equal to the preset similarity score judgment threshold, the fingerprint recognition result is a successful fingerprint recognition. If the similarity score is less than the preset similarity score judgment threshold, feature matching is performed between feature points with low confidence and feature points of the registered fingerprint template. If the similarity score is greater than or equal to the preset similarity score judgment threshold, the fingerprint recognition result is a successful fingerprint recognition. If the similarity score is less than the preset similarity score judgment threshold, the fingerprint recognition result is a failed fingerprint recognition. Among them, the similarity score represents the similarity between the input fingerprint template and the registered fingerprint template. The higher the similarity score, the more similar the input fingerprint template is to the registered fingerprint template. When the similarity score is greater than or equal to the preset similarity score judgment threshold, the watch believes that it is the user's own finger that is performing fingerprint recognition at this time, and the watch performs an action corresponding to the user's operation.

[0119] Exemplarily, different weights can be set according to different confidence settings, and the weight of feature points with high confidence is greater than the weight of feature points with low confidence. The first similarity score obtained when the feature points with high confidence are matched with the feature points of the registered fingerprint template is weighted and summed with the second similarity score obtained when the feature points with low confidence are matched with the feature points of the registered fingerprint template to obtain the total similarity score.

[0120] In some embodiments, if the fingerprint recognition result is a fingerprint recognition failure, the watch displays a prompt message indicating the fingerprint recognition failure on the screen, and returns to step S802 to continue executing the above steps. If the fingerprint recognition result is a fingerprint recognition success, the fingerprint image corresponding to the input fingerprint template is spliced ​​with the fingerprint image corresponding to the registered fingerprint template. If the clear area of ​​the fingerprint image corresponding to the input fingerprint template overlaps with the blank area of ​​the fingerprint image corresponding to the registered fingerprint template, the feature points of the clear area of ​​the fingerprint image corresponding to the input fingerprint template are added to the registered fingerprint template. The specific image splicing process is shown in the above embodiment and will not be repeated here.

[0121] A fingerprint image processing method provided by an embodiment of the present application, in a fingerprint authentication process, by performing image data analysis on a fingerprint image, a clear area and a fuzzy area of ​​the fingerprint image are determined, the clear area and the fuzzy area of ​​the fingerprint image are used together to extract fingerprint features, different confidences are set for feature points in the clear area and feature points in the fuzzy area, the confidence of the feature points in the clear area is higher than the confidence of the feature points in the fuzzy area, an input fingerprint template is obtained according to the feature points, the input fingerprint template is used to perform fingerprint similarity matching with a registered fingerprint template in a database, and feature points with high confidence are preferentially selected to perform similarity matching with feature points of the registered fingerprint template according to the confidence of the feature points to obtain a fingerprint authentication result, and when the fingerprint authentication result is a successful fingerprint recognition, the fingerprint image corresponding to the input fingerprint template and the fingerprint image corresponding to the registered fingerprint template are spliced, and if the clear area of ​​the fingerprint image corresponding to the input fingerprint template overlaps with the blank area of ​​the fingerprint image corresponding to the registered fingerprint template, the feature points of the clear area of ​​the fingerprint image corresponding to the input fingerprint template are added to the registered fingerprint template. The fingerprint image processing method provided in the embodiment of the present application can process fingerprint images with blurred areas, so that it can use the existing fingerprint algorithm to extract and match the fingerprint image features, and in the case of successful matching, update the registered fingerprint template, add feature points with high confidence to the registered fingerprint template, and improve the security and reliability of fingerprint recognition.

[0122] In the embodiment of the present application, the terminal device using the curved frame design can be a wearable device (such as a smart watch, a smart bracelet), a mobile phone, a tablet computer, a handheld computer, a smart home device (such as a TV), a car machine (such as a car computer), a smart screen, a game console, and an augmented reality (AR) / virtual reality (VR) device, etc. The embodiment of the present application does not impose any special restrictions on the specific device form of the terminal device.

[0123] For example, taking the terminal device as a watch with a curved frame design, Fig.15 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application is shown.

[0124] like Fig.15 As shown, the terminal device may include a processor 410, an external memory interface 420, an internal memory 421, a power interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, a wireless communication module 460, an audio module 470, a sensor module 480, a button 490, a motor 491, an indicator 492, a display screen 494, etc., wherein the sensor module 480 may include a pressure sensor 480A, a gyroscope sensor 480B, an air pressure sensor 480C, an acceleration sensor 480D, a fingerprint sensor 480E (such as an ultrasonic fingerprint recognition module, an optical fingerprint recognition module, and a capacitive fingerprint recognition module, etc.), a touch sensor 480F, etc.

[0125] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the terminal device. In other embodiments, the terminal device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0126] The processor 410 may include one or more processing units, for example, the processor 410 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0127] The controller can be the nerve center and command center of the terminal device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0128] The processor 410 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. The memory may store instructions or data that the processor 410 has just used or cyclically used. If the processor 410 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0129] In some embodiments, the processor 410 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0130] The wireless communication function of the terminal device can be realized through the antenna 1, the wireless communication module 460, the modem processor and the baseband processor.

[0131] The terminal device implements the display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs that execute program instructions to generate or change display information.

[0132] The display screen 494 is used to display images, videos, etc. The display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device may include 1 or N display screens 494, where N is a positive integer greater than 1.

[0133] The internal memory 421 can be used to store computer executable program codes, which include instructions. The processor 410 executes various functional applications and data processing of the terminal device by running the instructions stored in the internal memory 421. The internal memory 421 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0134] The fingerprint sensor 480E is used to collect the capacitance signal when the user's finger presses the fingerprint sensor or slides on the fingerprint sensor. In the embodiment of the present application, a capacitive fingerprint recognition module (also called a capacitive fingerprint sensor) is used, but other such as an optical fingerprint recognition module (also called an optical fingerprint sensor) and an ultrasonic fingerprint recognition module (also called an ultrasonic fingerprint sensor) are also applicable, which will not be described in detail here.

[0135] Corresponding to the method in the above-mentioned embodiment, the embodiment of the present application also provides an image processing device. The device can be applied to the above-mentioned terminal device to implement the method in the above-mentioned embodiment. The function of the device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions. For example, the device includes: an acquisition module, a processing module, and an identification module. The acquisition module, the processing module, and the identification module can be used together to implement the relevant methods in the above-mentioned embodiments.

[0136] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0137] In another implementation, the unit of the device implementing each step in the above method may be configured as one or more processing elements, which may be arranged on the corresponding electronic device, and the processing element here may be an integrated circuit, for example: one or more ASICs or one or more DSPs or one or more FPGAs, or a combination of these integrated circuits. These integrated circuits may be integrated together to form a chip.

[0138] For example, the present application also provides a chip system, which can be applied to the above terminal device. The chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through a line; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuit to implement the terminal device-related method in the above method embodiment.

[0139] An embodiment of the present application also provides a computer program product, including a terminal device, such as computer instructions executed by the above-mentioned terminal device.

[0140] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0144] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: In response to a fingerprint registration operation of a user, a first fingerprint image and a second fingerprint image are acquired; the first fingerprint image includes a first area, the second fingerprint image includes a second area, and the images in the first area and the second area are blurred; splicing the first fingerprint image and the second fingerprint image according to the fingerprint shape to obtain a spliced ​​fingerprint image; Removing blurred images from the stitched fingerprint images to obtain a third fingerprint image; A registered fingerprint template is generated according to the third fingerprint image.

2. The method according to claim 1, characterized in that The step of splicing the first fingerprint image and the second fingerprint image according to the fingerprint shape comprises: Respectively acquiring feature points in the first fingerprint image and the second fingerprint image, wherein the feature points are used to characterize the fingerprint shape; The first fingerprint image and the second fingerprint image are spliced ​​together so that similar feature points in the first fingerprint image and the second fingerprint image are spliced ​​together.

3. The method according to claim 2, characterized in that The first fingerprint image includes a third area, and the image in the third area is clear; After the first fingerprint image and the second fingerprint image are spliced, the first sub-region in the third region overlaps with the second sub-region in the second region, and the region in the spliced ​​fingerprint image where the third region overlaps with the second region uses the image of the first sub-region.

4. The method according to any of claims 1 to 3, characterized in that: The removing of the blurred image in the spliced ​​fingerprint image comprises: The grayscale value of the pixel point in the fuzzy area of ​​the spliced ​​fingerprint image is set to a preset value.

5. The method according to any one of claims 1 to 4, characterized in that: The images are blurred including: The grayscale level of the image is less than or equal to the first judgment threshold; or, The light-dark contrast of the image is less than the second judgment threshold.

6. The method according to any one of claims 1 to 5, characterized in that: The acquiring of the first fingerprint image and the second fingerprint image comprises: The first fingerprint image and the second fingerprint image are acquired through the curved surface packaged fingerprint sensor.

7. The method according to any one of claims 1 to 6, characterized in that: In response to the fingerprint registration operation of the user, acquiring the first fingerprint image and the second fingerprint image includes: In response to a fingerprint registration operation by a user, acquiring multiple frames of fingerprint images; The first fingerprint image and the second fingerprint image are determined according to the multiple frames of fingerprint images; the first fingerprint image and the second fingerprint image are fingerprint images whose similar feature point pairs in the multiple frames of fingerprint images meet the first screening condition.

8. The method according to claim 7, characterized in that The similar feature point pairs satisfying the first screening condition include: The number of similar feature point pairs is less than or equal to a maximum value of the judgment threshold interval, and greater than or equal to a preset minimum value of the judgment threshold interval.

9. The method according to any one of claims 1 to 8, characterized in that: The fingerprint registration operation includes a pressing operation or a sliding operation on the fingerprint collection area.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: In response to a fingerprint authentication operation of the user, a fourth fingerprint image is acquired; the fourth fingerprint image includes a fourth area and a fifth area, the image of the fourth area is clear, and the image of the fifth area is blurred; Performing similarity matching between the feature points in the fourth area and the registered fingerprint template to obtain a first similarity score; If the first similarity score meets the preset matching condition, it is determined that the fingerprint authentication is successful.

11. The method according to claim 10, characterized in that The method further comprises: If the first similarity score does not meet the preset matching condition, performing similarity matching between the feature points in the fifth area and the registered fingerprint template to obtain a second similarity score; A target similarity score is obtained by weighted summing the first similarity score and the second similarity score; wherein the weight of the first similarity score is greater than the weight of the second similarity score; If the target similarity score meets the preset matching condition, it is determined that the fingerprint authentication is successful.

12. The method according to claim 10 or 11, characterized in that: After the fingerprint authentication is successful, the method further includes: The registered fingerprint template is updated according to the feature points in the fourth area.

13. A terminal device, characterized in that: The terminal device comprises: a processor and a memory, the processor being coupled to the memory; the memory being used to store computer program code; the computer program code comprising computer instructions, and when the processor executes the above-mentioned computer instructions, the terminal device executes the method described in any one of claims 1-12.

14. A computer-readable storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 12.

15. A computer program product, characterized in that When the computer program product is executed on a terminal device, the terminal device executes the method described in any one of claims 1 to 12.

Images