A fingerprint image processing method and terminal device

Abstract

This application discloses a fingerprint image processing method and a terminal device, relating to the field of terminal technology. The method includes: the terminal device, in response to a user's fingerprint registration operation, acquiring a first fingerprint image and a second fingerprint image; stitching the first and second fingerprint images together using fingerprint shapes to obtain a stitched fingerprint image; removing blurred images from the stitched fingerprint image to obtain a third fingerprint image; and generating a registration fingerprint template based on the third fingerprint image. Because the blurred images in the third fingerprint image are removed, the confidence level of the registration fingerprint template generated from the third fingerprint image is high, solving the problem of reduced security in fingerprint recognition caused by using partially blurred fingerprint images.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a fingerprint image processing method and terminal device. Background Technology

[0002] Fingerprint recognition technology is a type of biometric technology. Due to the uniqueness, immutability, and ease of collection of fingerprints, fingerprint recognition technology is widely used in terminal devices.

[0003] Fingerprint recognition typically involves two processes: registration and authentication. For example, for a terminal device to successfully perform fingerprint recognition, the user must first register their fingerprint on the device; that is, the user pre-enrolls their fingerprint on the terminal device, and this pre-enrolled fingerprint can be called the registration fingerprint. When performing identity authentication, the terminal device compares the user's input fingerprint for authentication with the registration fingerprint to determine the identity of the user who entered the fingerprint.

[0004] Due to limitations such as the setting and capabilities of the fingerprint sensor on the terminal device, the fingerprint image captured by the fingerprint sensor may be incomplete or unclear, resulting in reduced security of fingerprint recognition. Summary of the Invention

[0005] This application provides a fingerprint image processing method and a terminal device. It can remove blurry images from fingerprint images, improve the confidence of registered fingerprint templates, and solve the problem of reduced security in fingerprint recognition caused by using partially blurred fingerprint images.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] Firstly, a fingerprint image processing method is provided, the method comprising:

[0008] In response to the user's fingerprint registration operation, the terminal device acquires a first fingerprint image and a second fingerprint image. The images in the first region of the first fingerprint image and the second region of the second fingerprint image are blurred. The images are stitched together based on the fingerprint shapes of the first and second fingerprint images to obtain a stitched fingerprint image. Then, the blurred images in the stitched fingerprint image are removed to obtain a third fingerprint image. The third fingerprint image is used to generate a registration fingerprint template.

[0009] In this embodiment, since the blurred image in the third fingerprint image is removed, the confidence level of the registered fingerprint template generated based on the third fingerprint image is high, thus solving the problem of reduced security of fingerprint recognition caused by using fingerprint images with blurred areas.

[0010] In one possible implementation, the first region of the first fingerprint image (also referred to as the blurred region of the first fingerprint image) and the second region of the second fingerprint image (also referred to as the blurred region of the second fingerprint image) are blurred in that the gray level of the image is less than or equal to the first judgment threshold, or the brightness contrast of the image is less than the second judgment threshold.

[0011] In one possible implementation, the first judgment threshold includes a gray level judgment threshold, and the second judgment threshold includes a brightness judgment threshold.

[0012] In one possible implementation, the first fingerprint image and the second fingerprint image are acquired by a fingerprint sensor installed in the terminal device. The fingerprint sensor adopts a curved package, with the middle area of ​​the package layer being farther from the fingerprint chip of the fingerprint sensor and the two end areas being closer to the fingerprint chip of the fingerprint sensor. This makes the first area of ​​the first fingerprint image and the second area of ​​the second fingerprint image acquired by the terminal device blurry.

[0013] In this embodiment, when there are blurred areas in the obtained first and second fingerprint images, a third fingerprint image is obtained by removing the blurred image from the stitched fingerprint image. Then, a registration fingerprint template is generated using the third fingerprint image. This allows the method to be applied to terminal devices with fingerprint sensors encapsulated with curved surfaces, enriching the application scenarios of fingerprint recognition.

[0014] In one possible implementation, the first fingerprint image and the second fingerprint image are valid fingerprint images obtained by deduplication and filtering multiple fingerprint images.

[0015] In one possible implementation, the terminal device responds to the user's fingerprint registration operation by acquiring multiple frames of fingerprint images and determining fingerprint images in which similar feature point pairs satisfy a preset first screening condition.

[0016] In one possible implementation, satisfying the preset first screening condition includes: the number of similar feature point pairs between the first fingerprint image and the second fingerprint image is less than or equal to the maximum value of a preset judgment threshold interval, and greater than or equal to the minimum value of the preset judgment threshold interval.

[0017] In one possible implementation, the number of first and second fingerprint images determined based on multiple fingerprint images can be multiple frames. The first fingerprint image can be a reference fingerprint image among the multiple fingerprint images, and the second fingerprint image is a fingerprint image that meets a preset first screening condition among the multiple fingerprint images, with the first fingerprint image as the reference fingerprint image. Both the first and second fingerprint images are valid fingerprint images.

[0018] In one possible implementation, when stitching the first fingerprint image and the second fingerprint image according to the fingerprint shape, feature points in the first fingerprint image and the second fingerprint image are obtained respectively, and the similar feature points of the first fingerprint image and the second fingerprint image are used for stitching.

[0019] In this embodiment, feature points are obtained from the first fingerprint image and the second fingerprint image. The first fingerprint image and the second fingerprint image can be aligned based on similar feature points, which is beneficial for fingerprint image stitching.

[0020] In one possible implementation, the first fingerprint image includes a third region where the image is clear. After the terminal device stitches the first fingerprint image and the second fingerprint image together, a first sub-region in the third region overlaps with a second sub-region in the second region. In the stitched fingerprint image, the area where the third region overlaps with the second region uses the image of the first sub-region.

[0021] In this embodiment, the image in the third region of the first fingerprint image is clear. When the first sub-region of the third region overlaps with the second sub-region of the second region, the first sub-region is used in the overlapping region. Since the first sub-region is part of the third region, the image is clear. This makes the area of ​​the clear region of the stitched fingerprint image larger, which is beneficial for subsequent processing.

[0022] In one possible implementation, the first fingerprint image may be the fingerprint image in which the third region as a whole, or the first sub-region of the third region overlaps with the second fingerprint image in the two fingerprint images involved in the stitching, that is, the clear region overlaps with the second fingerprint image in whole or in part. The second fingerprint image may be the fingerprint image in which the second region as a whole, or the second sub-region of the second region overlaps with the first fingerprint image in the two fingerprint images involved in the stitching, that is, the blurred region overlaps with the first fingerprint image in whole or in part.

[0023] In one possible implementation, the fingerprint registration operation includes a pressing or sliding operation in the fingerprint acquisition area.

[0024] In one possible implementation, the fingerprint acquisition area includes the packaged surface area of ​​the fingerprint sensor.

[0025] In one possible implementation, when removing blurry images from the stitched fingerprint image, the terminal device sets the grayscale value of the pixels in the blurry area of ​​the stitched fingerprint image to a preset value.

[0026] In this way, by setting the grayscale value of the pixels in the blurred area of ​​the stitched fingerprint image to a preset value, the influence of the blurred area in the stitched fingerprint image on fingerprint recognition can be avoided.

[0027] In one possible implementation, the preset value could be 255.

[0028] In one possible implementation, the terminal device responds to the user's fingerprint authentication operation by acquiring a fourth fingerprint image. The fourth region of the fourth fingerprint image is clear, while the fifth region is blurry. The terminal device performs similarity matching between the feature points of the fourth region of the fourth fingerprint image and the registered fingerprint template to obtain a first similarity score. If the first similarity score meets the preset matching conditions, the fingerprint authentication is determined to be successful.

[0029] In this embodiment, feature points of the fourth region of the fourth fingerprint image are preferentially used to perform similarity matching with the registered fingerprint template. That is, feature points of clear regions are preferentially used to perform similarity matching with the registered fingerprint template. When the first similarity score meets the preset matching conditions, the fingerprint verification is determined to be successful. The feature points of the fifth region of the fourth fingerprint image are no longer used to perform similarity matching with the registered fingerprint template. This can ensure the security of fingerprint recognition while taking into account the efficiency of fingerprint recognition.

[0030] In one possible implementation, if the first similarity score does not meet the preset matching conditions, the terminal device performs similarity matching between the feature points of the fifth region of the fourth fingerprint image and the registered fingerprint template to obtain a second similarity score. The first similarity score and the second similarity score are weighted and summed to obtain a target similarity score, where the weight of the first similarity score is greater than the weight of the second similarity score. If the target similarity score does not meet the preset matching conditions, fingerprint authentication is determined to have failed; if the target similarity score meets the preset matching conditions, fingerprint authentication is determined to have succeeded.

[0031] In this embodiment, the target similarity score is obtained by weighted summation of the first similarity score and the second similarity score. By setting different weights for the first similarity score and the second similarity score, and with the weight of the first similarity score being higher than that of the second similarity score, the influence of the second similarity score on the target similarity score can be reduced. This reduces the impact on fingerprint recognition security when feature points in the blurred area of ​​the fourth fingerprint image are matched with the registered fingerprint template.

[0032] In one possible implementation, after successful fingerprint authentication, the terminal device updates the registered fingerprint template based on the feature points of the fourth region of the fourth fingerprint image.

[0033] In one possible implementation, the fourth fingerprint image includes the input fingerprint image.

[0034] In a second aspect, a terminal device is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, and when the terminal device is running, the processor executes the computer execution instructions stored in the memory to cause the terminal device to perform the method as described in any one of the first aspects above.

[0035] Thirdly, a computer-readable storage medium is provided that stores instructions which, when executed on a computer, enable the computer to perform the method described in any one of the first aspects.

[0036] Fourthly, a computer program product containing instructions is provided, which, when run on a computer, enables the computer to perform the method described in any one of the first aspects above.

[0037] The technical effects of any of the design methods in the second to fourth aspects can be found in the technical effects of different design methods in the first aspect, and will not be repeated here. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of a fingerprint sensor;

[0039] Figure 2 A schematic diagram of a fingerprint image to which the method provided in the embodiments of this application is applicable;

[0040] Figure 3 This is a schematic diagram of a fingerprint sensor using a curved surface encapsulation.

[0041] Figure 4 A schematic diagram of a fingerprint image applicable to the method provided in the embodiments of this application;

[0042] Figure 5 A schematic diagram of a fingerprint sensor structure installed on a watch;

[0043] Figure 6 A schematic diagram of a fingerprint sensor structure disposed on a watch to which a method provided in an embodiment of this application is applicable;

[0044] Figure 7 This is a flowchart illustrating a method applicable to one embodiment of the method provided in this application.

[0045] Figure 8 This is a schematic diagram illustrating a scenario to which the method provided in one embodiment of this application is applicable;

[0046] Figure 9 A schematic diagram of a fingerprint image to which a method provided in an embodiment of this application is applicable;

[0047] Figure 10A schematic diagram of a stitched fingerprint image to which a method provided in an embodiment of this application applies;

[0048] Figure 11 This is a schematic diagram of a target fingerprint image to which a method provided in an embodiment of this application is applicable;

[0049] Figure 12 A schematic diagram illustrating fingerprint feature extraction applicable to a method provided in an embodiment of this application;

[0050] Figure 13 A flowchart illustrating the method applicable to yet another embodiment of the method provided in this application;

[0051] Figure 14 A flowchart illustrating the method applicable to yet another embodiment of the method provided in this application;

[0052] Figure 15 This is a schematic diagram of the structural composition of a terminal device provided in an embodiment of this application. Detailed Implementation

[0053] In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0054] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0055] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0056] Fingerprint recognition technology is a type of biometric technology. It involves capturing fingerprint images, obtaining their feature points, and then matching an input fingerprint template generated based on these feature points with a saved registered fingerprint template to obtain the fingerprint recognition result. Due to the uniqueness, immutability, and ease of acquisition inherent in fingerprints, fingerprint recognition technology is widely used in identity authentication scenarios such as unlocking, payment, and login.

[0057] With the development of smart terminal devices, more and more terminal devices are equipped with fingerprint recognition functions. A complete fingerprint recognition function includes a fingerprint registration process and a fingerprint authentication process. The fingerprint registration process includes: fingerprint image acquisition, fingerprint image processing, fingerprint feature extraction, and saving the registered fingerprint template, etc.; the fingerprint authentication process includes: fingerprint image acquisition, fingerprint image processing, fingerprint feature extraction, and matching with the registered fingerprint template, etc.

[0058] The terminal device is equipped with a fingerprint sensor to collect the user's fingerprint image. After obtaining the fingerprint image, the terminal device uses fingerprint algorithms such as image processing, feature extraction, and feature matching to perform fingerprint image stitching, fingerprint feature extraction, and fingerprint feature matching (also known as similarity matching) to obtain the fingerprint recognition result. If the recognition result is successful, the terminal device can respond to the user's operation and execute the corresponding action. For example, if the user's operation is to unlock, after the fingerprint recognition result is successful, the terminal device will unlock the screen and enter the system operation interface. Another example is if the user's operation is to make a payment, after the fingerprint recognition result is successful, the terminal device will execute the payment action.

[0059] Fingerprint sensors are categorized based on their recognition principles into capacitive fingerprint sensors, optical fingerprint sensors, and ultrasonic fingerprint sensors. Optical and ultrasonic fingerprint sensors are typically located under the screen of the terminal device, while capacitive fingerprint sensors are usually located in non-screen areas, such as the front cover, side bezel, and back cover. Capacitive fingerprint sensors located on the side bezel are limited by the size of the side bezel and are typically small-sized sliding capacitive fingerprint sensors. This application uses a small-sized sliding capacitive fingerprint sensor as an example for illustration. For ease of description, unless otherwise specified, all fingerprint sensors described below refer to small-sized sliding capacitive fingerprint sensors.

[0060] For example, the structure of a fingerprint sensor is as follows: Figure 1 As shown, the system includes a fingerprint chip, a substrate, an encapsulation material layer, a coating layer, and electrical connectors. The encapsulation material layer covers the fingerprint chip, protecting both the fingerprint chip and the electrical connectors. Typically, the surface of the encapsulation material layer is planar. A coating layer, also planar, is placed above the encapsulation material layer to further protect the fingerprint sensor. The coating layer is in direct contact with the finger. The fingerprint chip collects capacitance signals through the encapsulation material layer and the coating layer. The substrate is soldered to the fingerprint chip via electrical connectors, transmitting the capacitance signals collected by the fingerprint chip to other electronic modules on the substrate, or to other electronic modules electrically connected to the substrate for processing. These other electronic modules include, but are not limited to, processing modules or storage modules.

[0061] It is known that fingerprint chips have limited penetration capabilities. If the thickness of the encapsulation material layer and coating layer (hereinafter referred to as the encapsulation layer) exceeds the rated thickness corresponding to the penetration capability of the fingerprint chip, it will affect the imaging quality of the fingerprint image. When the thickness of the encapsulation layer does not exceed the rated thickness, the clarity of the fingerprint image can meet the requirements of the fingerprint algorithm. A clear fingerprint image helps the fingerprint algorithm identify more fingerprint feature points with higher confidence, which improves fingerprint security and fingerprint recognition efficiency. Fingerprint feature points include: the end points of ridges, bifurcation points, etc. Figure 2 The image shows the end point and bifurcation point of the ridge line.

[0062] The imaging quality of a fingerprint image is related to the penetration capability of the fingerprint chip and the thickness of the encapsulation layer. For example, if the thickness of the encapsulation layer of the fingerprint sensor exceeds the penetration capability of the fingerprint chip, the overall quality of the fingerprint image will be poor, the image will be blurry, and the features of the fingerprint points will be indistinct. If the thickness of the encapsulation layer of the fingerprint sensor is uneven, the fingerprint image will show a situation where some areas are clear and some areas are blurry, such as... Figure 3 As shown in (a), a fingerprint sensor with curved surface encapsulation includes: a 3D steel reinforcement (also known as a structural support), a substrate, a fingerprint chip, and an encapsulation layer, as shown in (a). Figure 3 As shown in (b), the fingerprint sensor is horizontally represented. The thickness of the encapsulation layer in the middle region is d1, and the thickness of the encapsulation layer in the edge region is d2. d2 is less than d1, and d1 is greater than the rated thickness d3 that the fingerprint chip can penetrate. The fingerprint image obtained by the terminal device based on the capacitance signal detected by the fingerprint sensor with curved surface encapsulation is as follows: Figure 4 As shown, the fingerprint image is clear at the top and bottom, but blurry in the middle.

[0063] In some implementations, some terminal devices adopt a rectangular frame design to allow for the placement of a planar-encapsulated fingerprint sensor on the device frame. These terminal devices include, for example, mobile phones, watches, and tablets.

[0064] In other implementations, some terminal devices employ a curved bezel design. If a planar fingerprint sensor is placed on a terminal device with a curved bezel, while ensuring the central protrusion of the bezel is at a normal height sufficient for the fingerprint sensor to collect capacitive signals, the two side protrusions will be significantly higher than the bezel, making the overall appearance of the terminal device less aesthetically pleasing. For example, terminal devices typically employing a curved bezel design include... Figure 5 In the smartwatch shown, the vertical distance between the middle area of ​​the fingerprint sensor's planar package and the fingerprint chip is h1, and the vertical distance between the two side areas and the fingerprint chip is h2, where h2 equals h1. This means that the vertical distance between each point on the fingerprint sensor's package surface (coating layer) and the fingerprint chip is equal, which can produce a uniform and clear fingerprint image. However, because the watch's bezel is curved, the two sides of the fingerprint sensor's surface package protrude too much from the device's bezel, resulting in an unsightly appearance.

[0065] However, if a curved-edge fingerprint sensor is installed on a terminal device with a curved frame design, such as... Figure 6In the smartwatch shown, the vertical distance between the central area of ​​the curved surface encapsulation of the fingerprint sensor and the fingerprint chip is h3, while the vertical distance between the two side areas and the fingerprint chip is h4. Since h3 is greater than h4, the thickness of the fingerprint sensor's encapsulation layer relative to the fingerprint chip is not uniform; the central area is thicker, and the side areas are thinner. This results in a fingerprint image that appears as shown... Figure 4 The image shown is a fingerprint image with a blurred center and clear sides. When extracting features from a fingerprint image with blurred areas, the feature points in the blurred areas have low confidence. Using these low-confidence feature points as fingerprint templates will reduce the security of fingerprint recognition in the fingerprint authentication process.

[0066] This application provides a fingerprint image processing method applicable to terminal devices equipped with a curved fingerprint sensor. When a user's finger presses the fingerprint sensor (also known as a pressing operation) or slides on the fingerprint sensor (also known as a sliding operation), the fingerprint sensor acquires multiple sets of capacitance signals in multiple scanning cycles. The terminal device obtains multiple frames of fingerprint images based on these capacitance signals. The terminal device performs image data analysis on these multiple frames to determine the clear and blurred regions of each frame. The clear and blurred regions of the fingerprint images are then stitched together. When clear and blurred regions overlap, the clear region is retained in the overlapping area. After obtaining the stitched fingerprint image, the blurred regions are discarded, resulting in a clear target fingerprint image. Since the target fingerprint image does not have blurred regions, the fingerprint algorithm extracts features from the target fingerprint image, obtaining feature points with high confidence. Using these high-confidence feature points as a registration fingerprint template can solve the problem of reduced fingerprint recognition security caused by using partially blurred fingerprint images.

[0067] Furthermore, the fingerprint image processing method provided in this application embodiment can be applied to the fingerprint registration process. After obtaining the target fingerprint image, the existing fingerprint algorithm can be used to extract features from the target fingerprint image and save the registration fingerprint template, etc., without the need for additional changes to the fingerprint algorithm, and it has better applicability.

[0068] Furthermore, the fingerprint image processing method provided in this application embodiment can process fingerprint images collected by a fingerprint sensor with curved surface packaging at the software level without changing the internal structure of the fingerprint sensor, especially the shape of the fingerprint chip and substrate, without increasing hardware costs, and has better economic efficiency.

[0069] In some implementations, the fingerprint image processing method provided in this application can be applied to the fingerprint registration process, such as... Figure 7As shown, taking a watch as the terminal device, the fingerprint registration process on the watch is described as an example. For instance, the fingerprint registration process includes S701 to S708, wherein:

[0070] S701. In response to receiving the user's fingerprint registration operation, the watch initiates fingerprint registration.

[0071] The user's fingerprint registration process includes, for example, triggering the "Register Fingerprint" control on the watch's settings page; or the user entering a voice command to register their fingerprint through the watch's voice assistant.

[0072] In some implementations, after detecting a user's fingerprint registration operation, the watch can activate the fingerprint sensor, which is in a dormant state, and can also display auxiliary prompts for fingerprint registration on the watch's settings page. These auxiliary prompts are used to guide the user to press or slide their finger to touch the fingerprint sensor, prompt the user to lift their finger to prepare to press or slide again, and prompt the user to complete the fingerprint registration.

[0073] S702, the watch captures multiple frames of fingerprint images.

[0074] In some implementations, the fingerprint sensor periodically scans the package surface to acquire capacitance signals, collecting capacitance signals generated when a user's finger presses or slides across the sensor. The fingerprint sensor's scanning cycle is very fast; multiple scanning cycles occur from the time the user's finger presses the sensor until the finger is removed, or when the finger completes a single slide. In each scanning cycle, the fingerprint sensor acquires a set of capacitance signals. For example, the user's finger sliding across the fingerprint sensor includes, for instance,... Figure 8 (a) to Figure 8 As shown in (b), sliding from bottom to top can also be as follows: Figure 8 (c) to Figure 8 (d) Slides from left to right. Alternatively, it can slide from the lower left to the upper right, or in other directions; this application does not impose any restrictions on the sliding direction.

[0075] In some implementations, the watch acquires the capacitive signal collected by the fingerprint sensor, performs analog-to-digital converter (ADC) and digital signal processor (DSP) operations on the capacitive signal to obtain a fingerprint image. Each set of capacitive signals yields one frame of fingerprint image. From the moment the user's finger presses the fingerprint sensor until the finger is removed, the watch acquires multiple frames of fingerprint images based on multiple sets of signals. These fingerprint images can be Raw images, which are images obtained directly from digital signal conversion without calibration processing.

[0076] For example, a fingerprint image can also be obtained by preprocessing a raw image. For instance, a preprocessed fingerprint image can be obtained by preprocessing a raw image with preset calibration (Base) data. Preprocessing a raw image using Base data includes, but is not limited to, image denoising, binarization, Gaussian filtering, and moiré removal.

[0077] S703: The watch performs image data analysis on multiple fingerprint images to determine the clear and blurry areas of each fingerprint image.

[0078] In some implementations, the watch can determine the clear and blurry areas of a fingerprint image based on changes in its grayscale levels (hereinafter referred to as grayscale). The grayscale represents the different brightness levels of the fingerprint image from darkest to brightest. More grayscale levels indicate a clearer fingerprint image. For example, an 8-bit fingerprint image has a maximum of 28 brightness levels, i.e., 0 to 255 brightness levels. The number of grayscale levels includes: 256, 128, 64, 32, 16, 8, 4, and 2. A higher number of grayscale levels (larger numbers) indicates a clearer fingerprint image and more obvious image features; a lower number of grayscale levels (smaller numbers) indicates a blurrier fingerprint image and less distinct image features. In this embodiment, the fingerprint image includes clear and blurry areas. Changes in grayscale indicate a change in the clarity of the fingerprint image. Grayscale changes include an increase in the number of grayscale levels and a decrease in the number of grayscale levels. An increase in the number of grayscale levels indicates that the fingerprint image has changed from blurry to clear, and a decrease in the number of grayscale levels indicates that the fingerprint image has changed from clear to blurry.

[0079] For example, such as Figure 9As shown, in a fingerprint image, the watch analyzes the grayscale changes to determine the clear and blurry regions. Region L1 is the transition area where the fingerprint image changes from clear to blurry, and the grayscale number continuously decreases in Region L1. Region L2 is the transition area where the fingerprint image changes from blurry to clear, and the grayscale number continuously increases in Region L2. In some specific implementations, a grayscale number judgment threshold can be set. When the grayscale number of the fingerprint image is less than or equal to the grayscale number judgment threshold, it indicates that the fingerprint features in that region cannot be accurately extracted from the fingerprint image, or the confidence of the extracted feature points is low. In one example, regions L1 and L2 have a certain pixel width, which is greater than 1 pixel. The watch divides the fingerprint image into clear and blurry regions based on the pixel positions of regions L1 and L2 in the fingerprint image and the grayscale number judgment threshold. As can be seen, taking the L1 region of a fingerprint image as an example of the transition from clear to blurry, if the grayscale of the fingerprint image is continuously decreasing, and at a certain point the grayscale level is less than or equal to the grayscale level judgment threshold, then the boundary between clear and blurry lies within the L1 region. Similarly, in the L2 region, if the grayscale of the fingerprint image is continuously increasing, and at a certain point the grayscale level is greater than the grayscale level judgment threshold, then the boundary between blurry and clear lies within the L2 region.

[0080] In another example, the fingerprint image can be divided into several regions, each containing a certain number of pixels. The watch determines the gray level of each region. If the gray level of a region is less than or equal to a preset gray level judgment threshold, the region is determined to be a blurry region. If the gray level of a region is greater than the preset gray level judgment threshold, the region is determined to be a clear region.

[0081] In some implementations, the watch can also determine the clear and blurry areas of a fingerprint image based on its brightness contrast. Since a fingerprint image is a grayscale image, pixels in the ridges have lower grayscale values, forming the dark areas of the fingerprint image, while pixels in the valleys have higher grayscale values, forming the bright areas. Brightness contrast represents the difference between the grayscale values ​​of pixels in the bright areas and those in the dark areas. The smaller this difference, the more blurred the boundaries between ridges and valleys in the fingerprint image; the larger the difference, the clearer the boundaries. For example, the clear and blurry areas of a fingerprint image can be determined based on the brightness contrast of pixels in each region. When the brightness contrast of a region is less than or equal to a preset brightness / darkness judgment threshold, that region is determined to be blurry; conversely, if the brightness contrast is greater than the preset brightness / darkness judgment threshold, that region is determined to be clear. The preset brightness / darkness judgment threshold can be set based on whether the fingerprint algorithm can accurately extract feature points from the fingerprint image. In blurry areas, the fingerprint algorithm has difficulty extracting feature points, and the confidence level of the extracted feature points is low.

[0082] It is known that not all fingerprint images exhibit the same characteristics. Figure 9 The fingerprint image shown is blurry in the middle and clear at both ends. In some embodiments, the fingerprint image may also show clear at the top and blurry at the bottom, or clear at the left and blurry at the right. All of these can be filtered using the method of the embodiments of this application. The embodiments of this application will not be described in detail here.

[0083] In some embodiments, after determining the clear and blurry areas of the fingerprint images, step S705 is performed to filter these fingerprint images.

[0084] In some embodiments, after determining the clear and blurry areas of a fingerprint image, multiple fingerprint images can be screened for image quality based on the ratio of the clear area to the total area to obtain multiple candidate fingerprint images.

[0085] For example, image quality filtering conditions can be set. If the ratio of the clear area to the total area of ​​the fingerprint image meets the image quality filtering conditions, the fingerprint image is determined as a candidate fingerprint image; otherwise, the fingerprint image is ignored. After comparing all fingerprint images, multiple candidate fingerprint images are obtained. Step S704 filters based on the multiple candidate fingerprint images.

[0086] In some embodiments, the image quality screening condition includes a quality judgment threshold. When the ratio of the clear area to the total area of ​​the fingerprint image is greater than this quality judgment threshold, the fingerprint image is determined as a candidate fingerprint image; otherwise, the fingerprint image is ignored. For example, in an application scenario according to the embodiments of this application, when the fingerprint image obtained by a fingerprint sensor with curved surface packaging includes a portion of blurred areas, if the image quality screening threshold is set too low, for example, 10%, the selected fingerprint images will have fewer clear areas and fewer high-confidence feature points. To obtain a sufficient number of high-confidence feature points, more fingerprint images are needed for stitching, resulting in low efficiency. If the image quality screening threshold is set too high, for example, 90%, the number of fingerprint images that meet the conditions will be small. This may lead to an insufficient number of fingerprint images that meet the conditions, requiring the acquisition of more fingerprint images. It is understood that the quality judgment threshold can be set by comprehensively considering actual needs, the penetration capability of the fingerprint chip, the packaging thickness, algorithm requirements, etc. The embodiments of this application do not impose any limitations on specific values.

[0087] S704: The watch performs deduplication and filtering on multiple fingerprint images to obtain multiple valid fingerprint images.

[0088] In some implementations, when the watch performs deduplication screening on multiple fingerprint images, if step S703 above performs image quality screening on the multiple fingerprint images to obtain multiple candidate fingerprint images, then the watch performs deduplication screening on the multiple candidate fingerprint images. If step S703 above does not perform image quality screening on the multiple fingerprint images, then the watch performs deduplication screening on the multiple fingerprint images with identified clear and blurry areas. Generally, when the user's finger is in good contact with the fingerprint sensor, and the surface of the finger and the packaging surface of the fingerprint sensor remain clean, the ratio of the clear area to the total area of ​​the fingerprint image is between 50% and 60%. The process of deduplication screening of multiple fingerprint images by the watch in the following example is also applicable to the process of deduplication screening of multiple candidate fingerprint images by the watch, and will not be described in detail in this application embodiment.

[0089] For example, in this embodiment, deduplication of fingerprint images can be achieved by removing fingerprint images with high similarity from multiple fingerprint images. Similarity can be represented by the number of similar feature point pairs between the fingerprint images participating in similarity matching. Taking the determination of the similarity between two fingerprint images as an example, the more similar feature point pairs between two fingerprint images, the higher the similarity of the fingerprint images. Alternatively, the higher the ratio of the number of similar feature point pairs to the total number of feature points, the higher the similarity of the fingerprint images. For example, if feature point A in one fingerprint image has a high similarity to feature point B in another fingerprint image when performing similarity matching on the feature points of two fingerprint images, then feature point A and feature point B are considered a similar feature point pair. Furthermore, similarity matching on the feature points of two fingerprint images can be implemented using common similarity matching algorithms. This embodiment does not provide a detailed description of which specific similarity matching algorithm is used to determine the similarity of feature points.

[0090] Optionally, when the obtained fingerprint image has clear and blurry areas, when judging similar feature point pairs between two fingerprint images, similar feature point pairs can be determined only in the clear areas of the two fingerprint images, and the blurry areas of the fingerprint images are not involved in the judgment of similar feature point pairs.

[0091] In some implementations, a preset deduplication filtering condition is set. If the number of feature point pairs between the compared fingerprint image and the reference fingerprint image meets the preset deduplication filtering condition, the compared fingerprint image is determined to be a valid fingerprint image. The preset deduplication filtering condition includes: if the number of similar feature point pairs between the compared fingerprint image and the reference fingerprint image is less than or equal to the maximum value of a preset judgment threshold interval, and greater than or equal to the minimum value of a preset judgment threshold interval, then the compared fingerprint image is determined to be a valid fingerprint image. If the number of similar feature point pairs between the compared fingerprint image and the reference fingerprint image is greater than the maximum value of the judgment threshold interval, or less than the minimum value of the judgment threshold interval, then the compared fingerprint image is ignored.

[0092] It is known that the preset deduplication filtering conditions are set to minimize the number of similar feature point pairs while retaining some similar feature point pairs to enable image stitching. This is because too many similar feature point pairs will cause significant overlap between two fingerprint images during stitching, requiring more fingerprint images to be stitched together to obtain a stitched fingerprint image that meets the specified size. If two fingerprint images have no similar feature point pairs at all, stitching is impossible, or the stitched fingerprint image cannot guarantee the continuity of the ridges, making subsequent fingerprint feature matching difficult.

[0093] In some implementations, if multiple fingerprint images are acquired by the user's finger repeatedly sliding across the fingerprint sensor, the following fingerprint image deduplication method is adopted: The multiple fingerprint images can be considered as consecutive fingerprint images. In consecutive fingerprint images, the number of similar feature point pairs between two fingerprint images acquired at adjacent times is relatively large. When deduplicating multiple consecutive fingerprint images, the first fingerprint image acquired earlier can be selected as the reference fingerprint image based on the chronological order of their acquisition times. Then, fingerprint images are sequentially selected from the other fingerprint images (excluding the reference fingerprint image) for comparison according to their acquisition times. Fingerprint images undergo similarity matching to determine the number of similar feature point pairs between the reference and comparison fingerprint images. If the number of similar feature point pairs between the comparison and reference fingerprint images in the nth frame meets a preset deduplication criteria, the nth frame comparison fingerprint image is designated as the new reference fingerprint image. This new reference fingerprint image is then compared sequentially with other fingerprint images that have not participated in similarity matching. This process is repeated until no fingerprint images in the multiple frames have not participated in similarity matching, or the number of reference fingerprint images in the multiple frames is greater than or equal to a preset number. At this point, the reference fingerprint image is determined as a valid fingerprint image. The preset number is set based on the nominal size of the stitched fingerprint images.

[0094] In some implementations, if the multi-frame fingerprint images are acquired by the user pressing the fingerprint sensor multiple times, the following fingerprint image deduplication method is adopted: the multi-frame fingerprint images can be regarded as discrete fingerprint images, that is, there may be no similar feature point pairs between two fingerprint images acquired at adjacent times. When deduplicating the multi-frame discrete fingerprint images, one fingerprint image can be randomly selected as a reference fingerprint image, and then its features are compared with other fingerprint images to determine the number of similar feature point pairs between the comparison fingerprint image and the reference fingerprint image. If the number of similar feature point pairs between the nth frame comparison fingerprint image and the reference fingerprint image meets the preset deduplication filtering conditions, the nth frame comparison fingerprint image is determined as a new reference fingerprint image. The new reference fingerprint image is compared with other fingerprint images, and the above process is repeated until there are no fingerprint images in the multi-frame fingerprint images that meet the preset deduplication filtering conditions, or the number of reference fingerprint images determined in the multi-frame fingerprint images is greater than or equal to the preset number of frames, and the reference fingerprint image is determined as a valid fingerprint image.

[0095] In some implementations, if the number of valid fingerprint images is less than a preset number, the process returns to step S701. The watch then displays a message in the settings interface prompting the user to press or slide the fingerprint sensor again. Steps S701 to S704 are repeated until the preset number of valid fingerprint images is obtained, or the fingerprint registration process terminates when the number of repetitions reaches a repetition threshold. The number of repetitions represents the number of times the user's finger presses or slides on the fingerprint sensor during the fingerprint registration process. If the number of repetitions exceeds the repetition threshold, the fingerprint registration process is terminated to ensure the user experience, and the user is prompted to clean the fingerprint sensor's encapsulation surface and keep their fingers clean.

[0096] S705: The watch stitches together multiple valid fingerprint images to obtain a stitched fingerprint image.

[0097] In some implementations, the watch selects two valid fingerprint images with similar feature point pairs from multiple valid fingerprint images for fingerprint image stitching. Based on the coordinate positions of the similar feature point pairs in their respective valid fingerprint images, and using one valid fingerprint image as a reference, the translation and rotation parameters of the other valid fingerprint image are determined. The two valid fingerprint images are then stitched together according to these parameters. If the valid fingerprint images contain both clear and blurred areas, and the size and position of the clear and blurred areas in the two valid fingerprint images being stitched are different, the clear area of ​​one valid fingerprint image may overlap with the blurred area of ​​another valid fingerprint image during fingerprint image stitching. In the overlapping area, if one fingerprint image is clear and the other is blurred, the clear fingerprint image will be overlaid on the blurred fingerprint image in the overlapping area. Figure 10 As shown, fingerprint images 11, 12, 13, and 14 are stitched together to obtain a stitched fingerprint image. In the stitched fingerprint image, the overlapping area of ​​fingerprint images 11 and 12 is covered by fingerprint image 12, the overlapping area of ​​fingerprint images 12 and 13 is covered by fingerprint image 13, and the overlapping area of ​​fingerprint images 13 and 14 is covered by fingerprint image 14. It is known that the operation of one fingerprint image overlaying another is a common image processing method. For example, it could involve removing the texture features of blurred fingerprint images in the overlapping area. This embodiment does not provide a detailed description of how to overlay one fingerprint image onto another.

[0098] In some implementations, the overlapping region is divided into several overlapping sub-regions. The fingerprint images superimposed on each overlapping sub-region are evaluated. If, in a certain overlapping sub-region, one fingerprint image is clear while the other is blurry, then in that overlapping sub-region, the clear fingerprint image covers the blurry one. In other words, the coverage relationship of the fingerprint images participating in image stitching is determined only within the overlapping sub-regions (determining which fingerprint image covers and which is covered). For example, if in the first sub-region of the overlapping region, the first fingerprint image is clear and the second fingerprint image is blurry, and in the second sub-region of the overlapping region, the first fingerprint image is blurry and the second fingerprint image is clear, then during fingerprint image stitching, in the first sub-region, the first fingerprint image covers the second fingerprint image, and in the second sub-region, the second fingerprint image covers the first fingerprint image.

[0099] S706 The watch performs a deblurring operation on the stitched fingerprint image to obtain the target fingerprint image.

[0100] For example, deblurring removes fingerprint features from the blurred areas of the stitched fingerprint image, so that the fingerprint algorithm cannot recognize any fingerprint features in those areas when extracting fingerprint features from the target fingerprint image. Deblurring includes setting the grayscale value of pixels in the blurred areas of the stitched fingerprint image to a preset value, such as 255; that is, removing fingerprint features from the blurred areas of the stitched fingerprint image and turning them into blank areas. Figure 11 As shown, after deblurring the stitched fingerprint image, the blurred areas of the target fingerprint image become blank areas. In another example, the deblurring process can also be achieved by setting the pixels of the blurred areas of the fingerprint image to other grayscale values ​​to remove fingerprint features; this application does not impose any limitations on this.

[0101] S707: The watch extracts fingerprint features from the target fingerprint image to obtain the feature points of the target fingerprint image.

[0102] For example, feature extraction of a target fingerprint image by a watch includes extracting feature points from a binary image. In some implementations, for extracting feature points from a binary image, the target fingerprint image is binarized and thinned to obtain a binary image where the ridges are one pixel wide. Then, a 3×3 pixel template is used to scan each pixel of the binary image, calculating its crossing number (cn). The crossing number is used to determine whether it is a feature point and the type of feature point (e.g., end point or bifurcation point). The crossing number is calculated, for example, as the number of white pixels adjacent to other black pixels in the pixel template, excluding the center pixel, in the top, bottom, left, and right directions. Figure 12As shown, in Figure 12 In (a), a black pixel P is surrounded by a black pixel P1, and adjacent to P1 are two white pixels P2 and P8. Therefore, the intersection number is 2. Figure 12 In (b), a black pixel P is surrounded by three black pixels, P1, P5, and P7. The white pixels adjacent to P1 are P2 and P8, those adjacent to P5 are P4 and P6, and those adjacent to P7 are P6 and P8. Therefore, the intersection number is 6. Based on the intersection number, we can determine whether a pixel is a feature point. If it is, we determine the type of feature point. When the intersection number is 2 or 6, the pixel is determined to be a feature point; otherwise, it is determined to be a normal point. When the intersection number is 2, the feature point type is an end point; when the intersection number is 6, the feature point type is a bifurcation point. Therefore, we can determine... Figure 12 In (a), the black pixel P is a feature point, and the type of the feature point is an end point. Figure 12 In (b), the black pixel P is a feature point, and the type of feature point is a bifurcation point. All feature points of the target fingerprint image are obtained by traversing all pixels in the binary image using a pixel template.

[0103] S708: The watch generates a registration fingerprint template based on the fingerprint feature points of the target fingerprint image.

[0104] For example, after obtaining the fingerprint feature points of the target fingerprint image, these feature points are stored in a feature point set, which can also be called a registered fingerprint template. The registered fingerprint template is then stored in a database.

[0105] This application provides a fingerprint image processing method. In the fingerprint registration process, image data analysis is performed on fingerprint images to determine the clear and blurred regions of each frame. Multiple fingerprint images are then stitched together using these clear and blurred regions. When clear and blurred regions overlap, the clear regions are retained. After obtaining the stitched fingerprint image, the blurred regions are discarded, resulting in a clear target fingerprint image. Feature extraction from the clear target fingerprint image yields high-confidence feature points. Using these high-confidence feature points for fingerprint recognition can solve the problem of reduced security caused by using partially blurred fingerprint images.

[0106] In some implementations, such as Figure 13As shown, during the fingerprint registration process, when a user's finger operates on the fingerprint sensor (including pressing or swiping), the watch acquires multiple frames of fingerprint images. When the watch detects that the user's finger has left the fingerprint sensor, it records one operation of the user's finger on the fingerprint sensor and increments the operation count by 1. The watch judges the operation count based on a preset count, which represents the maximum number of times the user's finger operates on the fingerprint sensor during the fingerprint registration process.

[0107] If the number of operations is less than the preset number, the watch determines the registration type based on the time it takes for the user's finger to touch and leave the fingerprint sensor. If the time is less than or equal to the preset duration, the watch classifies it as a press registration; if it is greater than or equal to the preset duration, the watch classifies it as a swipe registration. The watch generates registration fingerprint templates for both press and swipe registrations. If the number of registration fingerprint templates is greater than or equal to the preset number, the fingerprint registration process ends. If the number is less than the preset number, the watch displays a prompt on the screen asking the user to collect their fingerprint again. The fingerprint sensor repeats this process whenever it detects the user's finger touching the sensor. If the number of operations equals the preset number, the fingerprint registration process ends.

[0108] This application provides a fingerprint image processing method. In the fingerprint registration process, a registered fingerprint template can be generated whether the user presses their finger on the fingerprint sensor or slides their finger on the fingerprint sensor. When the number of registered fingerprint templates reaches a preset number or the number of user operations reaches a preset number, the fingerprint registration process ends. By limiting the number of registered fingerprint templates and the number of operations, a larger fingerprint recognition area can be provided in the fingerprint recognition process while avoiding a decline in user experience due to excessive user operations (pressing or sliding).

[0109] like Figure 14 As shown, in some embodiments, the fingerprint image processing method provided in this application can also be applied to a fingerprint recognition process. Taking a watch as an example, the fingerprint registration process on the watch is described. For example, the fingerprint authentication process includes S801 to S804, wherein:

[0110] S801, the watch acquires fingerprint images.

[0111] In some implementations, after the fingerprint sensor is activated, the process of acquiring a fingerprint image, as described in step S702 above, is executed. This will not be elaborated further here.

[0112] S802: The watch performs image data analysis on the fingerprint image to determine the clear and blurry areas of the fingerprint image.

[0113] In some implementations, the watch performs image data analysis on the fingerprint image, executing the image data analysis process as described in step S703 above. This identifies the clear and blurred areas of the fingerprint image.

[0114] S803: The watch extracts fingerprint features from the fingerprint image, obtains feature points, and sets confidence levels for the feature points.

[0115] In some implementations, the watch extracts features from the fingerprint image to obtain feature points, and sets a confidence level for the feature points based on the region where the feature points are located. For example, if the feature point is located in a clear region, the confidence level is high, and if the feature point is located in a blurry region, the confidence level is low. When performing fingerprint matching, feature points with high confidence are matched first.

[0116] Feature extraction of fingerprint images can be performed using a feature extraction process similar to step S708 described above.

[0117] S804. The watch generates an input fingerprint template based on fingerprint feature points, and performs similarity matching between the input fingerprint template and the registered fingerprint template to obtain the fingerprint recognition result.

[0118] For example, during similarity matching, feature points with high confidence are preferentially matched with feature points of the registered fingerprint template to obtain a similarity score between the input fingerprint template and the registered fingerprint template. A preset matching condition can be set, including: if the similarity score is greater than or equal to a preset similarity score threshold, the fingerprint recognition result is considered successful. If the similarity score is less than the preset similarity score threshold, feature points with low confidence are matched with feature points of the registered fingerprint template. If the similarity score is greater than or equal to the preset similarity score threshold, the fingerprint recognition result is considered successful; otherwise, the fingerprint recognition result is considered unsuccessful. Here, the similarity score represents the similarity between the input fingerprint template and the registered fingerprint template. A higher similarity score indicates a greater similarity between the input and registered fingerprint templates. When the similarity score is greater than or equal to the preset similarity score threshold, the watch recognizes that the user's own finger is performing fingerprint recognition and executes the action corresponding to the user's operation.

[0119] For example, different weights can be set according to different confidence levels. The weight of feature points with high confidence is greater than that of feature points with low confidence. When matching feature points with high confidence with feature points of the registered fingerprint template, the first similarity score is obtained. The first similarity score is obtained by weighting and summing the first similarity score with the second similarity score obtained when matching feature points with low confidence with feature points of the registered fingerprint template, and the total similarity score is obtained.

[0120] In some implementations, if the fingerprint recognition result is a failure, the watch displays a fingerprint recognition failure message on the screen and returns to step S802 to continue executing the above steps. If the fingerprint recognition result is a success, the fingerprint image corresponding to the input fingerprint template is stitched together with the fingerprint image corresponding to the registered fingerprint template. If the clear area of ​​the fingerprint image corresponding to the input fingerprint template overlaps with the blank area of ​​the fingerprint image corresponding to the registered fingerprint template, the feature points of the clear area of ​​the fingerprint image corresponding to the input fingerprint template are added to the registered fingerprint template. The specific image stitching process is as shown in the above embodiments and will not be repeated here.

[0121] This application provides a fingerprint image processing method. In the fingerprint authentication process, image data analysis is performed on the fingerprint image to determine the clear and blurry regions. Fingerprint features are extracted using both clear and blurry regions. Different confidence levels are set for feature points in the clear and blurry regions, with the confidence level of feature points in the clear region being higher than that in the blurry region. An input fingerprint template is obtained based on the feature points. The input fingerprint template is then matched with registered fingerprint templates in the database for fingerprint similarity. Based on the confidence level of the feature points, feature points with higher confidence are preferentially selected for similarity matching with feature points in the registered fingerprint template to obtain the fingerprint authentication result. If the fingerprint authentication result indicates successful fingerprint recognition, the fingerprint image corresponding to the input fingerprint template is stitched together with the fingerprint image corresponding to the registered fingerprint template. If the clear region of the fingerprint image corresponding to the input fingerprint template overlaps with the blank region of the fingerprint image corresponding to the registered fingerprint template, the feature points of the clear region of the fingerprint image corresponding to the input fingerprint template are added to the registered fingerprint template. The fingerprint image processing method provided in this application embodiment can process fingerprint images with blurred areas, enabling them to use existing fingerprint algorithms to extract and match fingerprint images. If a match is successful, the registered fingerprint template is updated by adding high-confidence feature points to the registered fingerprint template, thereby improving the security and reliability of fingerprint recognition.

[0122] In this application embodiment, the terminal device with a curved frame design can be a wearable device (such as a smartwatch or smart bracelet), a mobile phone, a tablet computer, a handheld computer, a smart home device (such as a television), an in-vehicle system (such as an in-vehicle computer), a smart screen, a game console, and an augmented reality (AR) / virtual reality (VR) device, etc. This application embodiment does not impose any special limitations on the specific device form of the terminal device.

[0123] For example, let's take a watch with a curved bezel design as an example. Figure 15 A schematic diagram of the structure of a terminal device provided in an embodiment of this application is shown.

[0124] like Figure 15 As shown, the terminal device may include a processor 410, an external memory interface 420, an internal memory 421, a power interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, a wireless communication module 460, an audio module 470, a sensor module 480, a button 490, a motor 491, an indicator 492, a display screen 494, etc. Among them, the sensor module 480 may include a pressure sensor 480A, a gyroscope sensor 480B, a barometric pressure sensor 480C, an accelerometer 480D, a fingerprint sensor 480E (such as an ultrasonic fingerprint recognition module, an optical fingerprint recognition module, and a capacitive fingerprint recognition module, etc.), a touch sensor 480F, etc.

[0125] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the terminal device. In other embodiments, the terminal device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0126] Processor 410 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0127] The controller can serve as the nerve center and command center of a terminal device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0128] The processor 410 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. This memory can store instructions or data that the processor 410 has just used or that are used repeatedly. If the processor 410 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0129] In some embodiments, the processor 410 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0130] The wireless communication function of the terminal device can be implemented through antenna 1, wireless communication module 460, modem processor and baseband processor, etc.

[0131] The terminal device implements display functions through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 494 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0132] Display screen 494 is used to display images, videos, etc. Display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device may include one or N displays 494, where N is a positive integer greater than 1.

[0133] Internal memory 421 can be used to store computer executable program code, which includes instructions. Processor 410 executes various functional applications and data processing of the terminal device by running the instructions stored in internal memory 421. Internal memory 421 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the terminal device (such as audio data, phonebook, etc.). Furthermore, internal memory 421 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0134] The fingerprint sensor 480E is used to collect capacitive signals when a user's finger presses on the fingerprint sensor or slides on the fingerprint sensor. In this embodiment, a capacitive fingerprint recognition module (also known as a capacitive fingerprint sensor) is used, but other modules such as optical fingerprint recognition modules (also known as optical fingerprint sensors) and ultrasonic fingerprint recognition modules (also known as ultrasonic fingerprint sensors) are also applicable, and will not be described in detail here.

[0135] Corresponding to the methods in the foregoing embodiments, this application also provides an image processing apparatus. This apparatus can be applied to the aforementioned terminal device to implement the methods in the foregoing embodiments. The functions of this apparatus can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. For example, the apparatus includes: an acquisition module, a processing module, and a recognition module, etc. The acquisition module, processing module, and recognition module can cooperate to implement the related methods in the foregoing embodiments.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0137] In another implementation, the unit implementing each step of the above method can be configured as one or more processing elements, which can be disposed on the corresponding electronic device. These processing elements can be integrated circuits, such as one or more ASICs, one or more DSPs, or one or more FPGAs, or combinations of these types of integrated circuits. These integrated circuits can be integrated together to form a chip.

[0138] For example, this application also provides a chip system that can be applied to the aforementioned terminal device. The chip system includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via lines; the processor receives and executes computer instructions from the electronic device's memory through the interface circuits to implement the methods related to the terminal device in the above method embodiments.

[0139] This application also provides a computer program product, including a terminal device, such as computer instructions executed by the terminal device described above.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to a terminal device, the terminal device including a curved-surface packaged fingerprint sensor, the method includes: Receives a single press or swipe operation from the user on the fingerprint sensor; Acquire a first fingerprint image and a second fingerprint image during the pressing or sliding operation; the first fingerprint image includes a first region, the second fingerprint image includes a second region, and the images in the first and second regions are blurred; The first fingerprint image and the second fingerprint image are stitched together according to the fingerprint shape to obtain the stitched fingerprint image; Remove the blurred image from the stitched fingerprint image to obtain a third fingerprint image; Fingerprint feature extraction is performed on the third fingerprint image to obtain the feature points of the third fingerprint image; A registration fingerprint template is generated based on the feature points of the third fingerprint image; The step of extracting fingerprint features from the third fingerprint image to obtain feature points of the third fingerprint image includes: The third fingerprint image is binarized to obtain a binary image; the ridges in the binary image are one pixel wide. The binary image is scanned using a pixel template, and the number of intersections in the binary image is calculated. The number of intersections in the binary image is the sum of the number of intersections obtained by scanning the pixels using the pixel template each time. The number of intersections obtained by scanning the pixels using the pixel template each time includes the number of white pixels adjacent to the black pixels in the pixel template corresponding to the current pixel scanning process, excluding the center pixel. The feature points and their types for the black pixels in each scanning process are determined based on the number of intersections obtained during each pixel scanning process; the types of feature points include end points and branching points.

2. The method according to claim 1, characterized in that, The step of stitching the first fingerprint image and the second fingerprint image together according to the fingerprint shape includes: Feature points are obtained from the first fingerprint image and the second fingerprint image respectively, and the feature points are used to characterize the fingerprint shape; The first fingerprint image and the second fingerprint image are stitched together so that similar feature points in the first fingerprint image and the second fingerprint image are stitched together.

3. The method according to claim 2, characterized in that, The first fingerprint image includes a third region, wherein the image in the third region is clear; After stitching the first fingerprint image and the second fingerprint image together, the first sub-region in the third region overlaps with the second sub-region in the second region. The area in the stitched fingerprint image where the third region overlaps with the second region uses the image of the first sub-region.

4. The method according to any one of claims 1-3, characterized in that, The removal of blurred images from the stitched fingerprint image includes: Set the grayscale value of the pixels in the blurred area of ​​the stitched fingerprint image to a preset value.

5. The method according to any one of claims 1-3, characterized in that, The image is blurred, including: The number of gray levels in the image is less than or equal to the first judgment threshold; or, The contrast between light and dark areas of the image is less than the second judgment threshold.

6. The method according to any one of claims 1-3, characterized in that, The acquisition of the first fingerprint image and the second fingerprint image during the single press or slide operation includes: Acquire multiple frames of fingerprint images during the single press or slide operation; The first fingerprint image and the second fingerprint image are determined based on the multi-frame fingerprint images; the first fingerprint image and the second fingerprint image are fingerprint images in the multi-frame fingerprint images whose similar feature point pairs satisfy the first screening condition.

7. The method according to claim 6, characterized in that, The similar feature point pairs that satisfy the first screening condition include: The number of similar feature point pairs is less than or equal to the maximum value of the judgment threshold interval, and greater than or equal to the minimum value of the preset judgment threshold interval.

8. The method according to any one of claims 1-3, characterized in that, The method further includes: In response to the user's fingerprint authentication operation, a fourth fingerprint image is acquired; the fourth fingerprint image includes a fourth region and a fifth region, wherein the image of the fourth region is clear and the image of the fifth region is blurry; A first similarity score is obtained by matching the feature points in the fourth region with the registered fingerprint template. If the first similarity score meets the preset matching conditions, the fingerprint authentication is confirmed to be successful.

9. The method according to claim 8, characterized in that, The method further includes: If the first similarity score does not meet the preset matching conditions, a second similarity score is obtained by matching the feature points in the fifth region with the registered fingerprint template based on their similarity. The target similarity score is obtained by weighted summation of the first similarity score and the second similarity score; wherein the weight of the first similarity score is greater than the weight of the second similarity score. If the target similarity score meets the preset matching conditions, the fingerprint authentication is confirmed to be successful.

10. The method according to claim 8, characterized in that, After successful fingerprint authentication, the method further includes: The registered fingerprint template is updated based on the feature points within the fourth region.

11. A terminal device, characterized in that, The terminal device includes: a processor and a memory, the processor being coupled to the memory; the memory being used to store computer program code; the computer program code including computer instructions, which, when executed by the processor, cause the terminal device to perform the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-10.

13. A computer program product, characterized in that, When the computer program product is run on a terminal device, it causes the terminal device to perform the method described in any one of claims 1-10 as described above.

Images