Fingerprint sensor water drop image judgment method and system

Abstract

This invention discloses a method and system for judging water droplet images from a fingerprint sensor. The judgment method includes: S1. acquiring an image collected by a fingerprint sensor; S2. performing expansion and scaling processing on the image; S3. calculating the block mean and block variance of each image block; S4. if the foreground image block is greater than or equal to a preset foreground / background threshold, then proceeding to step S5; S5. calculating the difference between the variances of two adjacent blocks and determining whether the difference is greater than a preset difference threshold; S6. obtaining the foreground region based on the boundary region and determining whether there are a large number of blank areas. If so, the current image is a water droplet image; if not, detecting whether the foreground region is continuous. If not, the current image is a water droplet image; if so, detecting whether the texture in the foreground region is smooth. If not, the current image is a water droplet image; S7. determining whether the number of abnormal blocks is greater than a preset mean threshold. If so, the current image is a water droplet image; if not, the current image is a fingerprint image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for judging water droplet images from a fingerprint sensor. Background Technology

[0002] As fingerprint recognition technology matures, the application scenarios for fingerprint recognition are also increasing. Nowadays, there are many scenarios where fingerprint recognition is affected by water droplets, which can cause background images with water droplets to be collected for fingerprint recognition, affecting the actual user experience of the fingerprint recognition device.

[0003] Currently, several technical solutions have attempted to address this problem. For example, patent CN113378797A discloses a method for detecting water droplets on a fingerprint sensor. This method calculates and determines the grayscale difference between pixels in adjacent columns of all rows of the fingerprint image to identify abnormal pixels and counts the total number of abnormal pixels. When the total number of abnormal pixels exceeds a certain threshold, it can be determined whether there are water droplets on the fingerprint sensor. Although the aforementioned patent has made some progress, it mainly relies on grayscale difference, morphological operations, and shape judgment of specific regions in the intermediate image for detection. While these methods can reflect some image features to a certain extent, they fail to fully analyze and utilize local image features, such as the mean and variance of image patches. In fact, there are significant differences in the mean and variance of local image patches between normal fingerprint images and water droplet images. Fully utilizing these differences for detection would help improve the accuracy and reliability of the detection.

[0004] Therefore, in view of the above-mentioned technical problems, the present invention provides a method and system for judging water droplet images of fingerprint sensors. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a method and system for judging water droplet images in fingerprint sensors. This method mainly calculates the mean and variance of image blocks in the image, and quickly judges normal fingerprint images and water droplet images based on the characteristics of the mean and variance of normal fingerprint images and water droplet images, thereby filtering out water droplet images and protecting the normal use of fingerprint sensors in scenarios with water droplet interference.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for determining water droplet images from a fingerprint sensor includes:

[0008] Step S1. Acquire the image captured by the fingerprint sensor;

[0009] Step S2. Expand and scale the acquired image to obtain a scaled image;

[0010] Step S3. Divide the scaled image into several image blocks, calculate the block mean and block variance of each image block, compare the calculated block variance with the variance threshold, determine whether the current image block belongs to the foreground image block or the background image block, and store all the determined foreground image blocks and background image blocks in the array pDir in order.

[0011] Step S4. Calculate the proportion of the number of foreground image blocks stored in the array pDir to the entire array pDir, and compare the proportion of the foreground image blocks with the preset foreground and background threshold. If the number of foreground image blocks is less than the preset foreground and background threshold, it means that the current image is a water droplet image; if the number of foreground image blocks is greater than or equal to the preset foreground and background threshold, then continue to step S5.

[0012] Step S5. Calculate the difference between the variances of two adjacent blocks, and determine whether the calculated difference is greater than the preset difference threshold. If so, it means that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and save the boundary region.

[0013] Step S6. Obtain the foreground region based on the saved boundary region, and determine whether there are a large number of blank areas in the foreground region. If yes, it means that the current image is a water droplet image; if no, check whether the foreground region is continuous. If no, it means that the current image is a water droplet image; if yes, check whether the texture in the foreground region is smooth. If no, it means that the current image is a water droplet image; if yes, continue to execute step S7.

[0014] Step S7. Calculate the number of abnormal blocks in all image blocks within and outside the foreground region, and determine whether the number of abnormal blocks is greater than the preset average threshold. If yes, the current image is a water droplet image; otherwise, the current image is a fingerprint image.

[0015] Furthermore, step S2 also includes performing Gaussian filtering and contrast enhancement on the scaled image.

[0016] Furthermore, after calculating the block mean and block variance of each image block in step S3, the method further includes storing the block mean in an array pMean and storing the block variance in an array pVar.

[0017] Furthermore, the calculation of the difference in variance between two adjacent blocks in step S5 includes: calculating the difference in variance between two adjacent blocks in the image row direction and the vertical direction, respectively.

[0018] Furthermore, the step S6 of determining whether there are a large number of blank areas within the foreground region specifically involves:

[0019] The foreground region is obtained based on the saved boundary region, and the number of all image blocks within the foreground region is calculated.

[0020] Calculate the number of image blocks with variance of 0 among all image blocks in the foreground region. If the proportion of the number of image blocks with variance of 0 in the total number of image blocks in the foreground region is greater than a preset threshold, then the current image is a water droplet image.

[0021] Furthermore, the step S6 of detecting whether the foreground region is continuous specifically involves:

[0022] Calculate the length between the left and right boundaries of each row in the row direction of the image, and store the length between the left and right boundaries in the array x_len. Count the number of rows in the array x_len whose length is less than the preset row length threshold num_x.

[0023] Calculate the height between the upper and lower boundaries of each column in the vertical direction of the image, and store the length between the upper and lower boundaries in the array y_len. Count the number of columns in the array y_len whose height is less than the preset column height threshold num_y.

[0024] Count the number of rows whose left and right boundaries are not equal, x_num; count the number of columns whose upper and lower boundaries are not equal, y_num.

[0025] Calculate the percentage of rows that are too short based on the number of rows num_x and the number of rows x_num; calculate the percentage of columns that are too short based on the number of columns num_y and the number of columns y_num.

[0026] Determine whether the proportion of excessively short rows or columns exceeds a preset proportion threshold. If so, the current image is a water droplet image.

[0027] Furthermore, the step S6 of detecting whether the texture in the foreground region is smooth specifically involves:

[0028] Calculate the number of rows in which the difference between the left and right column block variances exceeds a preset difference threshold, store them in array X, and compile a histogram of row variance mutations (hist_x).

[0029] Calculate the number of columns in each column whose difference between the upper and lower rows exceeds a preset difference threshold, store the number of columns in array Y, and compile a histogram of column variance mutations (hist_y).

[0030] If the sum of the number of rows in array X with variance mutations greater than 2 is greater than the proportion of the number of rows x_num, or if the sum of the number of columns in array Y with variance mutations greater than 2 is greater than the proportion of the number of columns y_num, then the current image is a water droplet image.

[0031] Furthermore, the calculation of the number of outlier blocks in all image patches outside the foreground region in step S7 specifically involves:

[0032] Determine whether the average value of the image block outside the boundary of the foreground region is less than the first preset average value threshold. If so, the current image block is an abnormal block.

[0033] Furthermore, the calculation of the number of abnormal blocks in all image blocks within the foreground region in step S7 specifically involves:

[0034] Determine whether the average value of the image block within the boundary of the foreground region is greater than the second preset average value threshold. If so, the current image block is an abnormal block.

[0035] Correspondingly, a fingerprint sensor water droplet image recognition system is also provided, including:

[0036] The acquisition module is used to acquire images captured by the fingerprint sensor;

[0037] The processing module is used to expand and scale the acquired image to obtain a scaled image;

[0038] The first calculation module is used to divide the scaled image into several image blocks, calculate the block mean and block variance of each image block, compare the calculated block variance with the variance threshold, determine whether the current image block belongs to the foreground image block or the background image block, and store all the determined foreground image blocks and background image blocks in the array pDir in order.

[0039] The second calculation module is used to calculate the proportion of the number of foreground image blocks stored in the array pDir in the entire array pDir, and compare the proportion of foreground image blocks with a preset foreground and background threshold. If the foreground image block is less than the preset foreground and background threshold, it means that the current image is a water droplet image; if the foreground image block is greater than or equal to the preset foreground and background threshold, the process continues.

[0040] The third calculation module is used to calculate the difference between the variances of two adjacent blocks, determine whether the calculated difference is greater than the preset difference threshold, and if so, it indicates that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and saves the boundary region.

[0041] The judgment module is used to obtain the foreground region based on the saved boundary region and determine whether there are a large number of blank areas in the foreground region. If so, it indicates that the current image is a water droplet image; if not, it checks whether the foreground region is continuous. If not, it indicates that the current image is a water droplet image; if so, it checks whether the texture in the foreground region is smooth. If not, it indicates that the current image is a water droplet image; if so, it continues to execute.

[0042] The fourth calculation module is used to calculate the number of abnormal blocks in all image blocks within and outside the foreground region, and to determine whether the number of abnormal blocks is greater than a preset average threshold. If so, the current image is a water droplet image; otherwise, the current image is a fingerprint image.

[0043] Compared with existing technologies, this invention solves the problem of false triggering of water droplet images when fingerprint sensors are used in actual outdoor scenarios, thereby improving the user experience and extending the lifespan of the device. This method reduces computation by shrinking the image size and utilizes the significant differences in the mean and variance of local image blocks between normal fingerprint images and water droplet images to reasonably set a control threshold. This allows for rapid water droplet image detection and filtering without affecting the display of normal fingerprint images, thus improving the user experience. Attached Figure Description

[0044] Figure 1 This is a flowchart of a fingerprint sensor water droplet image judgment method provided in Embodiment 1;

[0045] Figure 2 This is a schematic diagram of a normal fingerprint image provided in Example 1;

[0046] Figure 3 This is a schematic diagram of a water droplet image provided in Example 1;

[0047] Figure 4 This is a schematic diagram of the extended normal fingerprint image provided in Embodiment 1;

[0048] Figure 5 This is a schematic diagram of a scaled-down normal fingerprint image provided in Embodiment 1;

[0049] Figure 6 This is a schematic diagram of a normal fingerprint image after image processing provided in Embodiment 1. Detailed Implementation

[0050] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0051] The purpose of this invention is to address the shortcomings of existing technologies by providing a method and system for judging water droplet images from fingerprint sensors.

[0052] Example 1

[0053] This embodiment provides a method for judging water droplet images from a fingerprint sensor, such as... Figure 1 As shown, it includes:

[0054] S1. Acquire the image captured by the fingerprint sensor;

[0055] S2. Expand and scale the acquired image to obtain a scaled image;

[0056] S3. Divide the scaled image into several image blocks, calculate the block mean and block variance of each image block, compare the calculated block variance with the variance threshold, determine whether the current image block belongs to the foreground image block or the background image block, and store all the determined foreground image blocks and background image blocks in the array pDir in order.

[0057] S4. Calculate the proportion of the number of foreground image blocks stored in the array pDir to the entire array pDir, and compare the proportion of foreground image blocks with the preset foreground and background threshold. If the number of foreground image blocks is less than the preset foreground and background threshold, it means that the current image is a water droplet image; if the number of foreground image blocks is greater than or equal to the preset foreground and background threshold, continue to execute step S5.

[0058] S5. Calculate the difference between the variances of two adjacent blocks, and determine whether the calculated difference is greater than the preset difference threshold. If so, it means that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and save the boundary region.

[0059] S6. Obtain the foreground region based on the saved boundary region, and determine whether there are a large number of blank areas in the foreground region. If so, it means that the current image is a water droplet image; if not, check whether the foreground region is continuous. If not, it means that the current image is a water droplet image; if so, check whether the texture in the foreground region is smooth. If not, it means that the current image is a water droplet image; if so, continue to execute step S7.

[0060] S7. Calculate the number of abnormal blocks in all image blocks within and outside the foreground region, and determine whether the number of abnormal blocks is greater than the preset average threshold. If yes, the current image is a water droplet image; otherwise, the current image is a fingerprint image.

[0061] In step S1, the image captured by the fingerprint sensor is obtained.

[0062] This embodiment assumes that the acquired image size is 208*288 as an example for illustration. Figure 2 A normal fingerprint image, such as Figure 3 This is an image of a water droplet.

[0063] In step S2, the acquired image is expanded and scaled to obtain a scaled image.

[0064] by Figure 2 Taking a normal fingerprint image as an example, the image with a size of 208*288 is expanded to... Figure 4 The image shown is 256*288 in size. In order to be as similar as possible to the background, the expanded part is filled with a value close to 255. The main purpose of this expansion is to make the foreground area of ​​the image have a clear boundary with the background area, which is beneficial to the subsequent water droplet image judgment.

[0065] In embedded devices, due to limited computing power, it is desirable to minimize the amount of computation during algorithm execution. Therefore, in this embodiment, the image is divided into 4x4 regions, with one pixel taken from every 16 pixels in each region, and the image is scaled down to 1 / 16 of the original image. Figure 5 The image is reduced in size. Figure 5 The size is 64*72. This operation can greatly reduce the computational load of the algorithm and improve the running speed of the algorithm. Note that in this step, the original image is not destroyed. The reduced image will be stored in a block of memory that is 1 / 16 the size of the original image.

[0066] Next, this embodiment processes the scaled image, such as... Figure 6 As shown, the processing of the scaled image mainly consists of two parts. The first is to use Gaussian filtering to remove noise from the image and make the overall image smoother. The second is to enhance the contrast of the Gaussian-filtered image. Its main purpose is to improve the local contrast of the image. For normal fingerprint images, the increase in contrast will make the ridges and valleys in the fingerprint image more distinct, increase the local variance, and make the normal fingerprint image more recognizable.

[0067] In step S3, the scaled image is divided into several image blocks, the block mean and block variance of each image block are calculated, the calculated block variance is compared with the variance threshold, it is determined whether the current image block belongs to the foreground image block or the background image block, and all the foreground image blocks and background image blocks are stored in the array pDir in order.

[0068] For the image processed in step S2, calculate the block mean and variance in 4*4 blocks, and store the block mean in array pMean and the block variance in array pVar.

[0069] In this embodiment, a pre-set variance threshold SegHold is used. This pre-set variance threshold SegHold can be obtained by averaging the variances of all blocks. Based on this pre-set variance threshold SegHold, it is determined whether a 4*4 image block belongs to the foreground or background image block. If the block variance is greater than the pre-set variance threshold SegHold, the image block is considered to be a foreground image block; if it is less than or equal to the pre-set variance threshold SegHold, the image block is considered to be a background image block. Finally, the foreground and background image blocks are stored in the array pDir in sequence.

[0070] The order is usually based on the position of the image blocks in the original image, such as from left to right or from top to bottom. Assuming the image is a two-dimensional array, the blocks in each row might be processed first in row-major order (from left to right), and then processed row by row from top to bottom. For example, for the first block in the first row of the image, after determining whether it is foreground or background, the result is stored in the first position of the pDir array; then the second block in the first row is processed and stored in the second position, and so on. After processing the first row, the second row is processed, and so on.

[0071] In step S4, the proportion of the number of foreground image blocks stored in array pDir in the entire array pDir is calculated, and the proportion of foreground image blocks is compared with a preset foreground and background threshold. If the number of foreground image blocks is less than the preset foreground and background threshold, it means that the current image is a water droplet image; if the number of foreground image blocks is greater than or equal to the preset foreground and background threshold, then step S5 is executed.

[0072] Based on the foreground image blocks determined in step S3, the total number of foreground image blocks is counted, denoted as n. Then, the total number of image blocks stored in the array pDir is counted, denoted as m. Next, the proportion of the number of foreground image blocks n in the total number of image blocks m is calculated, i.e., n / m. In this embodiment, for ease of viewing, the result is magnified by 100 times. Therefore, the foreground region proportion ranges from 0 to 100. A preset foreground / background threshold Threshold1 is used (set to 40 in this embodiment). For images where the foreground / background proportion (n / m) is less than the preset foreground / background threshold Threshold1, they are judged as abnormal fingerprint images and classified as water droplet images for filtering. This is because when the foreground proportion of an image is too small, the image will be considered an invalid image and needs to be filtered out. This is to quickly filter out a batch of unqualified images and reduce the amount of computation.

[0073] In step S5, the difference between the variances of two adjacent blocks is calculated, and it is determined whether the calculated difference is greater than a preset difference threshold. If so, it indicates that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and the boundary region is saved.

[0074] Based on the variance array pVar calculated in step S3, the boundary of the foreground region in the image is found according to the variance. Whether it is a water droplet image or a normal fingerprint image, since the background area is filled with white, there will definitely be a region with a sudden change in variance at its boundary. Therefore, the difference threshold Threshold2 between adjacent variance blocks can be preset in advance (it can be set according to the actual image, and is set to 15 in this embodiment).

[0075] In this embodiment, calculating the difference in variance between two adjacent blocks includes: calculating the difference in variance between two adjacent blocks in the image row direction and the vertical direction, respectively.

[0076] Find the boundary of the row direction:

[0077] For each row, the left and right boundaries are found based on the difference between the two variance blocks. Specifically, the difference between the two adjacent variance blocks is compared. If the difference is greater than Threshold2, then this position may be the boundary between the foreground and the background. A two-dimensional array row[num1][2] is used to store the left and right positions of the foreground region of the image. Taking the first row as an example, its left and right boundary position values ​​are stored in row[0][0] and row[0][1] respectively. num1 is determined according to the row height of the image, and the specific value is 1 / 16 of the row height.

[0078] Find the vertical boundary:

[0079] For each column, the upper and lower boundaries are found based on the difference between the upper and lower variance blocks. Specifically, the difference between two adjacent variance blocks is compared. If the difference is greater than Threshold2, then this position may be the boundary between the foreground and the background. A two-dimensional array col[num2][2] is used to store the upper and lower boundary positions in the vertical direction of the image. The value of num2 is determined according to the width of the image, and the specific value is 1 / 16 of the image width.

[0080] In step S6, the foreground region is obtained based on the saved boundary region, and it is determined whether there are a large number of blank areas in the foreground region. If so, the current image is a water droplet image; if not, it is detected whether the foreground region is continuous. If not, the current image is a water droplet image; if so, it is detected whether the texture in the foreground region is smooth. If not, the current image is a water droplet image; if so, step S7 is continued.

[0081] A significant difference between normal fingerprint images and water droplet images is that the foreground region of a normal fingerprint image is a continuous area, such as... Figure 2 As shown, in a normal fingerprint image, the fingerprint area is a continuous region in the middle without large gaps; however, in a water droplet image, the foreground area is generally not a unified whole due to the varying shapes of the water droplets. Figure 3As shown, there are some blank "holes" in the middle of the water droplet image. This is because the splashing of water droplets causes the water droplets to be unevenly distributed on the sensor surface. Based on this characteristic, this embodiment mainly judges the water droplet image from the variance features corresponding to the shapes of the two.

[0082] In this embodiment, determining whether there are a large number of blank areas within the foreground region specifically involves:

[0083] In a normal fingerprint image, the foreground region is a continuous area, while in a water droplet image, the foreground region has many small holes (because the water droplet is irregularly shaped and unevenly distributed).

[0084] First, based on the foreground region boundaries found in the horizontal and vertical boundaries of the image as counted in step S5, calculate the total number of all 4×4 image blocks within the boundary region, denoted as count; then calculate the number of regions with a variance of 0 within the boundary region, denoted as count0, where a variance of 0 means that the pixel values ​​within these blocks are the same, usually the background region (white fill); calculate the proportion of regions with a variance of 0 within the foreground region, count0 / count, and determine whether the proportion of regions with a variance of 0, count0 / count, is greater than the preset threshold Threshold3. Threshold3 ranges from 0% to 100%, and is set to 30% in this embodiment. If it is, then the image is determined to be a water droplet image, as normal fingerprint images do not have a high proportion of regions with a variance of 0 within the foreground region. If count0 / count is less than Threshold3, then the judgment continues.

[0085] In this embodiment, detecting whether the foreground region is continuous specifically involves:

[0086] In a normal fingerprint image, the foreground area is usually continuous in both the horizontal and vertical directions, while in a water droplet image, the foreground area has many breaks.

[0087] Calculate the lengths of the left and right boundaries of each row and store these lengths in the array x_len. Based on the array x_len, count the number of rows num_x whose row length is less than the preset row length threshold Threshold4, where the threshold Threshold4 is generally taken as 1 / 3 of the total row length.

[0088] Calculate the upper and lower boundary heights of each column and store these heights in the array y_len. Based on the array y_len, count the number of columns num_y whose column heights are less than the preset column height threshold Threshold5, where the threshold Threshold5 is generally taken as 1 / 3 of the column height.

[0089] Calculate the total number of rows x_num where the left and right boundaries are not equal. Rows with unequal left and right boundaries indicate that the row has foreground information.

[0090] Calculate the total number of columns y_num where the upper and lower boundaries are not equal. Columns with unequal upper and lower boundaries indicate that the column has foreground information.

[0091] The percentage of rows with excessively short effective regions (num_x / x_num) and the percentage of columns with excessively short effective regions (num_y / y_num) are counted. If the percentage of rows with excessively short effective regions or the percentage of columns with excessively short effective regions is greater than the preset percentage threshold Threshold6 (60% in this embodiment), then it is judged as a water droplet image; otherwise, the judgment continues.

[0092] In this embodiment, detecting whether the texture in the foreground area is smooth specifically involves:

[0093] The variance variation in the foreground region of a normal fingerprint image is relatively smooth, while the variance of a water droplet image has more abrupt change points due to its irregular shape.

[0094] Calculate the number of variance differences between the left and right columns of each row that are greater than a given preset difference threshold Threshold2 (i.e. variance mutation points), and store these numbers in array X. Based on array X, compile a histogram of the number of variance mutations in the rows (hist_x).

[0095] Calculate the number of variance abrupt changes in each column where the difference between the upper and lower variances exceeds a given preset difference threshold Threshold2, and store these numbers in array Y. Based on array Y, compile a histogram of the number of variance abrupt changes in each column.

[0096] If the sum of rows with variance mutations greater than 2 in the X (row) direction accounts for a proportion greater than the preset row threshold Threshold7, or if the sum of columns with variance mutations greater than 2 in the Y (column) direction accounts for a proportion greater than the preset column threshold Threshold8, then in this embodiment, both the row threshold Threshold7 and the column threshold Threshold8 are set to 70%, and the image is determined to be a water droplet image; otherwise, the determination process continues.

[0097] In step S7, the number of abnormal blocks in all image blocks within and outside the foreground region is calculated, and it is determined whether the number of abnormal blocks is greater than a preset average threshold. If yes, the current image is a water droplet image; otherwise, the current image is a fingerprint image.

[0098] In a normal fingerprint image, the area outside the boundary of the foreground region (fingerprint portion) is considered the background region. The image blocks in the background region have a larger gray value. However, this is not the case for water droplet images. Due to issues such as water droplet splashing, some abnormal regions may be created. That is, there may be image average blocks with smaller gray values ​​outside the boundary of the foreground region, which are represented as small black blocks in the image. There may be image average blocks with larger gray values ​​inside the boundary of the foreground region, which are similar to blank areas. Based on this difference, this embodiment determines whether it is a water droplet image according to the number of abnormal blocks in the image.

[0099] First, statistical analysis is performed on the average values ​​of image blocks in the fingerprint image and water droplet image from the sensor, i.e., the average value of each image block calculated in step S3. Then, a first preset average value threshold Threshold9 and a second preset average value threshold Threshold10 for judging blocks outside the foreground region boundary are determined. In this embodiment, Threshold9 is set to 130 and Threshold10 is set to 180.

[0100] Examine image blocks outside the foreground region boundary; if the block mean is lower than Threshold9, it is considered an abnormal block.

[0101] Examine image blocks within the foreground region boundary; if the block mean is higher than Threshold10, it is considered an anomalous block.

[0102] Calculate the total number of abnormal blocks inside and outside the foreground area, num_err. If the total number of abnormal blocks, num_err, is greater than the preset threshold for the number of abnormal blocks in the image, Threshold11 (set to 5 in this embodiment), then it is determined to be a water droplet image.

[0103] This embodiment addresses the issue of false triggering of water droplet images in practical fingerprint sensor applications, improving the user experience and extending device lifespan. The method reduces computation by shrinking the image size. It leverages the significant differences in mean and variance of local image blocks between normal fingerprint images and water droplet images to rationally set a control threshold. This allows for rapid water droplet image detection and filtering without affecting the display of the normal fingerprint image, thus improving the user experience. This method also prevents interference with normal fingerprints (such as fingerprint injuries or calluses) from being mistakenly identified as water droplet images and filtered out; only water droplet images are filtered.

[0104] It should be noted that this judgment method is used in the scenario where the fingerprint sensor is outdoors and there are water droplets on the collection head. After the sensor determines that it is a water droplet image, it can be directly filtered without further processing.

[0105] Example 2

[0106] The fingerprint sensor water droplet image judgment method provided in this embodiment differs from that in Embodiment 1 in that:

[0107] This embodiment compares the judgment method of this scheme with existing recognition schemes.

[0108] In practical fingerprint sensor applications, many scenarios take place outdoors. Even a drop of water falling onto the sensor surface can cause it to capture an image, known as a water droplet image. The fingerprint sensor operates by starting the identification process as soon as it captures an image. This embodiment avoids including water droplet images in the identification process because it would have significant negative impacts. Firstly, water droplet images would frequently cause identification failures and trigger alarms, severely affecting the user experience (e.g., a doorway identification system constantly alarming with verification failures). Secondly, frequent identification would increase device power consumption and shorten the device's lifespan.

[0109] The shape of a water droplet image on a fingerprint sensor is not fixed; it is affected by factors such as the size of the droplet, the height from which it falls, and its falling speed. Using a simple and universal method based on image grayscale and variance (hereinafter referred to as the traditional recognition method) for filtering results in low accuracy and cannot effectively address the issue of water droplet images entering the recognition process. This embodiment focuses on improving the filtering accuracy of water droplet images, maximizing the distinction between water droplet images and normal fingerprint images, and reducing false recognition. According to the experiment, a water droplet image library was tested. This library contains 543 water droplet images captured by a fingerprint sensor. If only the traditional recognition method is used, the output of step S4 is 327 / 543, meaning out of 543 water droplet images, 327 are correctly identified as water droplets, and 216 are identified as fingerprint images, resulting in an accuracy rate of only 60.22%. The output of step S6 is 419 / 543, meaning out of 543 water droplet images, 419 are correctly identified as water droplets, resulting in an accuracy rate of 77.16%. However, the output of step S7 is 543 / 543, meaning out of 543 water droplet images, 543 are correctly identified as water droplets, achieving an accuracy rate of 99.9%. This demonstrates that the proposed fingerprint sensor water droplet image judgment method improves the recognition accuracy, and the traditional recognition method cannot accurately prevent or stop water droplet images from activating the fingerprint sensor's identity recognition function.

[0110] Example 3

[0111] This embodiment provides a fingerprint sensor water droplet image recognition system, including:

[0112] The acquisition module is used to acquire images captured by the fingerprint sensor;

[0113] The processing module is used to expand and scale the acquired image to obtain a scaled image;

[0114] The first calculation module is used to divide the scaled image into several image blocks, calculate the block mean and block variance of each image block, compare the calculated block variance with the variance threshold, determine whether the current image block belongs to the foreground image block or the background image block, and store all the determined foreground image blocks and background image blocks in the array pDir in order.

[0115] The second calculation module is used to calculate the proportion of the number of foreground image blocks stored in the array pDir in the entire array pDir, and compare the proportion of foreground image blocks with a preset foreground and background threshold. If the foreground image block is less than the preset foreground and background threshold, it means that the current image is a water droplet image; if the foreground image block is greater than or equal to the preset foreground and background threshold, the process continues.

[0116] The third calculation module is used to calculate the difference between the variances of two adjacent blocks, determine whether the calculated difference is greater than the preset difference threshold, and if so, it indicates that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and saves the boundary region.

[0117] The judgment module is used to obtain the foreground region based on the saved boundary region and determine whether there are a large number of blank areas in the foreground region. If so, it indicates that the current image is a water droplet image; if not, it checks whether the foreground region is continuous. If not, it indicates that the current image is a water droplet image; if so, it checks whether the texture in the foreground region is smooth. If not, it indicates that the current image is a water droplet image; if so, it continues to execute.

[0118] The fourth calculation module is used to calculate the number of abnormal blocks in all image blocks within and outside the foreground region, and to determine whether the number of abnormal blocks is greater than a preset average threshold. If so, the current image is a water droplet image; otherwise, the current image is a fingerprint image.

[0119] It should be noted that the fingerprint sensor water droplet image judgment system provided in this embodiment is similar to that in Embodiment 1, and will not be described in detail here.

[0120] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for determining water droplet images from a fingerprint sensor, characterized in that, include: Step S1. Acquire the image captured by the fingerprint sensor; Step S2. Expand and scale the acquired image to obtain a scaled image; Step S3. Divide the scaled image into several image blocks, calculate the block mean and block variance of each image block, compare the calculated block variance with the variance threshold, determine whether the current image block belongs to the foreground image block or the background image block, and store all the determined foreground image blocks and background image blocks in the array pDir in order. Step S4. Calculate the proportion of the number of foreground image blocks stored in the array pDir to the total number of the array pDir, and compare the proportion of the foreground image blocks with a preset threshold. If the number of foreground image blocks is less than the preset threshold, it means that the current image is a water droplet image. If the foreground image block is greater than or equal to the preset threshold, then continue to step S5; Step S5. Calculate the difference between the variances of two adjacent blocks, and determine whether the calculated difference is greater than the preset difference threshold. If so, it means that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and save the boundary region. Step S6. Obtain the foreground region based on the saved boundary region, and determine whether there are a large number of blank areas in the foreground region. If yes, it means that the current image is a water droplet image; if no, check whether the foreground region is continuous. If no, it means that the current image is a water droplet image; if yes, check whether the texture in the foreground region is smooth. If no, it means that the current image is a water droplet image; if yes, continue to execute step S7. The step S6 of detecting whether the foreground region is continuous specifically involves: Calculate the length between the left and right boundaries of each row in the row direction of the image, and store the length between the left and right boundaries in the array x_len. Count the number of rows in the array x_len whose length is less than the preset row length threshold num_x. Calculate the height between the upper and lower boundaries of each column in the vertical direction of the image, and store the length between the upper and lower boundaries in the array y_len. Count the number of columns in the array y_len whose height is less than the preset column height threshold num_y. Count the number of rows whose left and right boundaries are not equal, x_num; count the number of columns whose upper and lower boundaries are not equal, y_num. Calculate the percentage of rows that are too short based on the number of rows num_x and the number of rows x_num; calculate the percentage of columns that are too short based on the number of columns num_y and the number of columns y_num. Determine whether the proportion of excessively short rows or columns exceeds a preset proportion threshold. If so, the current image is a water droplet image. Step S7. Calculate the number of abnormal blocks in all image blocks within and outside the foreground region, and determine whether the number of abnormal blocks is greater than the preset threshold for the number of abnormal blocks in the image. If yes, the current image is a water droplet image; if no, the current image is a fingerprint image.

2. The fingerprint sensor water droplet image judgment method according to claim 1, characterized in that, Step S2 also includes performing Gaussian filtering and contrast enhancement on the scaled image.

3. The fingerprint sensor water droplet image judgment method according to claim 1, characterized in that, After calculating the block mean and block variance of each image block in step S3, the method further includes storing the block mean in the array pMean and the block variance in the array pVar.

4. The fingerprint sensor water droplet image judgment method according to claim 1, characterized in that, The step S5 of calculating the difference in variance between two adjacent blocks includes: calculating the difference in variance between two adjacent blocks in the row direction and the vertical direction of the image, respectively.

5. The fingerprint sensor water droplet image judgment method according to claim 1, characterized in that, The specific steps in step S6, such as determining whether there are a large number of blank areas in the foreground region, are as follows: The foreground region is obtained based on the saved boundary region, and the number of all image blocks within the foreground region is calculated. Calculate the number of image blocks with variance of 0 among all image blocks in the foreground region. If the proportion of the number of image blocks with variance of 0 in the total number of image blocks in the foreground region is greater than a preset threshold, it indicates that there are a large number of blank areas in the foreground region.

6. The fingerprint sensor water droplet image judgment method according to claim 5, characterized in that, The step S6 of detecting whether the texture in the foreground area is smooth specifically involves: Calculate the number of rows in which the difference between the left and right column block variances exceeds a preset difference threshold, store them in array X, and compile a histogram of row variance mutations (hist_x). Calculate the number of columns in each column whose difference between the upper and lower rows exceeds a preset difference threshold, store the number of columns in array Y, and compile a histogram of column variance mutations (hist_y). If the sum of the number of rows in array X with variance mutations greater than 2 is greater than the proportion of the number of rows x_num, or if the sum of the number of columns in array Y with variance mutations greater than 2 is greater than the proportion of the number of columns y_num, then the current image is a water droplet image.

7. The fingerprint sensor water droplet image judgment method according to claim 1, characterized in that, The specific steps in step S7 for calculating the number of outlier blocks in all image patches outside the foreground region are as follows: Determine whether the average value of the image block outside the boundary of the foreground region is less than the first preset average value threshold. If so, the current image block is an abnormal block.

8. The fingerprint sensor water droplet image judgment method according to claim 1, characterized in that, The specific steps in step S7 for calculating the number of outlier blocks in all image patches within the foreground region are as follows: Determine whether the average value of the image block within the boundary of the foreground region is greater than the second preset average value threshold. If so, the current image block is an abnormal block.

9. A judgment system based on the fingerprint sensor water droplet image judgment method according to any one of claims 1-8, characterized in that, include: The acquisition module is used to acquire images captured by the fingerprint sensor; The processing module is used to expand and scale the acquired image to obtain a scaled image; The first calculation module is used to divide the scaled image into several image blocks, calculate the block mean and block variance of each image block, compare the calculated block variance with the variance threshold, determine whether the current image block belongs to the foreground image block or the background image block, and store all the determined foreground image blocks and background image blocks in the array pDir in order. The second calculation module is used to calculate the proportion of the number of foreground image blocks stored in the array pDir in the entire array pDir, and compare the proportion of foreground image blocks with a preset threshold. If the number of foreground image blocks is less than the preset threshold, it means that the current image is a water droplet image. If the foreground image patch is greater than or equal to the preset threshold, continue execution; The third calculation module is used to calculate the difference between the variances of two adjacent blocks, determine whether the calculated difference is greater than the preset difference threshold, and if so, it indicates that the image blocks corresponding to the variances of the two blocks are the boundary between the foreground image block and the background image block, and saves the boundary region. The judgment module is used to obtain the foreground region based on the saved boundary region and determine whether there are a large number of blank areas in the foreground region. If so, it indicates that the current image is a water droplet image; if not, it checks whether the foreground region is continuous. If not, it indicates that the current image is a water droplet image; if so, it checks whether the texture in the foreground region is smooth. If not, it indicates that the current image is a water droplet image; if so, it continues to execute. The fourth calculation module is used to calculate the number of abnormal blocks in all image blocks within and outside the foreground region, and to determine whether the number of abnormal blocks is greater than the preset threshold for the number of abnormal blocks in the image. If yes, it indicates that the current image is a water droplet image; otherwise, it indicates that the current image is a fingerprint image.

Images